-
Notifications
You must be signed in to change notification settings - Fork 1
Communicate log info #22
Comments
First, in my oppinion the feature would be really helpful, no question. But I have some doubts and remarks regarding prior design choices and performance. I will try to list my thoughts on that:
For the R UDF service this is quite impossible, because the plumber webservice just supports single threads, as R does in general unless certain packages are used. However, for data processing it is possible to paralellize, but not for the webservice request handling. This means logging information is only available after the computation is done and the service is not blocked by the next computation request. Still the log-to-computation addressing is a problem. |
We discussed this a bit yesterday, and the conclusion basically was that without more error handling info developing a UDF is nearly impossible or at least very difficult for a user and thus UDFs would not get adopted widely. The issue mentioned is that at the moment for example Wageningen always has to contact the provider to ask what went wrong with their UDF because they don't get any info except that it did not work. That doesn't scale (in terms of number of users). So I think that needs to be addressed in the design of UDFs and might require some redesigns long-term (not saying it must be done now). I'm not deep into R, but would think it might be possible to catch errors/exceptions for the UDF code and then return it to the back-end instead of the data response. That can be synchronous. Something like:
That would at least capture errors. At some point it would be good to also transmit more log information with the data response, but one step at a time... ;-) |
This is openeo-udf, thus my intention was to clarify (and improve) this for the UDF API, not necessarily speaking about any of the UDF API implementations. |
Some ideas:
|
Sounds good!
Is that somehow standardized in the API? Or is it just a 4xx/5xx error returned by the server that is simply written to the log?
Yes, both sounds good. Best would be if the logger could expose the same object structure as in the API, then it could simply be "passed" through. stdout is a simple string and shouldn't be that hard to implement, I hope? |
What seems to be missing from the communication protocol from the API instance and the UDF server is a way to send the log information so that they can be shown in the API logging endpoints. I think we need to add this.
cc @flahn
The text was updated successfully, but these errors were encountered: