feature request: endpoint to delete a batch job entry

[soxofaan]

API needs support for DELETE /jobs/{id}, which should deletes the whole document related to the batch job.

To be discussed when this feature request is to be started:
should we actually delete the whole document, or just flag it as deleted in some way and hide it from default listings? Keeping some metadata about batch jobs might be important for reporting, usage tracking, post-mortem investigations, etc

(not high prio)

[JanssenBrm]

In which case is the client executing the DELETE request?

If you are using the data in ES for reporting purposes or post-mortems, it makes sense to not delete the document but to mark it as deleted and hide it from the listings.

[soxofaan]

On the level of the openEO API, users can delete batch jobs, which:

• removes the job from their batch job listing
• removes all generated intermediate/result data (geotiff, netcdf, ...) to save on resource usage

But indeed, on the level of job registry I would keep the metadata (for reporting/post-mortem reasons) and just flag the document as deleted, so that we can properly hide the job entry from a regular user's batch job listing.
@jdries what do you think?

I'd propose to add an optional toplevel field "deleted" which:

• missing (or null, or false) by default
• when user deletes batch job: set to deletion date

@JanssenBrm I'm not sure if this elegantly allows to write an ES query to only select docs where this field is missing/null?

[JanssenBrm]

As the query is sent in the request body, we can always extend it on the backend to always include an additional filter for these hidden jobs.

[soxofaan]

yes I understand,
but I mean I don't know ES query language enough to know if it is easy/efficient to do something along the lines of SELECT ... WHERE deleted IN [UNSET, NULL, EMPTY, FALSE, '']

[JanssenBrm]

These kinds of queries are indeed supported in ES. If we make sure that only the backend can manipulate the deleted key, the amount of cases is limited to 2 (either non-existing or a boolean value).