Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/draft' into workspaces
Browse files Browse the repository at this point in the history
  • Loading branch information
m-mohr committed Sep 9, 2024
2 parents 7eccf0b + 5feccf7 commit 038ef4c
Show file tree
Hide file tree
Showing 7 changed files with 209 additions and 117 deletions.
1 change: 1 addition & 0 deletions .spectral.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ rules:
tag-description: true
oas3-parameter-description: true
oas3-unused-component: false # Broken: https://github.com/stoplightio/spectral/issues/1271
array-items: false
operation-summary-formatted:
description: Operation `summary` should start with upper case and not end with a dot.
given: '$.paths.*[?( @property === ''get'' || @property === ''put'' || @property === ''post'' || @property === ''delete'' || @property === ''options'' || @property === ''head'' || @property === ''patch'' || @property === ''trace'' )]'
Expand Down
11 changes: 9 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## Unreleased / Draft

### Added

- **New extensions:**
- [Remote Process Definition Extension](./extensions/remote-process-definition/README.md)

### Fixed

- `GET /file_formats`: Base paramater on top of normal JSON Schema, not Process JSON Schema
- `PATCH /services/{service_id}` and `PATCH /jobs/{job_id}`: Explicitly allow updating back-end specific properties (as in `POST`)
- `GET /services/{service_id}` and `GET /jobs/{job_id}`: Explicitly allow listing back-end specific properties (as provided in `POST`)
- Clarified for log levels which default values apply
- Clarified how the relation types `license`, `version-history` and `author` can be used to enrich the process metadata. [#531](https://github.com/Open-EO/openeo-api/issues/531)

## [1.2.0] - 2021-05-25

### Added

- **New extensions:**
- [Commercial Data Extension](./extensions/commercial-data/README.md)
- [Federation Extension](./extensions/federation/README.md)
- [Commercial Data Extension](./extensions/commercial-data/README.md)
- [Federation Extension](./extensions/federation/README.md)
- `GET /`: New Relation types: [#404](https://github.com/Open-EO/openeo-api/issues/404)
- `create-form` to link to the registration page
- `recovery-form` to link to the credentials recovery page.
Expand Down
11 changes: 6 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,11 @@ See also the [changelog](CHANGELOG.md) and the [milestones](https://github.com/O

## Extensions

| Name | Version | Stability | Description |
| ---------------------------------------------- | ------- | ------------ | ----------- |
| [Commercial Data](extensions/commercial-data/) | 0.1.0 | experimental | Provides an interface for discovering, ordering and using commercial data. |
| [Federation](extensions/federation/) | 0.1.0 | experimental | Covers federation aspects, i.e. where multiple back-ends are exposed as a single API. |
| Name | Version | Stability | Description |
| ------------------------------------------------------------------ | ------- | ------------ | ----------- |
| [Commercial Data](extensions/commercial-data/) | 0.1.0 | experimental | Provides an interface for discovering, ordering and using commercial data. |
| [Federation](extensions/federation/) | 0.1.0 | experimental | Covers federation aspects, i.e. where multiple back-ends are exposed as a single API. |
| [Remote Process Definition](extensions/remote-process-definition/) | 0.1.0 | experimental | Load user-defined processes that are hosted externally through the process namespace into process graphs. |

## Repository

Expand All @@ -43,7 +44,7 @@ This repository contains a set of files formally describing the openEO API, each
* The [assets](assets/) folder contains some useful additional files such as examples or schemas. All of these are non-binding additions. The source of truth are the top-level specification files.
* The [extensions](extensions/) folder contains extensions to the openEO API.

# Development
## Development

The `draft` branch is the latest version and is the one to create Pull Requests against.

Expand Down
6 changes: 3 additions & 3 deletions errors.json
Original file line number Diff line number Diff line change
Expand Up @@ -200,7 +200,7 @@
]
},
"ProcessInvalid": {
"description": "The process given is invalid, which ususlly means that the process metadata is invalid.",
"description": "The given process definition is invalid, which usually means that the process metadata is invalid.",
"message": "Invalid process specified.",
"http": 400,
"tags": [
Expand Down Expand Up @@ -241,8 +241,8 @@
]
},
"ProcessGraphComplexity": {
"description": "The process graph is too complex for synchronous processing and will likely time out. Please use a batch job instead.",
"message": "The process is too complex for for synchronous processing. Please use a batch job instead.",
"description": "The process graph is computationally too heavy for synchronous processing and will likely time out. Please use a batch job instead.",
"message": "The process graph is too heavy for synchronous processing. Please use a batch job instead.",
"http": 400,
"tags": [
"Data Processing"
Expand Down
3 changes: 3 additions & 0 deletions extensions/federation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,9 +115,11 @@ Applies to:
- `GET /processes`
- `GET /file_formats`
- `GET /process_graphs`
- `GET /files`
- `GET /jobs`
- `GET /jobs/{job_id}`
- `GET /jobs/{job_id}/results`
- `GET /jobs/{job_id}/logs`
- `GET /services`

The following endpoints define the resources (UDF runtimes / service types) at the top level of their response as key-value pairs.
Expand Down Expand Up @@ -160,6 +162,7 @@ Every discoverable resource that is defined as an object and allows to contain a
- `GET /collections/{id}`
- `GET /processes` (per process, per parameter)
- `GET /file_formats` (per file format)
- `GET /service_types` (per service)
- `GET /udf_runtimes` (per UDF runtime, per version)
- `POST /validation` (the back-ends that can run the process)
- `GET /process_graphs/{id}`
Expand Down
70 changes: 70 additions & 0 deletions extensions/remote-process-definition/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Remote Process Definition Extension

The openEO API is a specification for interoperable cloud-based processing of large Earth observation datasets.

This extension enables user to load user-defined processes that are hosted external to the openEO API (e.g., GitHub or cloud storage) through the process namespace into process graphs.

- Version: **0.1.0**
- Stability: **experimental**
- Conformance class: `https://api.openeo.org/extensions/remote-process-definition/0.1.0`

## Justification

The openEO API defines the `namespace` property in a process node of a process graph as follows:

> The following options are predefined by the openEO API, but additional namespaces may be introduced by back-ends or in a future version of the API.
> * `null` [...]
> * `backend` [...]
> * `user` [...]
This makes it possible for this extension to add additional allowed values to the `namespace` property.

## Specification

This extension extends the `namespace` property of process graph nodes so that it accepts **absolute** URL with the protocols `https` (**recommended**) and `http` (discouraged). The URLs specified MUST return one of the following two options:

1. A single process, compatible\* to the endpoint `GET /process_graphs/{process_graph_id}`.
In this case, the `id` property of the process graph node MUST be equal to the `id` of the process,
otherwise a `ProcessNamespaceInvalid` error is thrown
2. A list of processes, compatible\* to the endpoint `GET /process_graphs`.
In this case, the `id` property of the process graph node is used to identify the process from the list.
If not found a `ProcessNamespaceInvalid` error is thrown

\* Compatible means in this context that the requests and responses must comply to the openEO API specification with the following exceptions:

- User credentials / tokens that are obtained through the openEO API MUST NOT not be sent to URIs that are external to the openEO API.
The requirement to provide an `Authorization` header for the respective endpoints doesn't apply.
- For a list of processes, the full process description MUST be provided for the process with the given ID within the first request.
This means that the recommendation to omit large roperties such as `process_graph` doesn't apply.
It also requires that the requester doesn't need to paginate through additional pages to find the process with the given ID.
Ideally, the list of processes is not paginated as otherwise the process with the given ID may move to other pages over time.

### Client Considerations

If a client is conncected to a specific backend, the client MUST only offer this functionality to users if the conformance class of this extension is listed in the `conformsTo` property of the `GET /` endpoint.

The protocol `http` is discouraged for URLs as web-based clients may not be able to retrieve HTTP URLs from a HTTPS context.
For the same reason it is also RECOMMENDED to enable CORS for all URLs.

### Error Handling

The following error SHOULD be reported if the namespace can't be resolved:

- Code: `ProcessNamespaceInvalid`
- Message: `The value passed for namespace '{namespace}' in process '{process}' is invalid: {reason}`
- HTTP Status Code: 400

## Example

An exemplary process graph node:

```json
{
"process_id": "echo",
"namespace": "https://openeo.example/processes/echo",
"arguments": {
"message": "Hello World"
},
"result": true
}
```
Loading

0 comments on commit 038ef4c

Please sign in to comment.