Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for labeled arrays #133

Merged
merged 9 commits into from
Jan 27, 2020
11 changes: 8 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,17 +13,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `all`
- `any`
- `array_apply`
- `array_contains`
- `array_filter`
- `array_find`
- `array_labels`
- `dimension_labels`
- `drop_dimension`
- `filter_labels`
- `labels`
- `load_uploaded_files`
- `mask_polygon`
- `reduce_dimension`
- `reduce_dimension_binary`
- `rename_labels`
- Added further examples
- Support for labeled arrays. [API#245](https://github.com/Open-EO/openeo-api/issues/245)
- Added further examples.

### Changed
- The JSON Schema keyword `format` has been replaced with the custom keyword `subtype`.
Expand All @@ -34,9 +37,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Processes supporting multiple data types in parameters or return values with `anyOf` are now listing the data types directly as array. `anyOf` is discouraged.
- Comparison processes `eq`, `gt`, `gte`, `lt`, `lte`, `neq` and `between` accept all data types as input for the operands.
- `add_dimension`: Parameter `value` renamed to `label`.
- `apply_dimension`: Replaced with a completely new definition. [#73](https://github.com/Open-EO/openeo-processes/issues/73)
- `aggregate_polygon`, `aggregate_temporal`, `apply_dimension`, `array_element` and `resample_cube_temporal`: Support labeled arrays. [API#245](https://github.com/Open-EO/openeo-api/issues/245)
- `aggregate_polygon`: The data cube implicitly gets restricted to the bounds of the polygons as if `filter_polygon` would have been used beforehand. [#101](https://github.com/Open-EO/openeo-processes/issues/101)
- `aggregate_temporal`: Parameter `labels` is optional. [#19](https://github.com/Open-EO/openeo-processes/issues/19)
- `apply_dimension`: Replaced with a completely new definition. [#73](https://github.com/Open-EO/openeo-processes/issues/73)
- `apply_kernel`: Only supported 2D kernels on the horizontal spatial dimensions. [#69](https://github.com/Open-EO/openeo-processes/issues/69)
- `clip`: Works on a single value instead on arrays (replaced parameter `data` with `x`). [#75](https://github.com/Open-EO/openeo-processes/issues/75)
- `count`: Renamed parameter `expression` to `condition`.
Expand All @@ -58,6 +62,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- The following operations work on two values instead on a sequence of values: `and`, `divide`, `multiply`, `or`, `subtract`, `xor`. [#85](https://github.com/Open-EO/openeo-processes/issues/85)
- `product` works as before, but is not an alias of `multiply` any longer. [#85](https://github.com/Open-EO/openeo-processes/issues/85)
- `text_begins`, `text_contains`, `text_ends`: `null` values are supported and get passed through.
- `trim`: Renamed to `trim_cube`.

### Deprecated
- `filter_bbox`, `load_collection`, `resample_spatial`: PROJ definitions are deprecated in favor of EPSG codes and WKT2. [#58](https://github.com/Open-EO/openeo-processes/issues/58)
Expand Down
3 changes: 2 additions & 1 deletion aggregate_polygon.json
Original file line number Diff line number Diff line change
Expand Up @@ -40,10 +40,11 @@
"parameters": [
{
"name": "data",
"description": "An array with elements of any type.",
"description": "A labeled array with values of any type.",
"required": true,
"schema": {
"type": "array",
"subtype": "labeled-array",
"items": {
"description": "Any data type."
}
Expand Down
3 changes: 2 additions & 1 deletion aggregate_temporal.json
Original file line number Diff line number Diff line change
Expand Up @@ -89,10 +89,11 @@
"parameters": [
{
"name": "data",
"description": "An array with elements of any type.",
"description": "A labeled array with elements of any type.",
"required": true,
"schema": {
"type": "array",
"subtype": "labeled-array",
"items": {
"description": "Any data type."
}
Expand Down
3 changes: 2 additions & 1 deletion apply_dimension.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,11 @@
"parameters": [
{
"name": "data",
"description": "An array with elements of any type.",
"description": "A labeled array with elements of any type.",
"required": true,
"schema": {
"type": "array",
"subtype": "labeled-array",
"items": {
"description": "Any data type."
}
Expand Down
16 changes: 16 additions & 0 deletions array_apply.json
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,22 @@
"schema": {
"type": "integer"
}
},
{
"name": "label",
"description": "The label of the current element being processed. Only populated for labeled arrays.",
"schema": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null
}
]
},
Expand Down
2 changes: 1 addition & 1 deletion array_contains.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"id": "array_contains",
"summary": "Check whether the array contains a given value",
"description": "Checks whether the array specified for `data` contains the value specified in `value`. Returns `true` if there's a match, otherwise `false`.\n\n**Remarks:**\n\n* To get the index of the value found, use ``array_find()``.\n* All definitions for the process ``eq()`` regarding the comparison of values apply here as well. This includes that data types MUST be checked strictly, for example a string with the content *1* is not equal to the number *1*.\n* An integer *1* is equal to a floating point number *1.0* as `integer` is a sub-type of `number`. Still, this process may return unexpectedly `false` when comparing floating point numbers due to floating point inaccuracy in machine-based computation.\n* Temporal strings are treated as normal strings and MUST NOT be interpreted.",
"description": "Checks whether the array specified for `data` contains the value specified in `value`. Returns `true` if there's a match, otherwise `false`.\n\n**Remarks:**\n\n* To get the index or the label of the value found, use ``array_find()``.\n* All definitions for the process ``eq()`` regarding the comparison of values apply here as well. This includes that data types MUST be checked strictly, for example a string with the content *1* is not equal to the number *1*.\n* An integer *1* is equal to a floating point number *1.0* as `integer` is a sub-type of `number`. Still, this process may return unexpectedly `false` when comparing floating point numbers due to floating point inaccuracy in machine-based computation.\n* Temporal strings are treated as normal strings and MUST NOT be interpreted.",
"categories": [
"arrays",
"comparison"
Expand Down
29 changes: 23 additions & 6 deletions array_element.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"id": "array_element",
"summary": "Get an element from an array",
"description": "Returns the element at the specified zero-based index from the array.",
"description": "Returns the element with the specified index or label from the array.\n\nEither the parameter `index` or `label` must be specified, otherwise the `ArrayElementParameterMissing` exception is thrown. If both parameters are set the `ArrayElementParameterConflict` exception is thrown.",
"categories": [
"arrays"
],
Expand All @@ -22,12 +22,23 @@
"description": "The zero-based index of the element to retrieve.",
"schema": {
"type": "integer"
},
"required": true
}
},
{
"name": "label",
"description": "The label of the element to retrieve.",
"schema": [
{
"type": "number"
},
{
"type": "string"
}
]
},
{
"name": "return_nodata",
"description": "By default this process throws an `IndexOutOfBounds` exception if the index is invalid. If you want to return `null` instead, set this flag to `true`.",
"description": "By default this process throws an `ArrayElementNotAvailable` exception if the index or label is invalid. If you want to return `null` instead, set this flag to `true`.",
"schema": {
"type": "boolean"
},
Expand All @@ -41,8 +52,14 @@
}
},
"exceptions": {
"IndexOutOfBounds": {
"message": "The array has no element with the specified index."
"ArrayElementNotAvailable": {
"message": "The array has no element with the specified index or label."
},
"ArrayElementParameterMissing": {
"message": "The process 'array_element' requires either the 'index' or 'labels' parameter to be set."
},
"ArrayElementParameterConflict": {
"message": "The process 'array_element' only allows that either the 'index' or the 'labels' parameter is set."
}
},
"examples": [
Expand Down
16 changes: 16 additions & 0 deletions array_filter.json
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,22 @@
"schema": {
"type": "integer"
}
},
{
"name": "label",
"description": "The label of the current element being processed. Only populated for labeled arrays.",
"schema": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null
}
]
},
Expand Down
27 changes: 16 additions & 11 deletions array_find.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"id": "array_find",
"summary": "Get the index for a value in an array",
"description": "Checks whether the array specified for `data` contains the value specified in `value` and returns the zero-based index for the first match. If there's no match, `-1` is returned.\n\n**Remarks:**\n\n* To get a boolean value returned use ``array_contains()``.\n* All definitions for the process ``eq()`` regarding the comparison of values apply here as well. This includes that data types MUST be checked strictly, for example a string with the content *1* is not equal to the number *1*.\n* An integer *1* is equal to a floating point number *1.0* as `integer` is a sub-type of `number`. Still, this process may return unexpectedly `false` when comparing floating point numbers due to floating point inaccuracy in machine-based computation.\n* Temporal strings are treated as normal strings and MUST NOT be interpreted.",
"description": "Checks whether the array specified for `data` contains the value specified in `value` and returns the zero-based index for the first match. If there's no match, `null` is returned.\n\n**Remarks:**\n\n* To get a boolean value returned use ``array_contains()``.\n* All definitions for the process ``eq()`` regarding the comparison of values apply here as well. This includes that data types MUST be checked strictly, for example a string with the content *1* is not equal to the number *1*.\n* An integer *1* is equal to a floating point number *1.0* as `integer` is a sub-type of `number`. Still, this process may return unexpectedly `false` when comparing floating point numbers due to floating point inaccuracy in machine-based computation.\n* Temporal strings are treated as normal strings and MUST NOT be interpreted.",
"categories": [
"arrays"
],
Expand All @@ -27,11 +27,16 @@
}
],
"returns": {
"description": "Returns the index of the first element with the specified value. If no element was found, `-1` is returned.",
"schema": {
"type": "integer",
"minimum": -1
}
"description": "Returns the index of the first element with the specified value. If no element was found, `null` is returned.",
"schema": [
{
"type": "null"
},
{
"type": "integer",
"minimum": 0
}
]
},
"examples": [
{
Expand All @@ -54,7 +59,7 @@
],
"value": "b"
},
"returns": -1
"returns": null
},
{
"arguments": {
Expand All @@ -65,7 +70,7 @@
],
"value": "2"
},
"returns": -1
"returns": null
},
{
"arguments": {
Expand Down Expand Up @@ -96,7 +101,7 @@
2
]
},
"returns": -1
"returns": null
},
{
"arguments": {
Expand All @@ -112,7 +117,7 @@
],
"value": 2
},
"returns": -1
"returns": null
},
{
"arguments": {
Expand All @@ -128,7 +133,7 @@
"a": "b"
}
},
"returns": -1
"returns": null
}
]
}
35 changes: 35 additions & 0 deletions array_labels.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{
"id": "array_labels",
"summary": "Get the labels for an array",
"description": "Returns all labels for a labeled array in the data cube. The labels have the same order as in the array.",
"categories": [
"arrays"
],
"parameters": [
{
"name": "data",
"description": "An array with labels.",
"schema": {
"type": "array",
"subtype": "labeled-array"
},
"required": true
}
],
"returns": {
"description": "The labels as array.",
"schema": {
"type": "array",
"items": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
}
]
}
}
}
}
2 changes: 1 addition & 1 deletion labels.json → dimension_labels.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"id": "labels",
"id": "dimension_labels",
"summary": "Get the dimension labels",
"description": "Returns all labels for a dimension in the data cube. The labels have the same order as in the data cube.",
"categories": [
Expand Down
2 changes: 1 addition & 1 deletion quantiles.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"id": "quantiles",
"summary": "Quantiles",
"description": "Calculates quantiles, which are cut points dividing the range of a probability distribution into either\n\n* intervals corresponding to the given `probabilities` or\n* (nearly) equal-sized intervals (q-quantiles based on the parameter `q`).\n\nEither the parameter `probabilites` or `q` must be specified, otherwise the `QuantilesParameterMissing` exception must be thrown. If both parameters are set the `QuantilesParameterConflict` exception must be thrown.",
"description": "Calculates quantiles, which are cut points dividing the range of a probability distribution into either\n\n* intervals corresponding to the given `probabilities` or\n* (nearly) equal-sized intervals (q-quantiles based on the parameter `q`).\n\nEither the parameter `probabilites` or `q` must be specified, otherwise the `QuantilesParameterMissing` exception is thrown. If both parameters are set the `QuantilesParameterConflict` exception is thrown.",
"categories": [
"math"
],
Expand Down
3 changes: 2 additions & 1 deletion reduce_dimension.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,11 @@
"parameters": [
{
"name": "data",
"description": "An array with elements of any type.",
"description": "A labeled array with elements of any type.",
"required": true,
"schema": {
"type": "array",
"subtype": "labeled-array",
"items": {
"description": "Any data type."
}
Expand Down
3 changes: 2 additions & 1 deletion resample_cube_temporal.json
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,11 @@
"parameters": [
{
"name": "data",
"description": "An array with elements of any type.",
"description": "A labeled array with elements of any type.",
"required": true,
"schema": {
"type": "array",
"subtype": "labeled-array",
"items": {
"description": "Any data type."
}
Expand Down
3 changes: 2 additions & 1 deletion tests/.words
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,5 @@ outputMin
outputMax
Sentinel-2
Sentinel-2A
Sentinel-2B
Sentinel-2B
labeled
4 changes: 3 additions & 1 deletion tests/processes.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ var summaryDotRegexp = /[^\.]$/;
var files = glob.sync("../*.json", {realpath: true});

var anyOfRequired = [
"quantiles"
"quantiles",
"array_element"
];

var subtypes = {
Expand All @@ -34,6 +35,7 @@ var subtypes = {
'input-format-options': {type: 'object'},
'job-id': {type: 'string'},
'kernel': {type: 'array'},
'labeled-array': {type: 'array'},
'output-format': {type: 'string'},
'output-format-options': {type: 'object'},
'process-graph': {type: 'object'},
Expand Down
2 changes: 1 addition & 1 deletion trim.json → trim_cube.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"id": "trim",
"id": "trim_cube",
"summary": "Remove slices with no-data values",
"description": "Removes slices solely containing no-data values. If the dimension is irregular categorical then slices in the middle can be removed.",
"categories": [
Expand Down