Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add load_geojson #427

Merged
merged 7 commits into from
Apr 3, 2023
Merged

Add load_geojson #427

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
f303ee3
Add load_geojson #346 #415
m-mohr Mar 31, 2023
3a6fa58
Fix CI
m-mohr Mar 31, 2023
26aa3bd
Fix description
m-mohr Mar 31, 2023
005b4f0
Updates
m-mohr Mar 31, 2023
b21b2d8
Fix typo
m-mohr Mar 31, 2023
dce5834
Feedback from code review
m-mohr Apr 3, 2023
77bf6a6
Merge branch 'draft' into load_geojson
m-mohr Apr 3, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `date_difference`
- `filter_vector`
- `flatten_dimensions`
- `load_geojson`
- `unflatten_dimension`
- `vector_buffer`
- `vector_reproject`
Expand Down Expand Up @@ -54,7 +55,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Deprecated

- `aggregate_spatial`, `filter_spatial`, `load_collection`, `mask_polygon`: GeoJSON input is deprecated. [#346](https://github.com/Open-EO/openeo-processes/issues/346)
- `aggregate_spatial`, `filter_spatial`, `load_collection`, `mask_polygon`: GeoJSON input is deprecated in favor of `load_geojson`. [#346](https://github.com/Open-EO/openeo-processes/issues/346)

### Removed

Expand All @@ -65,7 +66,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `load_result`:
- Renamed to `load_stac`
- The subtype `job-id` was removed in favor of providing a URL. [#322](https://github.com/Open-EO/openeo-processes/issues/322), [#377](https://github.com/Open-EO/openeo-processes/issues/377), [#384](https://github.com/Open-EO/openeo-processes/issues/384)
- GeoJSON input is not supported any longer. [#346](https://github.com/Open-EO/openeo-processes/issues/346)
- GeoJSON input is not supported any longer. Use `load_geojson` instead. [#346](https://github.com/Open-EO/openeo-processes/issues/346)
- The comparison processes `eq`, `neq`, `lt`, `lte`, `gt`, `gte` and `array_contains`:
- Removed support for temporal comparison. Instead explicitly use `date_difference`.
- Removed support for the input data types array and object. [#208](https://github.com/Open-EO/openeo-processes/issues/208)
Expand Down
2 changes: 1 addition & 1 deletion aggregate_spatial.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
"title": "GeoJSON",
"type": "object",
"subtype": "geojson",
"description": "Deprecated. The GeoJSON type `GeometryCollection` is not supported.",
"description": "Deprecated in favor of ``load_geojson()``. The GeoJSON type `GeometryCollection` is not supported.",
"deprecated": true
}
]
Expand Down
2 changes: 1 addition & 1 deletion filter_spatial.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
"title": "GeoJSON",
"type": "object",
"subtype": "geojson",
"description": "Deprecated. The GeoJSON type `GeometryCollection` is not supported.",
"description": "Deprecated in favor of ``load_geojson()``. The GeoJSON type `GeometryCollection` is not supported.",
"deprecated": true
}
]
Expand Down
2 changes: 1 addition & 1 deletion load_collection.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@
},
{
"title": "GeoJSON",
"description": "Deprecated. Limits the data cube to the bounding box of the given geometries. For raster data, all pixels inside the bounding box that do not intersect with any of the polygons will be set to no data (`null`).\n\nThe GeoJSON type `GeometryCollection` is not supported. Empty geometries are ignored.",
"description": "Deprecated in favor of ``load_geojson()``. Limits the data cube to the bounding box of the given geometries. For raster data, all pixels inside the bounding box that do not intersect with any of the polygons will be set to no data (`null`).\n\nThe GeoJSON type `GeometryCollection` is not supported. Empty geometries are ignored.",
"type": "object",
"subtype": "geojson",
"deprecated": true
Expand Down
2 changes: 1 addition & 1 deletion mask_polygon.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@
"title": "GeoJSON",
"type": "object",
"subtype": "geojson",
"description": "Deprecated. The GeoJSON type `GeometryCollection` is not supported.",
"description": "Deprecated in favor of ``load_geojson()``. The GeoJSON type `GeometryCollection` is not supported.",
"deprecated": true
}
]
Expand Down
2 changes: 1 addition & 1 deletion meta/subtype-schemas.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@
"type": "object",
"subtype": "geojson",
"title": "GeoJSON",
"description": "Deprecated. GeoJSON as defined by [RFC 7946](https://www.rfc-editor.org/rfc/rfc7946.html). The GeoJSON type `GeometryCollection` is not supported.",
"description": "GeoJSON as defined by [RFC 7946](https://www.rfc-editor.org/rfc/rfc7946.html). The GeoJSON type `GeometryCollection` is not supported.",
"deprecated": true,
"allOf": [
{
Expand Down
53 changes: 53 additions & 0 deletions proposals/load_geojson.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
{
"id": "load_geojson",
"summary": "Converts GeoJSON into a vector data cube",
"description": "Converts GeoJSON data as defined by [RFC 7946](https://www.rfc-editor.org/rfc/rfc7946.html) into a vector data cube. Feature properties are preserved.",
"categories": [
"import",
"vector"
],
"experimental": true,
"parameters": [
{
"name": "data",
"description": "A GeoJSON object to inline into the process. The GeoJSON type `GeometryCollection` is not supported. Each geometry in the GeoJSON data results in a dimension label in the `geometries` dimension.",
m-mohr marked this conversation as resolved.
Show resolved Hide resolved
"schema": {
"type": "object",
"subtype": "geojson"
}
},
{
"name": "properties",
"description": "A list of properties from the GeoJSON file to construct an additional dimension from. A new dimension is created if at least one property is provided. Only applies for GeoJSON Features and FeatureCollections. Missing values are generally set to no-data (`null`).\n\nDepending on the number of properties provided, the process creates the dimension differently:\n\n- Single property with scalar values: A single dimension label with the name of the property and a single value per geometry.\n- Single property of type array: The dimension labels correspond to the array indices. There are as many values and labels per geometry as there are for the largest array.\n- Multiple properties with scalar values: The dimension labels correspond to the property names. There are as many values and labels per geometry as there are properties provided here.\n\nDepending on the number of properties the dimension name is:\n\n- Single property: Name of the property\n- Multiple properties: `properties`.",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some notes about the "properties" parameter. A bit nitpicking, so not necessarily blockers to merge already (can be moved to follow up ticket).

Single property with scalar values: ...
Single property of type array: ...

This assumes the a feature collection is internally consistent: the property is available for each geometry and it has same type for each geometry. When this is not the case: should this raise an error? Or should we describe type coercion rules: e.g. missing/null -> empty array, scalar value -> array with one item, so that everything can be described in terms of arrays.

Depending on the number of properties the dimension name is:\n\n- Single property: Name of the property\n- Multiple properties: properties.

I find it a bit strange to change the dimension name based on number of labels. We don't do that with other dimensions I think

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assumes the a feature collection is internally consistent: the property is available for each geometry and it has same type for each geometry. When this is not the case: should this raise an error? Or should we describe type coercion rules: e.g. missing/null -> empty array, scalar value -> array with one item, so that everything can be described in terms of arrays.

As we also discusssed in the last meeting, openEO allows mixed-type data cubes, so in theory it should not throw an error, but if a "back-end limitation" requires consistent types and they are not consistent, yes, the implementation should throw an error.

I find it a bit strange to change the dimension name based on number of labels. We don't do that with other dimensions I think

Changed to be always properties (and type: 'other').

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you think there's more room for improvements, please open an issue as I'll merge this now. Thanks :-)

"schema": {
"type": "array",
"uniqueItems": true,
"items": {
"type": "string"
}
},
"default": [],
"optional": true
}
],
"returns": {
"description": "A vector data cube containing the geometries, either one or two dimensional.",
"schema": {
"type": "object",
"subtype": "datacube",
"dimensions": [
{
"type": "geometry"
}
]
}
},
"links": [
{
"href": "https://www.rfc-editor.org/rfc/rfc7946.html",
"title": "RFC 7946: The GeoJSON Format",
"type": "text/html",
"rel": "about"
}
]
}
1 change: 1 addition & 0 deletions tests/.words
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,3 +46,4 @@ Breiman
Hyndman
date1
date2
favor