Special no-data handling

Open-EO · Dec 14, 2023 · ccedf9f · ccedf9f
1 parent 71f06f0
commit ccedf9f
Show file tree

Hide file tree

Showing 74 changed files with 379 additions and 357 deletions.
diff --git a/tests/README.md b/tests/README.md
@@ -141,7 +141,6 @@ This folder contains test cases for the openEO processes.
 
 **Important:** The differentiation of null and NaN is to be discussed and reflected in the tests.
 See <https://github.com/Open-EO/openeo-processes/issues/480> for details.
-Also, several processes would be affected by <https://github.com/Open-EO/openeo-processes/pull/476>.
 
 ## Missing processes
 
@@ -170,7 +169,7 @@ We don't expect that we can provide meaningful test cases for these processes.
 
 The test cases assume a couple of things as they are an abstraction and not bound to specific implementations:
 - The JSON Schema type `number` explicitly includes the values `+Infinity`, `-Infinity` and `NaN`.
-- The input and output values for no-data values are `null`, so there's no mapping to e.g. `0` as no-data value.
+- The input and output values for no-data values are `null` by default unless otherwise specified by a runner.
 - Input that is not valid according to the schemas, will be rejected upfront and will not be checked on. For example, the absolute process only tests against the data types `number` and `null`. There are no tests for a boolean or string input.
 - Numerical data types such as uint8 don't matter, i.e. tests don't check for overflows etc. This suite can't provide such tests as the underlying data type is not known.
 - If not otherwise specified for numbers, a precision of 10 decimals is checked so return values should have at least 11 decimals.
@@ -262,6 +261,17 @@ properties:
           pattern: 'L\d([\w-])?'
 ```
 
+### No-data values
+
+No-data values have a special encoding in tests (see below).
+The encoding is replaced with `null` unless otherwise specified by the runners.
+
+```json
+{
+  "type": "nodata"
+}
+```
+
 ### Datetimes
 
 Datetimes as strings have a varying precision, especially regarding the milliseconds.
@@ -286,6 +296,8 @@ Arguments and return values can point to external files, e.g.
 }
 ```
 
+The test suite can currently only load JSON and JSON5 files.
+
 ### Labeled arrays
 
 Labeled arrays can't be represented in JSON5 and will be provided as an object instead.
@@ -317,6 +329,7 @@ Vector datacubes are currently not supported.
   "type": "datacube",
   "data": [
     // multi-dimensional array
+    // can be set to `null` if the data values are irrelevant for the test.
   ],
   "nodata": [
     NaN
@@ -351,7 +364,3 @@ Vector datacubes are currently not supported.
   ]
 }
 ```
-
-### Nodata
-
-todo
diff --git a/tests/absolute.json5 b/tests/absolute.json5
@@ -52,9 +52,9 @@
     },
     {
       "arguments": {
-        "x": null
+        "x": {"type": "nodata"}
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     }
   ]
 }
diff --git a/tests/add.json5 b/tests/add.json5
@@ -124,37 +124,37 @@
     {
       "arguments": {
         "x": -Infinity,
-        "y": null
+        "y": {"type": "nodata"}
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     },
     {
       "arguments": {
-        "x": null,
+        "x": {"type": "nodata"},
         "y": Infinity
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     },
     {
       "arguments": {
-        "x": null,
+        "x": {"type": "nodata"},
         "y": 1
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     },
     {
       "arguments": {
         "x": 1,
-        "y": null
+        "y": {"type": "nodata"}
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     },
     {
       "arguments": {
-        "x": null,
-        "y": null
+        "x": {"type": "nodata"},
+        "y": {"type": "nodata"}
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     }
   ]
 }
diff --git a/tests/all.json5 b/tests/all.json5
@@ -6,7 +6,7 @@
       "arguments": {
         "data": [
           false,
-          null
+          {"type": "nodata"}
         ]
       },
       "returns": false
@@ -15,7 +15,7 @@
       "arguments": {
         "data": [
           true,
-          null
+          {"type": "nodata"}
         ]
       },
       "returns": true
@@ -24,7 +24,7 @@
       "arguments": {
         "data": [
           false,
-          null
+          {"type": "nodata"}
         ],
         "ignore_nodata": false
       },
@@ -34,11 +34,11 @@
       "arguments": {
         "data": [
           true,
-          null
+          {"type": "nodata"}
         ],
         "ignore_nodata": false
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     },
     {
       "arguments": {
@@ -80,17 +80,17 @@
     {
       "arguments": {
         "data": [
-          null
+          {"type": "nodata"}
         ],
         "ignore_nodata": false
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     },
     {
       "arguments": {
         "data": []
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     }
   ]
 }
diff --git a/tests/and.json5 b/tests/and.json5
@@ -19,9 +19,9 @@
     {
       "arguments": {
         "x": true,
-        "y": null
+        "y": {"type": "nodata"}
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     },
     {
       "arguments": {
@@ -40,30 +40,30 @@
     {
       "arguments": {
         "x": false,
-        "y": null
+        "y": {"type": "nodata"}
       },
       "returns": false
     },
     {
       "arguments": {
-        "x": null,
+        "x": {"type": "nodata"},
         "y": true
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     },
     {
       "arguments": {
-        "x": null,
+        "x": {"type": "nodata"},
         "y": false
       },
       "returns": false
     },
     {
       "arguments": {
-        "x": null,
-        "y": null
+        "x": {"type": "nodata"},
+        "y": {"type": "nodata"}
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     }
   ]
 }
diff --git a/tests/any.json5 b/tests/any.json5
@@ -6,7 +6,7 @@
       "arguments": {
         "data": [
           false,
-          null
+          {"type": "nodata"}
         ]
       },
       "returns": false
@@ -15,7 +15,7 @@
       "arguments": {
         "data": [
           true,
-          null
+          {"type": "nodata"}
         ]
       },
       "returns": true
@@ -24,17 +24,17 @@
       "arguments": {
         "data": [
           false,
-          null
+          {"type": "nodata"}
         ],
         "ignore_nodata": false
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     },
     {
       "arguments": {
         "data": [
           true,
-          null
+          {"type": "nodata"}
         ],
         "ignore_nodata": false
       },
@@ -80,17 +80,17 @@
     {
       "arguments": {
         "data": [
-          null
+          {"type": "nodata"}
         ],
         "ignore_nodata": false
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     },
     {
       "arguments": {
         "data": []
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     }
   ]
 }
diff --git a/tests/arccos.json5 b/tests/arccos.json5
@@ -64,9 +64,9 @@
     },
     {
       "arguments": {
-        "x": null
+        "x": {"type": "nodata"}
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     }
   ]
 }
diff --git a/tests/arcosh.json5 b/tests/arcosh.json5
@@ -52,9 +52,9 @@
     },
     {
       "arguments": {
-        "x": null
+        "x": {"type": "nodata"}
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     }
   ]
 }
diff --git a/tests/arctan.json5 b/tests/arctan.json5
@@ -64,9 +64,9 @@
     },
     {
       "arguments": {
-        "x": null
+        "x": {"type": "nodata"}
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     }
   ]
 }
diff --git a/tests/arctan2.json5 b/tests/arctan2.json5
@@ -130,24 +130,24 @@
     },
     {
       "arguments": {
-        "y": null,
+        "y": {"type": "nodata"},
         "x": 1
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     },
     {
       "arguments": {
         "y": 1,
-        "x": null
+        "x": {"type": "nodata"}
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     },
     {
       "arguments": {
-        "y": null,
-        "x": null
+        "y": {"type": "nodata"},
+        "x": {"type": "nodata"}
       },
-      "returns": null
+      "returns": {"type": "nodata"}
     }
   ]
 }