Identify dataset and endpoint

[dzole0311]

We would like to transition from the heatmax_median_multivariate endpoint to the heatmax_median_multivariate_historical_annual endpoint in the EIC app, but there are discrepancies in the data, particularly missing StdTime values after 2014 (the historical cut off date), and differences in the returned values between the two endpoints.

Key points:

1. Endpoint:

• We intend to switch to the heatmax_median_multivariate_historical_annual endpoint as monthly data is no longer needed
• However, after 2014, the StdTime values are missing from the returned data for future years (until 2100)

2. Discrepancies in returned values:

• The two endpoints return different values for the same time period and location
• Example: For Phoenix, AZ in 2013, the two endpoints return different values:
• Annual endpoint returns a heatmax_ssp585 value of 102.53°F for 2013 (with the StdTime showing December as the chosen month)
• Multivariate endpoint returns monthly values for 2013, where no values exceed 100°F, with the December value being 64.23°F

Detailed example:

1. Heatmax Median Multivariate Annual Endpoint (heatmax_median_multivariate_historical_annual):

• Example query for Phoenix, AZ
  
• Filtered response for December, 2013:

    {
      "location": {
        "x": -112.069953983482,
        "y": 33.4506584527509,
        "spatialReference": {
          "wkid": 4326,
          "latestWkid": 4326
        }
      },
      "locationId": 0,
      "value": "102.534088135",
      "resolution": 0.25,
      "attributes": {
        "heatmax_historical": "98.598838806",
        "StdTime": 1388534399914,
        "StdTime_Max": 1388534399914,
        "Variables": "heatmax_historical,heatmax_ssp126,heatmax_ssp245,heatmax_ssp370,heatmax_ssp585",
        "Dimensions": "StdTime",
        "heatmax_ssp126": "100.433685303",
        "heatmax_ssp245": "101.222366333",
        "heatmax_ssp370": "101.406921387",
        "heatmax_ssp585": "102.534088135"
      }
    },

2. Heatmax Median Multivariate (heatmax_median_multivariate):

• Example query for Phoenix, AZ
  
• Filtered response for the whole year 2013, with December being the last sample in the blob (also no values throughout the year go above 100°F so it's unclear what the data serves or represents)

{
      "location": {
        "x": -112.068201294649,
        "y": 33.4279757101746,
        "spatialReference": {
          "wkid": 4326,
          "latestWkid": 4326
        }
      },
      "locationId": 0,
      "value": "64.965362549",
      "resolution": 0.25,
      "attributes": {
        "heatmax_ssp126": "64.965362549",
        "StdTime": 1359590400000,
        "StdTime_Max": 1359590400000,
        "Variables": "heatmax_ssp126,heatmax_ssp245,heatmax_ssp370,heatmax_ssp585",
        "Dimensions": "StdTime",
        "heatmax_ssp245": "64.965362549",
        "heatmax_ssp370": "64.965362549",
        "heatmax_ssp585": "64.965362549"
      }
    },
    {
      "location": {
        "x": -112.068201294649,
        "y": 33.4279757101746,
        "spatialReference": {
          "wkid": 4326,
          "latestWkid": 4326
        }
      },
      "locationId": 0,
      "value": "68.672439575",
      "resolution": 0.25,
      "attributes": {
        "heatmax_ssp126": "68.672439575",
        "StdTime": 1362009600000,
        "StdTime_Max": 1362009600000,
        "Variables": "heatmax_ssp126,heatmax_ssp245,heatmax_ssp370,heatmax_ssp585",
        "Dimensions": "StdTime",
        "heatmax_ssp245": "68.672439575",
        "heatmax_ssp370": "68.672439575",
        "heatmax_ssp585": "68.672439575"
      }
    },
    {
      "location": {
        "x": -112.068201294649,
        "y": 33.4279757101746,
        "spatialReference": {
          "wkid": 4326,
          "latestWkid": 4326
        }
      },
      "locationId": 0,
      "value": "74.079086304",
      "resolution": 0.25,
      "attributes": {
        "heatmax_ssp126": "74.079086304",
        "StdTime": 1364688000000,
        "StdTime_Max": 1364688000000,
        "Variables": "heatmax_ssp126,heatmax_ssp245,heatmax_ssp370,heatmax_ssp585",
        "Dimensions": "StdTime",
        "heatmax_ssp245": "74.079086304",
        "heatmax_ssp370": "74.079086304",
        "heatmax_ssp585": "74.079086304"
      }
    },
    {
      "location": {
        "x": -112.068201294649,
        "y": 33.4279757101746,
        "spatialReference": {
          "wkid": 4326,
          "latestWkid": 4326
        }
      },
      "locationId": 0,
      "value": "81.966514587",
      "resolution": 0.25,
      "attributes": {
        "heatmax_ssp126": "81.966514587",
        "StdTime": 1367280000000,
        "StdTime_Max": 1367280000000,
        "Variables": "heatmax_ssp126,heatmax_ssp245,heatmax_ssp370,heatmax_ssp585",
        "Dimensions": "StdTime",
        "heatmax_ssp245": "81.966514587",
        "heatmax_ssp370": "81.966514587",
        "heatmax_ssp585": "81.966514587"
      }
    },
    {
      "location": {
        "x": -112.068201294649,
        "y": 33.4279757101746,
        "spatialReference": {
          "wkid": 4326,
          "latestWkid": 4326
        }
      },
      "locationId": 0,
      "value": "89.827903748",
      "resolution": 0.25,
      "attributes": {
        "heatmax_ssp126": "89.827903748",
        "StdTime": 1369958400000,
        "StdTime_Max": 1369958400000,
        "Variables": "heatmax_ssp126,heatmax_ssp245,heatmax_ssp370,heatmax_ssp585",
        "Dimensions": "StdTime",
        "heatmax_ssp245": "89.827903748",
        "heatmax_ssp370": "89.827903748",
        "heatmax_ssp585": "89.827903748"
      }
    },
    {
      "location": {
        "x": -112.068201294649,
        "y": 33.4279757101746,
        "spatialReference": {
          "wkid": 4326,
          "latestWkid": 4326
        }
      },
      "locationId": 0,
      "value": "96.286590576",
      "resolution": 0.25,
      "attributes": {
        "heatmax_ssp126": "96.286590576",
        "StdTime": 1372550400000,
        "StdTime_Max": 1372550400000,
        "Variables": "heatmax_ssp126,heatmax_ssp245,heatmax_ssp370,heatmax_ssp585",
        "Dimensions": "StdTime",
        "heatmax_ssp245": "96.286590576",
        "heatmax_ssp370": "96.286590576",
        "heatmax_ssp585": "96.286590576"
      }
    },
    {
      "location": {
        "x": -112.068201294649,
        "y": 33.4279757101746,
        "spatialReference": {
          "wkid": 4326,
          "latestWkid": 4326
        }
      },
      "locationId": 0,
      "value": "98.598838806",
      "resolution": 0.25,
      "attributes": {
        "heatmax_ssp126": "98.598838806",
        "StdTime": 1375228800000,
        "StdTime_Max": 1375228800000,
        "Variables": "heatmax_ssp126,heatmax_ssp245,heatmax_ssp370,heatmax_ssp585",
        "Dimensions": "StdTime",
        "heatmax_ssp245": "98.598838806",
        "heatmax_ssp370": "98.598838806",
        "heatmax_ssp585": "98.598838806"
      }
    },
    {
      "location": {
        "x": -112.068201294649,
        "y": 33.4279757101746,
        "spatialReference": {
          "wkid": 4326,
          "latestWkid": 4326
        }
      },
      "locationId": 0,
      "value": "97.450241089",
      "resolution": 0.25,
      "attributes": {
        "heatmax_ssp126": "97.450241089",
        "StdTime": 1377907200000,
        "StdTime_Max": 1377907200000,
        "Variables": "heatmax_ssp126,heatmax_ssp245,heatmax_ssp370,heatmax_ssp585",
        "Dimensions": "StdTime",
        "heatmax_ssp245": "97.450241089",
        "heatmax_ssp370": "97.450241089",
        "heatmax_ssp585": "97.450241089"
      }
    },
    {
      "location": {
        "x": -112.068201294649,
        "y": 33.4279757101746,
        "spatialReference": {
          "wkid": 4326,
          "latestWkid": 4326
        }
      },
      "locationId": 0,
      "value": "94.168045044",
      "resolution": 0.25,
      "attributes": {
        "heatmax_ssp126": "94.168045044",
        "StdTime": 1380499200000,
        "StdTime_Max": 1380499200000,
        "Variables": "heatmax_ssp126,heatmax_ssp245,heatmax_ssp370,heatmax_ssp585",
        "Dimensions": "StdTime",
        "heatmax_ssp245": "94.168045044",
        "heatmax_ssp370": "94.168045044",
        "heatmax_ssp585": "94.168045044"
      }
    },
    {
      "location": {
        "x": -112.068201294649,
        "y": 33.4279757101746,
        "spatialReference": {
          "wkid": 4326,
          "latestWkid": 4326
        }
      },
      "locationId": 0,
      "value": "86.392776489",
      "resolution": 0.25,
      "attributes": {
        "heatmax_ssp126": "86.392776489",
        "StdTime": 1383177600000,
        "StdTime_Max": 1383177600000,
        "Variables": "heatmax_ssp126,heatmax_ssp245,heatmax_ssp370,heatmax_ssp585",
        "Dimensions": "StdTime",
        "heatmax_ssp245": "86.392776489",
        "heatmax_ssp370": "86.392776489",
        "heatmax_ssp585": "86.392776489"
      }
    },
    {
      "location": {
        "x": -112.068201294649,
        "y": 33.4279757101746,
        "spatialReference": {
          "wkid": 4326,
          "latestWkid": 4326
        }
      },
      "locationId": 0,
      "value": "73.458404541",
      "resolution": 0.25,
      "attributes": {
        "heatmax_ssp126": "73.458404541",
        "StdTime": 1385769600000,
        "StdTime_Max": 1385769600000,
        "Variables": "heatmax_ssp126,heatmax_ssp245,heatmax_ssp370,heatmax_ssp585",
        "Dimensions": "StdTime",
        "heatmax_ssp245": "73.458404541",
        "heatmax_ssp370": "73.458404541",
        "heatmax_ssp585": "73.458404541"
      }
    },
    {
      "location": {
        "x": -112.068201294649,
        "y": 33.4279757101746,
        "spatialReference": {
          "wkid": 4326,
          "latestWkid": 4326
        }
      },
      "locationId": 0,
      "value": "64.231315613",
      "resolution": 0.25,
      "attributes": {
        "heatmax_ssp126": "64.231315613",
        "StdTime": 1388448000000,
        "StdTime_Max": 1388448000000,
        "Variables": "heatmax_ssp126,heatmax_ssp245,heatmax_ssp370,heatmax_ssp585",
        "Dimensions": "StdTime",
        "heatmax_ssp245": "64.231315613",
        "heatmax_ssp370": "64.231315613",
        "heatmax_ssp585": "64.231315613"
      }
    },

Action items

• Clarify the the returned data from the endpoints, so we can adjust the app's UI
• Investigate the missing StdTime values after 2014 in the heatmax_median_multivariate_historical_annual endpoint

[j08lue]

Request

We need to

1. Clarify what to show: tasmax (daily maximum near-surface air temperature; aggregated into annual max?) + precipitation (aggregated how?)
2. Create data services that have accurate names and ideally also short descriptions of what they are showing, so we are never in doubt and can ideally surface those descriptions in the app
3. Make sure the services behave as expected - e.g. that they consistently return stdTime, which is currently not the case (see above)
4. Validate the values against known / expected ranges

Response

https://gis.earthdata.nasa.gov/eic/rest/services

The production server and services have been deployed/published on the minimum recommend AWS instance size. These can be incorporated into the app now. We will continue load testing this week and provide considerations for increasing AWS instance RAM. We would also like to propose restricting access to EIC app and Worldview.

1. Aggregated annual tasmax and annual pr
2. Replicated NEX naming schema; added basic description and variable units not intended to be public - welcome guidance to change
3. Won’t have the inconsistent stdTime issue – these services have historical data appended to each scenario; will explain the issue in a dev sync
4. Validated the service results match the source data… assuming EIC/Alex will validate the source data