Correctly orient (y, x) arrays

[ianthomas23]

Candidate fix for #1054. Surprisingly small amount of code changed in the end.

Canvas.raster and tf.shade both accept xr.DataArrays with dims[-2:] == ('y', 'x') and the direction of each of x and y coords may be increasing or decreasing. The orientation of the DataArray returned from these functions should be the same as the input, so if e.g. the input to Canvas.raster is flipped in the x-direction then so will the output be. This keeps the data and coordinates in sync. This was not previously the case, the data was flipped within datashader but the coordinates weren't.

Because the data and its coordinates are synchronised, you can plot the DataArrays in holoviews or hvplot and they are correctly rendered with increasing x and y.

Here are two examples of this working. Firstly the OP's test case of issue #1054 using a GeoTIFF obtained from https://github.com/mommermi/geotiff_sample. The land is on the southeast, the x-coords are increasing and the y-coords are decreasing:

At the top the datashader Image has y-flipped as the input data does because it just draws the image data and ignores the coordinates. Below this it is plotted directly by hvplot and also using datashader which use the coordinates correctly.

Second example to be completely explicit.

There are 4 DataArrays, one for each of the 4 x/y flipping combinations. hv.Image and shade(image) draw them all correctly as the coordinates remain correctly oriented with respect to the data regardless of if they are passed through datashader or not. At the bottom are datashader.Images that are returned from ds.Canvas.raster followed by ds.tf.shade. The images are shown in their flipped orientation as the coordinates are ignored for rendering.

One thing I have not done is consider DataArrays with dims == ('x', 'y'). These are allowed but always interpreted as dims == ('y', 'x') so strange things can result. We will need to do something about this as even some of the reproducers in issue #1054 use dims == ('x', 'y') which made them not very helpful in diagnosing the cause of the problem. I don't think we can forceably transpose them to get ('y', 'x') but we could do some checking and either raise an exception or print a warning. But we need to be careful as we cannot just look for x and y but also need to consider lat, lon, latitude and so on.

I have kept the calc_res function as it is. This returns the x and y resolutions (i.e. spacings) of the specified DataArray, but it chooses to reverse the yresolution. This annoys me a lot as it is unnecessary, but I have left it in case of external libraries that are using this function as it is considered public.

[codecov]

Codecov Report

Merging #1095 (432adf5) into master (de63b66) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #1095   +/-   ##
=======================================
  Coverage   83.36%   83.37%           
=======================================
  Files          34       34           
  Lines        7491     7495    +4     
=======================================
+ Hits         6245     6249    +4     
  Misses       1246     1246           

Impacted Files	Coverage Δ
datashader/transfer_functions/__init__.py	86.61% <100.00%> (ø)
datashader/utils.py	72.78% <100.00%> (+0.37%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update de63b66...432adf5. Read the comment docs.

[maximlt]

Thanks Ian! I haven't dived into the changes but I remembered that some issues might have been related to this bug so tested them with this PR as it is:

• Interactivity error with datashade=True option, image moving opposite to cursor displacement hvplot#739 is fixed 🎉
• datashade a Polygons element renders it upside down with the Matplotlib backend geoviews#541 is not fixed but since it applies to Matplotlib only that certainly points to another error

Sorry for the distraction!

[ianthomas23]

@maximlt I am happy to have accidentally fixed another issue! I claim double credit!

I am not surprised that the polygon/matplotlib issue is not affected as I don't think the polygon code touches anything that this PR changes.

[ianthomas23]

I think we can scratch my concerns about dims=('x', 'y') as the changes here deal with it in a reasonable way. Now that datashader returns DataArrays with the correct coordinates, flipped or not, holoviews renders them correctly regardless of dim order. Here is the second example above but using dims=('x', 'y') rather than ('y', 'x'):

Holoviews output is correct. Datashader output as images ignores the attached coords so is displayed using the conventional increasing ('y', 'x'). This can give you a headache if you are trying to interpret it as shown, but if you pass it on to any other code that cares about the dims then everything turns out fine.

[philippjfr]

Very glad to see this and thanks for the detailed writeup. Also great to see that we didn't have any tests that were testing the wrong result. Overall I'd be in favor of merging this asap and then tagging a dev release so we can test this more easily.

[ianthomas23]

Self merging so we can try it out in dev release.