Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

handle cupy array in spread and dynspread functions #1015

Merged
merged 2 commits into from
Apr 5, 2022

Conversation

AjayThorve
Copy link
Contributor

This PR aims to fix an issue where cuDF based aggregates when passed to spread & dynspread, throw a typing error.

import datashader as ds
from datashader.transfer_functions import spread, dynspread
import cudf
import numpy as np

df = cudf.DataFrame(np.random.multivariate_normal((0,0), [[0.1, 0.1], [0.1, 1.0]], (1000000,)), columns=['x', 'y'])
dynspread(ds.Canvas().points(df,x='x', y='y')) #or spread(ds.Canvas().points(df,x='x', y='y'))
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
non-precise type pyobject
During: typing of argument at datashader/datashader/transfer_functions/__init__.py (775)

File "../datashader/datashader/transfer_functions/__init__.py", line 775:
def _array_density(arr, float_type, px=1):
    <source elided>
    """
    M, N = arr.shape
    ^

This error may have been caused by the following argument(s):
- argument 0: Cannot determine Numba type of <class 'cupy._core.core.ndarray'>

As per the suggestion made in the corresponding holoviews issue, just converting the aggregate img.data to a numpy from cupy before calling the @nb.jit _array_utils() function seems to fix the issue.

@AjayThorve
Copy link
Contributor Author

AjayThorve commented Sep 28, 2021

seems like the failing CI is unrelated to this PR, is there anything that can be done to facilitate the merge? would be happy to do it!

@jbednar
Copy link
Member

jbednar commented Sep 29, 2021

Thanks, @AjayThorve . Agreed; the test failures don't look related. I don't think there's anything for you to do as the author of this PR; the fixes need to be made on master. #1022 might address those, but I've been trying to pay attention to my duties on other libraries and the wheel hasn't spun back around to Datashader just yet. It's overdue for a review of the outstanding PRs and issues and a new release. Sorry about the delay! I'll see if 1022 makes the tests pass, then review the changes there, then come back to this PR...

@maximlt maximlt added this to the v0.13.1 milestone Nov 29, 2021
@jbednar jbednar merged commit a324164 into holoviz:master Apr 5, 2022
@jbednar jbednar modified the milestones: v0.14.1, 0.14.0 Apr 6, 2022
@AjayThorve AjayThorve deleted the fix/spread-fix-for-cudf branch April 3, 2023 16:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants