Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace raw streams with rmm::cuda_stream_view (part 1) #6646

Merged
merged 21 commits into from
Nov 13, 2020

Conversation

harrism
Copy link
Member

@harrism harrism commented Nov 3, 2020

Converting libcudf to use rmm::cuda_stream_view will require a LOT of changes, so I'm splitting it into multiple PRs to ease reviewing. This is the first PR in the series. This series of PRs will

This first PR converts:

  • column.hpp (and source)
  • device_column_view.cuh
  • copying.hpp (and source) : moves functions that had streams in public APIs to namespace detail and adds streamless public versions.
  • null_mask.hpp (and source) : moves functions that had streams in public APIs to namespace detail and adds streamless public versions.
  • AST (transform)
  • Usages of the above APIs in other source files
  • Some benchmarks

Contributes to #6645 and #5119

Depends on #6732.

@harrism harrism added 3 - Ready for Review Ready for review by team libcudf Affects libcudf (C++/CUDA) code. RMM labels Nov 3, 2020
@harrism harrism requested a review from a team as a code owner November 3, 2020 01:26
@harrism harrism self-assigned this Nov 3, 2020
@GPUtester
Copy link
Collaborator

Please update the changelog in order to start CI tests.

View the gpuCI docs here.

@codecov
Copy link

codecov bot commented Nov 3, 2020

Codecov Report

Merging #6646 (4a4be90) into branch-0.17 (00e073b) will decrease coverage by 0.47%.
The diff coverage is n/a.

Impacted file tree graph

@@               Coverage Diff               @@
##           branch-0.17    #6646      +/-   ##
===============================================
- Coverage        82.27%   81.79%   -0.48%     
===============================================
  Files               94       95       +1     
  Lines            15533    16320     +787     
===============================================
+ Hits             12779    13349     +570     
- Misses            2754     2971     +217     
Impacted Files Coverage Δ
python/cudf/cudf/_fuzz_testing/utils.py 0.00% <0.00%> (ø)
python/cudf/cudf/_fuzz_testing/orc.py 0.00% <0.00%> (ø)
python/cudf/cudf/utils/utils.py 83.13% <0.00%> (+0.50%) ⬆️
python/cudf/cudf/core/index.py 93.62% <0.00%> (+1.05%) ⬆️
python/cudf/cudf/core/window/rolling.py 91.20% <0.00%> (+2.97%) ⬆️
python/dask_cudf/dask_cudf/core.py 77.75% <0.00%> (+3.40%) ⬆️
python/cudf/cudf/testing/testing.py 85.00% <0.00%> (+4.13%) ⬆️
python/cudf/cudf/comm/gpuarrow.py 84.68% <0.00%> (+4.92%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 00e073b...4a4be90. Read the comment docs.

Copy link
Contributor

@devavret devavret left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two concerns:

  1. Discussion on default stream param in detail APIs
  2. Header include location and style

cpp/benchmarks/copying/shift_benchmark.cu Show resolved Hide resolved
cpp/include/cudf/ast/detail/transform.cuh Show resolved Hide resolved
cpp/include/cudf/detail/copy_if_else.cuh Show resolved Hide resolved
cpp/include/cudf/detail/null_mask.hpp Show resolved Hide resolved
cpp/include/cudf/detail/null_mask.hpp Outdated Show resolved Hide resolved
cpp/include/cudf/ast/detail/transform.cuh Outdated Show resolved Hide resolved
cpp/include/cudf/detail/valid_if.cuh Outdated Show resolved Hide resolved
cpp/include/cudf/scalar/scalar.hpp Outdated Show resolved Hide resolved
cpp/include/cudf/strings/detail/copy_if_else.cuh Outdated Show resolved Hide resolved
@harrism
Copy link
Member Author

harrism commented Nov 11, 2020

I moved the CMake changes out to #6732

@harrism harrism requested a review from a team as a code owner November 11, 2020 01:47
@harrism
Copy link
Member Author

harrism commented Nov 11, 2020

This PR doesn't actually need java-codeowners or cmake-codeowners reviews, those changes have been merged separately.

Copy link
Member

@jlowe jlowe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no-op Java approval

@harrism
Copy link
Member Author

harrism commented Nov 11, 2020

Yes: I need to test that we don't break other RAPIDS libraries (cuGraph, cuML, cuSpatial). Also, I am surprised by the fact that all the python tests pass without changing any Cython.

This seems to not be a problem. Everything builds OK. This is because of the implicit conversion from cudaStream_t constructor provided by rmm::cuda_stream_view.

@harrism harrism added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 5 - Merge After Dependencies labels Nov 11, 2020
@kkraus14 kkraus14 removed the 3 - Ready for Review Ready for review by team label Nov 11, 2020
@kkraus14
Copy link
Collaborator

rerun tests

@harrism
Copy link
Member Author

harrism commented Nov 11, 2020

I can't repro the dask_cudf test failures locally. What's up with those?

@kkraus14
Copy link
Collaborator

rerun tests

1 similar comment
@harrism
Copy link
Member Author

harrism commented Nov 12, 2020

rerun tests

@kkraus14 kkraus14 merged commit dea44d9 into rapidsai:branch-0.17 Nov 13, 2020
harrism added a commit that referenced this pull request Nov 20, 2020
Converting libcudf to use rmm::cuda_stream_view will require a LOT of changes, so I'm splitting it into multiple PRs to ease reviewing. This is the second PR in the series. This series of PRs will

    Replace usage of cudaStream_t with rmm::cuda_stream_view
    Replace usage of 0 or nullptr as a stream identifier with rmm::cuda_stream_default
    Ensure all APIs always order the stream parameter before the memory resource parameter. #5119

Contributes to #6645 and #5119

Depends on #6646 so this PR will look much bigger until that one is merged.

Also fixes #6706 (to_arrow and to_dlpack are not properly synchronized).

This second PR converts:

    table.hpp (and source / dependencies)
    column_factories.hpp (and source / dependencies)
    headers in include/detail/aggregation (and source / dependencies)
    include/detail/groupby/sort_helper.hpp (and source / dependencies)
    include/detail/utilities/cuda.cuh (and dependencies)
    binary ops
    concatenate
    copy_if
    copy_range
    fill
    gather
    get_value
    hash groupby
    quantiles
    reductions
    repeat
    replace
    reshape
    round
    scatter
    search
    sequence
    sorting
    stream compaction
harrism added a commit to rapidsai/cuspatial that referenced this pull request Nov 24, 2020
This PR converts all usage of cudaStream_t in cuSpatial to rmm::cuda_stream_view, following on from rapidsai/cudf#6646 and rapidsai/cudf#6648

Also reorders stream parameters to occur before MR parameters in all functions.
kkraus14 pushed a commit that referenced this pull request Nov 24, 2020
Converting libcudf to use `rmm::cuda_stream_view` will require a LOT of changes, so I'm splitting it into multiple PRs to ease reviewing. This is the third PR in the series. This series of PRs will

- Replace usage of `cudaStream_t` with `rmm::cuda_stream_view`
- Replace usage of `0` or `nullptr` as a stream identifier with `rmm::cuda_stream_default`
- Ensure all APIs always order the stream parameter before the memory resource parameter. #5119

Contributes to #6645 and #5119

Depends on #6646 and #6648 so this PR will look much bigger until they are merged.

This third PR converts:

 - remaining dictionary functionality
 - cuio
 - lists
 - scalar
 - strings
 - groupby
 - join
 - contiguous_split
 - get_element
 - datetime_ops
 - extract
 - merge
 - partitioning
 - minmax reduction
 - scan
 - byte_cast
 - clamp
 - interleave_columns
 - is_sorted
 - groupby
 - rank
 - tests
 - concurrent map classes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 - Ready to Merge Testing and reviews complete, ready to merge libcudf Affects libcudf (C++/CUDA) code.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants