Add `gdb` pretty-printers for simple types #11499

upsj · 2022-08-09T17:22:50Z

Description

This adds gdb pretty printers for rmm::device_uvector, thrust::*_vector, thrust::device_reference and cudf::*_span. The implementation is based on NVIDIA/thrust#1631. I will probably backport the thrust-specific changes to there as well, but since the location of the thrust source is not fixed, I'd prefer having all types in a self-contained file.

Checklist

I am familiar with the Contributing Guidelines.
New or existing tests cover these changes.
The documentation is up to date with these changes.

upsj · 2022-08-11T06:19:13Z

This seems to be working for all the cases I tested. The csv reader is a good place to try them out, since it uses host_span, device_span and device_uvector.

harrism · 2022-08-11T21:24:38Z

Is this the right home for all of this? You already added pretty printers for thrust in thrust. Pretty printers for RMM should probably live in RMM so more of RAPIDS can benefit?

upsj · 2022-08-12T05:30:15Z

I believe it would be hard to import dependencies from other components, since they may be placed in an arbitrary location (build/$config/_deps/.../scripts/gdb-pretty-printers.py in rapids-compose if pulling it via CPM, somewhere else if using an existing library). If I wanted to use pretty-printers from the individual files, I'd have to run

source path/to/rmm/scripts/gdb-pretty-printer.py
source path/to/thrust/scripts/gdb-pretty-printer.py
source path/to/cudf/scripts/gdb-pretty-printer.py

manually instead of just sourcing a single file.
All pretty-printers here rely on DeviceIterator and HostIterator, so they need to either pull in the file containing them or we duplicate the code across all of them.

OTOH, you mentioned thrust::*_vector no longer being used inside cudf, so maybe this isn't too much of a roadblock after all?

harrism · 2022-08-15T21:45:19Z

Selfishly, I want to be able to find RMM pretty printers in RMM so that I can use them there and in other libraries. So I guess it's OK to duplicate them in cuDF, but I think they should be added to RMM as well.

codecov · 2022-08-15T23:11:00Z

Codecov Report

❗ No coverage uploaded for pull request base (branch-22.10@65a7821). Click here to learn what that means.
Patch has no changes to coverable lines.

Additional details and impacted files

@@               Coverage Diff               @@
##             branch-22.10   #11499   +/-   ##
===============================================
  Coverage                ?   86.41%           
===============================================
  Files                   ?      145           
  Lines                   ?    22993           
  Branches                ?        0           
===============================================
  Hits                    ?    19870           
  Misses                  ?     3123           
  Partials                ?        0

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

davidwendt · 2022-08-16T11:45:55Z

I'm also in favor of putting these in their respective repos. Perhaps a script can be used to wrap the

I believe it would be hard to import dependencies from other components, since they may be placed in an arbitrary location (build/$config/_deps/.../scripts/gdb-pretty-printers.py in rapids-compose if pulling it via CPM, somewhere else if using an existing library). If I wanted to use pretty-printers from the individual files, I'd have to run
source path/to/rmm/scripts/gdb-pretty-printer.py
source path/to/thrust/scripts/gdb-pretty-printer.py
source path/to/cudf/scripts/gdb-pretty-printer.py
manually instead of just sourcing a single file.

Could we not just have a script to do these calls? I'd rather not have duplicates to maintain.

upsj · 2022-08-16T11:51:43Z

I totally see the issue - we could configure the rmm and thrust path via CMake and create the pretty-printer/load script inside build. Would that be an acceptable solution?

davidwendt · 2022-08-16T11:56:31Z

I totally see the issue - we could configure the rmm and thrust path via CMake and create the pretty-printer/load script inside build. Would that be an acceptable solution?

I think that makes sense. These are only useful if you do a debug build after all.

upsj · 2022-08-17T18:05:18Z

Considering the horrible things I have to do to try and make the deduplication work (concatenating files in CMake! exec(open(...).read()) in Python), I'm very open to hearing alternative approaches. gdb has its own command file syntax that could be used to load the individual files one after the other.

davidwendt · 2022-08-17T18:10:20Z

Considering the horrible things I have to do to try and make the deduplication work (concatenating files in CMake! exec(open(...).read()) in Python), I'm very open to hearing alternative approaches. gdb has its own command file syntax that could be used to load the individual files one after the other.

The gdb approach seems reasonable. Perhaps we could create some documentation on debugging and include a gdb helper script to illustrate how to load/use these.

karthikeyann · 2022-08-26T20:01:00Z

style check cmake-format.............................................................Failed

upsj · 2022-08-26T20:03:14Z

yes, this depends on rapidsai/rmm#1088 being merged, so I removed the incorrect label. I'll update it soon™️

This PR adds a pretty-printer for `device_uvector` and pulls in Thrust pretty-printers that were added in NVIDIA/thrust#1631. CMake provides a convenience script to load all of the pretty-printers, to resolve the duplication concerns raised in rapidsai/cudf#11499. Example output: <details> ``` $ gdb -q gtests/DEVICE_UVECTOR_TEST Reading symbols from gtests/DEVICE_UVECTOR_TEST... (gdb) b cudaMalloc Function "cudaMalloc" not defined. Make breakpoint pending on future shared library load? (y or [n]) y Breakpoint 1 (cudaMalloc) pending. (gdb) run Starting program: /home/nfs/tribizel/rapids/rmm/build/cuda-11.5.0/feature__pretty-printers/debug/gtests/DEVICE_UVECTOR_TEST warning: Error disabling address space randomization: Operation not permitted [Thread debugging using libthread_db enabled] Using host libthread_db library "/usr/lib/x86_64-linux-gnu/libthread_db.so.1". Running main() from gmock_main.cc [==========] Running 95 tests from 5 test suites. [----------] Global test environment set-up. [----------] 19 tests from TypedUVectorTest/0, where TypeParam = signed char [ RUN ] TypedUVectorTest/0.MemoryResource [New Thread 0x7f741efc1000 (LWP 86147)] Thread 1 "DEVICE_UVECTOR_" hit Breakpoint 1, 0x00007f7426dddc80 in cudaMalloc () from /home/nfs/tribizel/rapids/compose/etc/conda/cuda_11.5/envs/rapids/lib/libcudart.so.11.0 (gdb) c Continuing. [New Thread 0x7f741e7c0000 (LWP 86148)] [ OK ] TypedUVectorTest/0.MemoryResource (999 ms) [ RUN ] TypedUVectorTest/0.ZeroSizeConstructor [ OK ] TypedUVectorTest/0.ZeroSizeConstructor (0 ms) [ RUN ] TypedUVectorTest/0.NonZeroSizeConstructor Thread 1 "DEVICE_UVECTOR_" hit Breakpoint 1, 0x00007f7426dddc80 in cudaMalloc () from /home/nfs/tribizel/rapids/compose/etc/conda/cuda_11.5/envs/rapids/lib/libcudart.so.11.0 (gdb) finish Run till exit from #0 0x00007f7426dddc80 in cudaMalloc () from /home/nfs/tribizel/rapids/compose/etc/conda/cuda_11.5/envs/rapids/lib/libcudart.so.11.0 rmm::mr::cuda_memory_resource::do_allocate (bytes=<optimized out>, this=<optimized out>) at /home/nfs/tribizel/rapids/rmm/include/rmm/mr/device/cuda_memory_resource.hpp:70 70 RMM_CUDA_TRY_ALLOC(cudaMalloc(&ptr, bytes)); (gdb) finish Run till exit from #0 rmm::mr::cuda_memory_resource::do_allocate (bytes=<optimized out>, this=<optimized out>) at /home/nfs/tribizel/rapids/rmm/include/rmm/mr/device/cuda_memory_resource.hpp:70 0x00005587874feef0 in rmm::device_buffer::allocate_async (bytes=12345, this=0x7ffefebb46b0) at /home/nfs/tribizel/rapids/rmm/include/rmm/device_buffer.hpp:418 418 _data = (bytes > 0) ? memory_resource()->allocate(bytes, stream()) : nullptr; Value returned is $1 = (void *) 0x7f73ff000000 (gdb) finish Run till exit from #0 0x00005587874feef0 in rmm::device_buffer::allocate_async (bytes=12345, this=0x7ffefebb46b0) at /home/nfs/tribizel/rapids/rmm/include/rmm/device_buffer.hpp:418 TypedUVectorTest_NonZeroSizeConstructor_Test<signed char>::TestBody (this=<optimized out>) at /home/nfs/tribizel/rapids/rmm/tests/device_uvector_tests.cpp:55 55 EXPECT_EQ(vec.size(), size); (gdb) print vec $2 = {_storage = {_data = 0x7f73ff000000, _size = 12345, _capacity = 12345, _stream = {stream_ = 0x0}, _mr = 0x558787561008 <rmm::mr::detail::initial_resource()::mr>}} (gdb) source load-pretty-printers (gdb) print vec $3 = rmm::device_uvector<signed char> of length 12345, capacity 12345 = {0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000'...} (gdb) ``` </details> Authors: - Tobias Ribizel (https://github.com/upsj) Approvers: - Mark Harris (https://github.com/harrism) - Bradley Dice (https://github.com/bdice) URL: #1088

cpp/scripts/gdb-pretty-printers.py

bdice

Nice work!

karthikeyann

LGTM 👍

karthikeyann · 2022-09-08T20:09:04Z

cpp/scripts/gdb-pretty-printers.py

+
+
+def lookup_cudf_type(val):
+    if not str(val.type.unqualified()).startswith("cudf::"):


does this printer work if we use using namespace cudf ?

yes, using is a source-level alias that doesn't affect the generated debug info, afaik.

I can confirm that's the case after some small experiments :)

upsj · 2022-09-09T07:00:39Z

@gpucibot merge

github-actions bot added the libcudf Affects libcudf (C++/CUDA) code. label Aug 9, 2022

upsj changed the title ~~Add gdb pretty-printers~~ Add gdb pretty-printers for simple types Aug 9, 2022

upsj added non-breaking Non-breaking change feature request New feature or request labels Aug 11, 2022

upsj self-assigned this Aug 11, 2022

upsj marked this pull request as ready for review August 11, 2022 06:19

upsj requested a review from a team as a code owner August 11, 2022 06:19

upsj requested review from harrism, karthikeyann, a team, galipremsagar and brandon-b-miller and removed request for a team August 11, 2022 06:19

upsj added the 3 - Ready for Review Ready for review by team label Aug 16, 2022

upsj added 2 - In Progress Currently a work in progress and removed 3 - Ready for Review Ready for review by team labels Aug 16, 2022

upsj mentioned this pull request Aug 17, 2022

Add gdb pretty-printers for rmm types rapidsai/rmm#1088

Merged

3 tasks

upsj added 3 commits August 17, 2022 18:58

add gdb pretty-printer

6c897b8

formatting and cleanup

7350f3d

add copyright header

7998689

rename script

6e0aa55

upsj added 2 - In Progress Currently a work in progress and removed 3 - Ready for Review Ready for review by team labels Aug 26, 2022

upsj added 2 commits August 30, 2022 13:40

formatting

18abcca

remove unnecessary check

b5f1107

upsj added 3 - Ready for Review Ready for review by team and removed 2 - In Progress Currently a work in progress labels Aug 30, 2022

bdice reviewed Aug 30, 2022

View reviewed changes

cpp/scripts/gdb-pretty-printers.py Outdated Show resolved Hide resolved

cpp/scripts/gdb-pretty-printers.py Outdated Show resolved Hide resolved

cpp/scripts/gdb-pretty-printers.py Outdated Show resolved Hide resolved

cpp/scripts/gdb-pretty-printers.py Outdated Show resolved Hide resolved

review updates

950eca5

bdice reviewed Aug 30, 2022

View reviewed changes

cpp/scripts/gdb-pretty-printers.py Outdated Show resolved Hide resolved

bdice reviewed Aug 30, 2022

View reviewed changes

cpp/scripts/gdb-pretty-printers.py Outdated Show resolved Hide resolved

bdice reviewed Aug 30, 2022

View reviewed changes

cpp/scripts/gdb-pretty-printers.py Outdated Show resolved Hide resolved

formatting and output fixes

3debc6e

bdice approved these changes Aug 31, 2022

View reviewed changes

upsj added 4 - Needs Review Waiting for reviewer to review or respond and removed 3 - Ready for Review Ready for review by team labels Sep 5, 2022

upsj removed request for galipremsagar and brandon-b-miller September 5, 2022 14:07

karthikeyann approved these changes Sep 8, 2022

View reviewed changes

upsj added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 4 - Needs Review Waiting for reviewer to review or respond labels Sep 9, 2022

rapids-bot bot merged commit c8f57dd into rapidsai:branch-22.10 Sep 9, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `gdb` pretty-printers for simple types #11499

Add `gdb` pretty-printers for simple types #11499

upsj commented Aug 9, 2022

upsj commented Aug 11, 2022

harrism commented Aug 11, 2022

upsj commented Aug 12, 2022

harrism commented Aug 15, 2022

codecov bot commented Aug 15, 2022 •

edited

Loading

davidwendt commented Aug 16, 2022

upsj commented Aug 16, 2022 •

edited

Loading

davidwendt commented Aug 16, 2022

upsj commented Aug 17, 2022 •

edited

Loading

davidwendt commented Aug 17, 2022

karthikeyann commented Aug 26, 2022

upsj commented Aug 26, 2022 •

edited

Loading

bdice left a comment

karthikeyann left a comment

karthikeyann Sep 8, 2022

upsj Sep 9, 2022

upsj Sep 9, 2022

upsj commented Sep 9, 2022



		def lookup_cudf_type(val):
		if not str(val.type.unqualified()).startswith("cudf::"):

Add gdb pretty-printers for simple types #11499

Add gdb pretty-printers for simple types #11499

Conversation

upsj commented Aug 9, 2022

Description

Checklist

upsj commented Aug 11, 2022

harrism commented Aug 11, 2022

upsj commented Aug 12, 2022

harrism commented Aug 15, 2022

codecov bot commented Aug 15, 2022 • edited Loading

Codecov Report

davidwendt commented Aug 16, 2022

upsj commented Aug 16, 2022 • edited Loading

davidwendt commented Aug 16, 2022

upsj commented Aug 17, 2022 • edited Loading

davidwendt commented Aug 17, 2022

karthikeyann commented Aug 26, 2022

upsj commented Aug 26, 2022 • edited Loading

bdice left a comment

Choose a reason for hiding this comment

karthikeyann left a comment

Choose a reason for hiding this comment

karthikeyann Sep 8, 2022

Choose a reason for hiding this comment

upsj Sep 9, 2022

Choose a reason for hiding this comment

upsj Sep 9, 2022

Choose a reason for hiding this comment

upsj commented Sep 9, 2022

Add `gdb` pretty-printers for simple types #11499

Add `gdb` pretty-printers for simple types #11499

codecov bot commented Aug 15, 2022 •

edited

Loading

upsj commented Aug 16, 2022 •

edited

Loading

upsj commented Aug 17, 2022 •

edited

Loading

upsj commented Aug 26, 2022 •

edited

Loading