Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add RMM devcontainers #1328

Merged
merged 72 commits into from
Sep 26, 2023
Merged

Add RMM devcontainers #1328

merged 72 commits into from
Sep 26, 2023

Conversation

trxcllnt
Copy link
Contributor

@trxcllnt trxcllnt commented Aug 22, 2023 •
edited
Loading

Description

This PR adds some devcontainers to help simplify building the RMM C++ and Python libraries.

It also adds an optional job to the pr.yaml to build the RMM libs in each devcontainer, so the build caches are populated for devs by CI.

A devcontainer can be launched by clicking the "Reopen in Container" button that VSCode shows when opening the repo (or by using the "Rebuild and Reopen in Container" command from the command palette):
image

Clicking this button will cause VSCode to prompt the user to select one of these devcontainer variants:
image

On startup, the devcontainer creates or updates the conda/pip environment using rmm/dependencies.yaml. The envs/package caches are cached on the host via volume mounts, which are described in more detail in .devcontainer/README.md.

The container includes convenience functions to clean, configure, and build the various RMM components:

$ clean-rmm-cpp # only cleans the C++ build dir
$ clean-rmm-python # only cleans the Python build dir
$ clean-rmm # cleans both C++ and Python build dirs

$ configure-rmm-cpp # only configures rmm C++ lib

$ build-rmm-cpp # only builds rmm C++ lib
$ build-rmm-python # only builds rmm Python lib
$ build-rmm # builds both C++ and Python libs
  • The C++ build script is a small wrapper around cmake -S ~/rmm -B ~/rmm/build and cmake --build ~/rmm/build
  • The Python build script is a small wrapper around pip install --editable ~/rmm

Unlike build.sh, these convenience scripts don't install the libraries after building them. Instead, they automatically inject the correct arguments to build the C++ libraries from source and use their build dirs as package roots:

$ cmake -S ~/rmm -B ~/rmm/build
$ CMAKE_ARGS="-Drmm_ROOT=~/rmm/build" \ # <-- this argument is automatic
  pip install -e ~/rmm

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@trxcllnt
add devcontainers
1eaf6b2
@trxcllnt
add compilers to conda dependencies
9e4dfaf
@trxcllnt
fix tag for CUDA 12.0
47ce294
@trxcllnt
use CUDA 11.8 for now
236ca3e
@trxcllnt
default to CUDA 12.0
2c37e73
@trxcllnt
Merge branch 'branch-23.08' of github.com:rapidsai/rmm into fea/devco…
5d5f5eb 
…ntainers
@trxcllnt
update conda environments
7db5386
@trxcllnt
build all RAPIDS archs to take maximum advantage of sccache
73813ce
@trxcllnt
add compile_commands to .gitignore
0e0d6bd
@trxcllnt
add clangd and nsight vscode customizations
ba8e155
@trxcllnt
copy in default clangd config
40e4e07
@trxcllnt
remove options for pip vs. conda unless using the launch script
9670b10
@trxcllnt
fix unified mounts
77306d6
@trxcllnt
define cxx and cuda standard for tests
d0c1717
@trxcllnt
ensure dirs exist before mounting
108c74e
@trxcllnt
add kvikio
decf68e
@trxcllnt
Merge branch 'branch-23.08' of github.com:rapidsai/rmm into fea/devco…
4b4dc41 
…ntainers
@trxcllnt
use volumes for isolated devcontainer source dirs
eecb891
@trxcllnt
update README.md
9f88c95
@trxcllnt
Merge branch 'branch-23.10' of github.com:rapidsai/rmm into fea/devco…
aed0a42 
…ntainers
@trxcllnt
update to rapidsai/devcontainers 23.10
53c7cd3
@trxcllnt
update rapids-build-utils version to 23.10
885fb08
@trxcllnt
update RAPIDS versions in devcontainer files
e7b1bf4
@trxcllnt
add .clangd config file and add clang and clang-tools 16
490b95c
@trxcllnt
Merge branch 'branch-23.10' of github.com:rapidsai/rmm into fea/devco…
5ffa268 
…ntainers
@trxcllnt
Merge branch 'fea/devcontainers' of github.com:trxcllnt/rmm into fea/…
2f2e7dd 
…devcontainers
@trxcllnt
Merge branch 'branch-23.10' of github.com:rapidsai/rmm into fea/devco…
1e11799 
…ntainers
@trxcllnt
remove isolated and unified devcontainers, make single the default
6890e87
@trxcllnt
separate CUDA 11.8 and 12.0 devcontainers
61913f0
@trxcllnt
update conda envs
8a20231
@trxcllnt
Copy link
Contributor Author

/ok to test

@trxcllnt
Copy link
Contributor Author

/ok to test

@trxcllnt
Copy link
Contributor Author

/ok to test

bdice
bdice approved these changes Sep 19, 2023
View reviewed changes
Copy link
Contributor

@bdice bdice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, once we update to clang 16.0.6.

dependencies.yaml Outdated Show resolved Hide resolved
dependencies.yaml Outdated Show resolved Hide resolved
@trxcllnt
Copy link
Contributor Author

/ok to test

@trxcllnt
Copy link
Contributor Author

/ok to test

@trxcllnt
Copy link
Contributor Author

/ok to test

@trxcllnt
Copy link
Contributor Author

/ok to test

@trxcllnt
Copy link
Contributor Author

/ok to test

rapids-bot bot pushed a commit to rapidsai/cuspatial that referenced this pull request Sep 25, 2023
@trxcllnt
Update devcontainers to align with other RAPIDS repositories (#1263)
73412e6 
Removes the unified and isolated devcontainers in favor of just the single one. The unified devcontainers are [still available](https://github.com/rapidsai/devcontainers/blob/d970ac17a894f5d1047fcfd17cffa6d14fe66ec8/.devcontainer/cuda12.0-conda/devcontainer.json#L23-L31) in the [`rapidsai/devcontainers`](https://github.com/rapidsai/devcontainers) repository.


Adds an optional job to the `pr.yaml` to [build the cuGraph libs in each devcontainer](https://github.com/trxcllnt/cuspatial/blob/fea/devcontainers/.github/workflows/pr.yaml#L106-L111), so the build caches are populated for devs by CI.

Other PRs:

* rapidsai/rmm#1328
* rapidsai/kvikio#273
* rapidsai/cudf#14015
* rapidsai/raft#1791
* rapidsai/cumlprims_mg#149
* rapidsai/cuml#5568
* rapidsai/cugraph-ops#538
* rapidsai/cugraph#3838

Authors:
  - Paul Taylor (https://github.com/trxcllnt)
  - H. Thomson Comer (https://github.com/thomcom)

Approvers:
  - Mark Harris (https://github.com/harrism)
  - Jake Awe (https://github.com/AyodeAwe)

URL: #1263
ajschmidt8
ajschmidt8 requested changes Sep 26, 2023
View reviewed changes
Copy link
Member

@ajschmidt8 ajschmidt8 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

requesting some small changes

ci/release/update-version.sh Outdated Show resolved Hide resolved
trxcllnt and others added 2 commits September 26, 2023 10:02
@trxcllnt @ajschmidt8
Update ci/release/update-version.sh
0b3ee78 
Co-authored-by: AJ Schmidt <ajschmidt8@users.noreply.github.com>
@trxcllnt
Merge branch 'branch-23.10' of https://github.com/rapidsai/rmm into f…
a0de1b0 
…ea/devcontainers
@trxcllnt
Copy link
Contributor Author

/ok to test

@harrism
Copy link
Member

harrism commented Sep 26, 2023

/merge

@rapids-bot rapids-bot bot merged commit 7a9451d into rapidsai:branch-23.10 Sep 26, 2023
46 checks passed
@raydouglass raydouglass mentioned this pull request Oct 2, 2023
Merged
rapids-bot bot pushed a commit that referenced this pull request Oct 2, 2023
@raydouglass
Update devcontainers to 23.12 (#1355)
da3ed7b 
Since #1328 was merged after `branch-23.12` was created, the files need to be updated manually in `branch-23.12`.

Authors:
  - Ray Douglass (https://github.com/raydouglass)

Approvers:
  - AJ Schmidt (https://github.com/ajschmidt8)

URL: #1355
@bdice bdice mentioned this pull request Nov 9, 2023
Closed
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@raydouglass raydouglass raydouglass left review comments

@harrism harrism harrism approved these changes

@bdice bdice bdice approved these changes

@ajschmidt8 ajschmidt8 ajschmidt8 approved these changes

Assignees

@trxcllnt trxcllnt

Labels
3 - Ready for review Ready for review by team ci CMake conda cpp Pertains to C++ code improvement Improvement / enhancement to an existing function non-breaking Non-breaking change
Projects
RMM Project Board
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@trxcllnt @harrism @raydouglass @bdice @ajschmidt8