-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement judicious skipping of CI jobs #94
Comments
pytest-incremental may help us with skipping unneeded Python tests: https://pypi.org/project/pytest-incremental/ It hasn't been updated in a while though, it may need some work. |
KyleFromNVIDIA
added a commit
to KyleFromNVIDIA/cudf
that referenced
this issue
Aug 22, 2024
Only run tests based on things that have actually changed. For example, if only Python files have changed, we don't need to run the C++ tests. Contributes to rapidsai/build-planning#94
3 tasks
rapids-bot bot
pushed a commit
to rapidsai/cudf
that referenced
this issue
Aug 27, 2024
Only run tests based on things that have actually changed. For example, if only Python files have changed, we don't need to run the C++ tests. Contributes to rapidsai/build-planning#94 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - Bradley Dice (https://github.com/bdice) - Vyas Ramasubramani (https://github.com/vyasr) - Robert Maynard (https://github.com/robertmaynard) URL: #16642
KyleFromNVIDIA
added a commit
to KyleFromNVIDIA/cugraph
that referenced
this issue
Aug 27, 2024
Only run tests based on things that have actually changed. For example, if only Python files have changed, we don't need to run the C++ tests. Contributes to rapidsai/build-planning#94
This was referenced Aug 27, 2024
rapids-bot bot
pushed a commit
to rapidsai/cudf
that referenced
this issue
Sep 26, 2024
Contributes to rapidsai/build-planning#94 Depends on rapidsai/shared-workflows#239 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) - Vyas Ramasubramani (https://github.com/vyasr) Approvers: - Bradley Dice (https://github.com/bdice) - GALI PREM SAGAR (https://github.com/galipremsagar) URL: #16713
copy-pr-bot bot
pushed a commit
to rapidsai/cudf
that referenced
this issue
Sep 28, 2024
Contributes to rapidsai/build-planning#94 Depends on rapidsai/shared-workflows#239 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) - Vyas Ramasubramani (https://github.com/vyasr) Approvers: - Bradley Dice (https://github.com/bdice) - GALI PREM SAGAR (https://github.com/galipremsagar) URL: #16713
rapids-bot bot
pushed a commit
to rapidsai/cugraph
that referenced
this issue
Oct 1, 2024
Only run tests based on things that have actually changed. For example, if only Python files have changed, we don't need to run the C++ tests. Contributes to rapidsai/build-planning#94 Depends on rapidsai/shared-workflows#239 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - Robert Maynard (https://github.com/robertmaynard) - Jake Awe (https://github.com/AyodeAwe) URL: #4634
@KyleFromNVIDIA now that we have a common shared workflow and about a month of data on how pruning the jobs works, do we want to roll this feature out to the rest of RAPIDS and then close this issue? |
KyleFromNVIDIA
added a commit
to KyleFromNVIDIA/cuml
that referenced
this issue
Oct 3, 2024
KyleFromNVIDIA
added a commit
to KyleFromNVIDIA/raft
that referenced
this issue
Oct 3, 2024
rapids-bot bot
pushed a commit
to rapidsai/cuml
that referenced
this issue
Oct 3, 2024
Contributes to rapidsai/build-planning#94 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - James Lamb (https://github.com/jameslamb) URL: #6094
KyleFromNVIDIA
added a commit
to KyleFromNVIDIA/kvikio
that referenced
this issue
Oct 3, 2024
KyleFromNVIDIA
added a commit
to KyleFromNVIDIA/cuvs
that referenced
this issue
Oct 3, 2024
rapids-bot bot
pushed a commit
to rapidsai/raft
that referenced
this issue
Oct 4, 2024
Contributes to rapidsai/build-planning#94 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - James Lamb (https://github.com/jameslamb) URL: #2466
rapids-bot bot
pushed a commit
to rapidsai/cuvs
that referenced
this issue
Oct 4, 2024
Contributes to rapidsai/build-planning#94 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - James Lamb (https://github.com/jameslamb) URL: #392
KyleFromNVIDIA
added a commit
to KyleFromNVIDIA/ucxx
that referenced
this issue
Oct 4, 2024
KyleFromNVIDIA
added a commit
to KyleFromNVIDIA/cuxfilter
that referenced
this issue
Oct 4, 2024
KyleFromNVIDIA
added a commit
to KyleFromNVIDIA/rmm
that referenced
this issue
Oct 4, 2024
3 tasks
KyleFromNVIDIA
added a commit
to KyleFromNVIDIA/wholegraph
that referenced
this issue
Oct 4, 2024
rapids-bot bot
pushed a commit
to rapidsai/rmm
that referenced
this issue
Oct 4, 2024
Contributes to rapidsai/build-planning#94 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - James Lamb (https://github.com/jameslamb) URL: #1695
rapids-bot bot
pushed a commit
to rapidsai/cuxfilter
that referenced
this issue
Oct 4, 2024
Contributes to rapidsai/build-planning#94 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - James Lamb (https://github.com/jameslamb) URL: #635
rapids-bot bot
pushed a commit
to rapidsai/wholegraph
that referenced
this issue
Oct 4, 2024
Contributes to rapidsai/build-planning#94 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - James Lamb (https://github.com/jameslamb) URL: #226
rapids-bot bot
pushed a commit
to rapidsai/ucxx
that referenced
this issue
Oct 4, 2024
Contributes to rapidsai/build-planning#94 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - James Lamb (https://github.com/jameslamb) URL: #296
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Currently any change to any RAPIDS repo triggers a complete run of the entire build and test suite of CI jobs. This is very expensive, and is often unnecessary. RAPIDS libraries are typically structured such that they have a clear, linear dependency chain between different components. Some examples:
We can reduce the number of unnecessary jobs that we run in CI by more judiciously skipping jobs that are unnecessary. The simplest approach to do this is by simply checking what files have changed. We have previously implemented a form of this in cudf for both cudf.pandas and now for cudf-polars. To do this, I would propose the following steps:
The text was updated successfully, but these errors were encountered: