Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Params for build and test CI scripts on Databricks #9904

Merged
merged 2 commits into from
Dec 3, 2023

Conversation

NvTimLiu
Copy link
Collaborator

Params for build and test CI scripts on Databricks

To fix #9903

Add params to support specify build, test args on Databricks for CI scripts, e.g.

python jenkins/databricks/run-build.py -e SKIP_DEP_INSTALL=1 : Run maven build on Databricks and skip installing dependencies into maven repo

python jenkins/databricks/run-test.py -e TEST_MODE=DELTA_LAKE_ONLY : only run Delta Lake test cases on Databricks

Signed-off-by: timl@nvidia.com

To fix NVIDIA#9903

Add params to support specify build, test args on Databricks for CI scripts, e.g.

   `python jenkins/databricks/run-build.py -e SKIP_DEP_INSTALL=1` : Run maven build on Databricks and skip installing dependencies into maven repo

   `python jenkins/databricks/run-test.py -e TEST_MODE=DELTA_LAKE_ONLY` : only run Delta Lake  test cases on Databricks

Signed-off-by: Tim Liu <timl@nvidia.com>
@NvTimLiu NvTimLiu self-assigned this Nov 30, 2023
@NvTimLiu NvTimLiu added the build Related to CI / CD or cleanly building label Nov 30, 2023
jlowe
jlowe previously approved these changes Nov 30, 2023
tgravescs
tgravescs previously approved these changes Nov 30, 2023
@razajafri
Copy link
Collaborator

razajafri commented Nov 30, 2023

Is there a way to test this from blossom jobs-for-developers?

pxLi
pxLi previously approved these changes Dec 1, 2023
@NvTimLiu
Copy link
Collaborator Author

NvTimLiu commented Dec 1, 2023

Is there a way to test this from blossom jobs-for-developers?

Yes verified it from dev pipeline, I'll add a parameter(e.g. params.EXTRA_ENVS: "TEST_MODE=DELTA_LAKE_ONLY in the dev CI pipeline after this PR get merged

Signed-off-by: Tim Liu <timl@nvidia.com>
@NvTimLiu NvTimLiu dismissed stale reviews from pxLi, tgravescs, and jlowe via 54be175 December 1, 2023 05:35
@NvTimLiu
Copy link
Collaborator Author

NvTimLiu commented Dec 1, 2023

build

@pxLi
Copy link
Collaborator

pxLi commented Dec 1, 2023

pre-merge failed #9916

@pxLi
Copy link
Collaborator

pxLi commented Dec 1, 2023

build

@pxLi
Copy link
Collaborator

pxLi commented Dec 1, 2023

re-kicked after merging the fix

@NvTimLiu NvTimLiu merged commit 557680b into NVIDIA:branch-24.02 Dec 3, 2023
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Related to CI / CD or cleanly building
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[TASK] Add params to support specify test cases for Databricks CI scripts
5 participants