Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Databricks CI improvements and support runtime env parameter to xfail certain tests #297

Merged
merged 73 commits into from
Jun 29, 2020
Merged

Databricks CI improvements and support runtime env parameter to xfail certain tests #297

merged 73 commits into from
Jun 29, 2020

Commits on Jun 19, 2020

  1. Add a profile to build for Databricks

    @tgravescs
    tgravescs committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    4f961b2 View commit details
    Browse the repository at this point in the history

  2. move dependency to management section for databricks annoation

    @tgravescs
    tgravescs committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    3874aa5 View commit details
    Browse the repository at this point in the history

  3. Databricks build and deploy scripts

    @tgravescs
    tgravescs committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    4fb2cda View commit details
    Browse the repository at this point in the history

  4. Change the way we match DeserializeToObjectExec because some Spark 

    releases have an extra parameter to it
    @tgravescs
    tgravescs committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    d3f73a1 View commit details
    Browse the repository at this point in the history

  5. Update description on jenkins file

    @tgravescs
    tgravescs committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    e6e14e3 View commit details
    Browse the repository at this point in the history

  6. cleanup copyrights

    @tgravescs
    tgravescs committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    320ff12 View commit details
    Browse the repository at this point in the history

  7. remove extra databricks exclude

    @tgravescs
    tgravescs committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    b171ce5 View commit details
    Browse the repository at this point in the history

  8. more cleanup

    @tgravescs
    tgravescs committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    2db2618 View commit details
    Browse the repository at this point in the history

  9. disable slack

    @tgravescs
    tgravescs committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    888c7f8 View commit details
    Browse the repository at this point in the history

  10. remove empty post

    @tgravescs
    tgravescs committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    9fce430 View commit details
    Browse the repository at this point in the history

  11. Try moving agent

    @tgravescs
    tgravescs committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    0e8f41f View commit details
    Browse the repository at this point in the history

  12. remove agent from post

    @tgravescs
    tgravescs committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    986393f View commit details
    Browse the repository at this point in the history

  13. change to use docker for entire

    @tgravescs
    tgravescs committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    d1fae38 View commit details
    Browse the repository at this point in the history

  14. right file

    @tgravescs
    tgravescs committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    985cbb6 View commit details
    Browse the repository at this point in the history

  15. remove docker file from in post

    @tgravescs
    tgravescs committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    6e03e85 View commit details
    Browse the repository at this point in the history

  16. try global docker file with post

    @tgravescs
    tgravescs committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    8fce933 View commit details
    Browse the repository at this point in the history

  17. remove docker agent

    @tgravescs
    tgravescs committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    bb1cd1d View commit details
    Browse the repository at this point in the history

  18. try using image

    @tgravescs
    tgravescs committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    127f5ab View commit details
    Browse the repository at this point in the history

  19. use docker instead of dockerfile

    @tgravescs
    tgravescs committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    28e8ae5 View commit details
    Browse the repository at this point in the history

  20. fix build_version

    @tgravescs
    tgravescs committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    c97cf8d View commit details
    Browse the repository at this point in the history

Commits on Jun 22, 2020

  1. parameterize builds and try subprocess

    @tgravescs
    tgravescs committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    4b86e8e View commit details
    Browse the repository at this point in the history

  2. add back deploy

    @tgravescs
    tgravescs committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    fe15d39 View commit details
    Browse the repository at this point in the history

  3. update comment

    @tgravescs
    tgravescs committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    e4fb805 View commit details
    Browse the repository at this point in the history

  4. add quotes

    @tgravescs
    tgravescs committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    64d1df7 View commit details
    Browse the repository at this point in the history

  5. use os.system

    @tgravescs
    tgravescs committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    b2b3a61 View commit details
    Browse the repository at this point in the history

  6. move python to bash script

    @tgravescs
    tgravescs committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    45a2166 View commit details
    Browse the repository at this point in the history

  7. pass in more variables

    @tgravescs
    tgravescs committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    277a214 View commit details
    Browse the repository at this point in the history

  8. fix syntax

    @tgravescs
    tgravescs committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    2f8e2c7 View commit details
    Browse the repository at this point in the history

  9. fix version

    @tgravescs
    tgravescs committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    d9d44e8 View commit details
    Browse the repository at this point in the history

  10. fix spark version

    @tgravescs
    tgravescs committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    094b9eb View commit details
    Browse the repository at this point in the history

  11. Fix missing space

    @tgravescs
    tgravescs committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    8fbb899 View commit details
    Browse the repository at this point in the history

  12. set spark conf dir so we don't pick up databricks confs

    @tgravescs
    tgravescs committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    877a020 View commit details
    Browse the repository at this point in the history

Commits on Jun 23, 2020

  1. change to pass parameters to python

    @tgravescs
    tgravescs committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    ae5e879 View commit details
    Browse the repository at this point in the history

  2. Switch to check_call isntead of os.system

    @tgravescs
    tgravescs committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    512d101 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'branch-0.2' of github.com:NVIDIA/spark-rapids into dbfo… 

    …llowup
    @tgravescs
    tgravescs committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    722c9b6 View commit details
    Browse the repository at this point in the history

  4. Update release file for databricks

    @tgravescs
    tgravescs committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    b456cc0 View commit details
    Browse the repository at this point in the history

  5. Tests failure handling

    @tgravescs
    tgravescs committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    d12f5de View commit details
    Browse the repository at this point in the history

  6. remove duplicate profiles

    @tgravescs
    tgravescs committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    2244d7f View commit details
    Browse the repository at this point in the history

  7. remove slack notifications

    @tgravescs
    tgravescs committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    139eadd View commit details
    Browse the repository at this point in the history

  8. use 0.14 cudf for now

    @tgravescs
    tgravescs committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    0be7c94 View commit details
    Browse the repository at this point in the history

  9. Try exiting with pipestatus

    @tgravescs
    tgravescs committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    38bce86 View commit details
    Browse the repository at this point in the history

  10. Explicit check pipestatus from running tests

    @tgravescs
    tgravescs committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    f8d6592 View commit details
    Browse the repository at this point in the history

  11. don't pip to out on running tests

    @tgravescs
    tgravescs committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    9391a77 View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2020

  1. Add in xfail options and runtime_env

    @tgravescs
    tgravescs committed Jun 24, 2020
    Configuration menu
    Copy the full SHA
    254118a View commit details
    Browse the repository at this point in the history

  2. Update runtime env checks

    @tgravescs
    tgravescs committed Jun 24, 2020
    Configuration menu
    Copy the full SHA
    b7fcf4a View commit details
    Browse the repository at this point in the history

  3. remove extra runtime_env fixture 

    w
    @tgravescs
    tgravescs committed Jun 24, 2020
    Configuration menu
    Copy the full SHA
    9bc70dd View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2020

  1. handle pipstatus properly

    @tgravescs
    tgravescs committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    666c94c View commit details
    Browse the repository at this point in the history

  2. Fix quotes

    @tgravescs
    tgravescs committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    074fef7 View commit details
    Browse the repository at this point in the history

  3. always pass post

    @tgravescs
    tgravescs committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    5798352 View commit details
    Browse the repository at this point in the history

  4. Update to download cudf version

    @tgravescs
    tgravescs committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    af8a0e0 View commit details
    Browse the repository at this point in the history

  5. Fix cudf version poarameter number

    @tgravescs
    tgravescs committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    a89e28e View commit details
    Browse the repository at this point in the history

  6. Fixes

    @tgravescs
    tgravescs committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    33a96bd View commit details
    Browse the repository at this point in the history

  7. ichange order mvn get cudfY

    @tgravescs
    tgravescs committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    091fa5f View commit details
    Browse the repository at this point in the history

  8. print rat

    @tgravescs
    tgravescs committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    49bd0d9 View commit details
    Browse the repository at this point in the history

  9. move download back after clean

    @tgravescs
    tgravescs committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    8359140 View commit details
    Browse the repository at this point in the history

  10. Fix rat

    @tgravescs
    tgravescs committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    f769fd2 View commit details
    Browse the repository at this point in the history

  11. fix removals

    @tgravescs
    tgravescs committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    25057e6 View commit details
    Browse the repository at this point in the history

  12. remove files for rat

    @tgravescs
    tgravescs committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    e4e3b02 View commit details
    Browse the repository at this point in the history

  13. ichange from mvn to git

    @tgravescs
    tgravescs committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    539eb88 View commit details
    Browse the repository at this point in the history

  14. ichange to package from verify

    @tgravescs
    tgravescs committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    790fdb5 View commit details
    Browse the repository at this point in the history

  15. iAdd cuda version

    @tgravescs
    tgravescs committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    965783c View commit details
    Browse the repository at this point in the history

  16. ifix location of m2dir

    @tgravescs
    tgravescs committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    1d12fe8 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2020

  1. Merge remote-tracking branch 'origin/branch-0.2' into dbfollowup

    @tgravescs
    tgravescs committed Jun 26, 2020
    Configuration menu
    Copy the full SHA
    10f5dae View commit details
    Browse the repository at this point in the history

  2. Make rat consoleoutput configurable

    @tgravescs
    tgravescs committed Jun 26, 2020
    Configuration menu
    Copy the full SHA
    5a97640 View commit details
    Browse the repository at this point in the history

  3. Update patch

    @tgravescs
    tgravescs committed Jun 26, 2020
    Configuration menu
    Copy the full SHA
    2ffa143 View commit details
    Browse the repository at this point in the history

  4. fix extra +

    @tgravescs
    tgravescs committed Jun 26, 2020
    Configuration menu
    Copy the full SHA
    aed02b9 View commit details
    Browse the repository at this point in the history

  5. Update to copy cudf jar to databricks jar loc

    @tgravescs
    tgravescs committed Jun 26, 2020
    Configuration menu
    Copy the full SHA
    95bd698 View commit details
    Browse the repository at this point in the history

  6. Update databricks ci cluster jars

    @tgravescs
    tgravescs committed Jun 26, 2020
    Configuration menu
    Copy the full SHA
    c37142c View commit details
    Browse the repository at this point in the history

  7. Change back

    @tgravescs
    tgravescs committed Jun 26, 2020
    Configuration menu
    Copy the full SHA
    2b27fbd View commit details
    Browse the repository at this point in the history

  8. make cluster id configurable

    @tgravescs
    tgravescs committed Jun 26, 2020
    Configuration menu
    Copy the full SHA
    b6c2729 View commit details
    Browse the repository at this point in the history

  9. fix clusteerid parameter

    @tgravescs
    tgravescs committed Jun 26, 2020
    Configuration menu
    Copy the full SHA
    6ff10ce View commit details
    Browse the repository at this point in the history

  10. Merge remote-tracking branch 'origin/branch-0.2' into dbfollowup

    @tgravescs
    tgravescs committed Jun 26, 2020
    Configuration menu
    Copy the full SHA
    b680220 View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2020

  1. Change xfail to be annotation

    @tgravescs
    tgravescs committed Jun 29, 2020
    Configuration menu
    Copy the full SHA
    91d3c8e View commit details
    Browse the repository at this point in the history