Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing conflicts in branch-0.3 #762

Merged
merged 7 commits into from
Sep 14, 2020

Conversation

abellina
Copy link
Collaborator

Fixes some issues that could not be auto merged into 0.3.

Unit tests pass locally.

abellina and others added 7 commits September 12, 2020 01:19
NVIDIA#732)

* Add a timeout for RapidsShuffleIterator to prevent jobs to hang infinitely in cases where the transport doesn't report an error

Signed-off-by: Alessandro Bellina <abellina@nvidia.com>

* Read the timeout configuration once, and store as a val

Signed-off-by: Alessandro Bellina <abellina@nvidia.com>

* Use spark.network.timeout as our fetch timeout setting

Signed-off-by: Alessandro Bellina <abellina@nvidia.com>

* No really, dont change SparkEnv

Signed-off-by: Alessandro Bellina <abellina@nvidia.com>

* Pass the timeout, to make it easier to mock in tests
…#754)

Signed-off-by: Robert (Bobby) Evans <bobby@apache.org>
Signed-off-by: Firestarman <firestarmanllc@gmail.com>
Signed-off-by: Jason Lowe <jlowe@nvidia.com>
Error log:
=========================== short test summary info ============================
01:45:41 FAILED ../../src/main/python/udf_test.py::test_group_aggregate_udf[Long]
[IGNORE_ORDER, ALLOW_NON_GPU(AggregateInPandasExec,PythonUDF,Alias)]

Signed-off-by: Firestarman <firestarmanllc@gmail.com>
* Add CHANGELOG for release 0.2

Signed-off-by: Peixin Li <pxli@nyu.edu>
@revans2
Copy link
Collaborator

revans2 commented Sep 14, 2020

build

@jlowe
Copy link
Member

jlowe commented Sep 14, 2020

NOTE: This must be merged with a merge-commit, not a squash commit!

@abellina
Copy link
Collaborator Author

The breaking change was a conflict where GpuShuffleEnv.getDeviceStorage was removed by @revans2 in a refactor. I had added an accessor next to it (for the shuffle timeout feature). So I just cleared that conflict, leaving the timeout accessor there.

@abellina
Copy link
Collaborator Author

build

@jlowe jlowe merged commit 34d52d4 into NVIDIA:branch-0.3 Sep 14, 2020
@abellina abellina deleted the branch_3_fix_conflict branch September 14, 2020 17:59
@sameerz sameerz added the build Related to CI / CD or cleanly building label Sep 18, 2020
@sameerz sameerz added this to the Sep 14 - Sep 25 milestone Sep 18, 2020
nartal1 pushed a commit to nartal1/spark-rapids that referenced this pull request Jun 9, 2021
nartal1 pushed a commit to nartal1/spark-rapids that referenced this pull request Jun 9, 2021
tgravescs pushed a commit to tgravescs/spark-rapids that referenced this pull request Nov 30, 2023
…IDIA#762)

Signed-off-by: spark-rapids automation <70000568+nvauto@users.noreply.github.com>

Signed-off-by: spark-rapids automation <70000568+nvauto@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Related to CI / CD or cleanly building
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants