Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix miscellaneous integ tests for Spark 4 [databricks] #11097

Merged
merged 2 commits into from
Jul 3, 2024

Conversation

mythrocks
Copy link
Collaborator

Fixes #11020. (grouping_sets_test.py)
Fixes #11023. (dpp_test.py)
Fixes #11025. (date_time_test.py)
Fixes #11026. (map_test.py)

This commit prepares miscellaneous integration tests to be run on Spark 4.

Certain integration tests fail on Spark 4 because of ANSI mode being enabled by default. This commit disables ANSI on the failing tests, or introduces other fixes so that the tests may pass correctly.

Fixes NVIDIA#11020. (grouping_sets_test.py)
Fixes NVIDIA#11023. (dpp_test.py)
Fixes NVIDIA#11025. (date_time_test.py)
Fixes NVIDIA#11026. (map_test.py)

This commit prepares miscellaneous integration tests to be run on Spark
4.

Certain integration tests fail on Spark 4 because of ANSI mode being
enabled by default.  This commit disables ANSI on the failing tests, or
introduces other fixes so that the tests may pass correctly.

Signed-off-by: MithunR <mithunr@nvidia.com>
@mythrocks mythrocks self-assigned this Jun 25, 2024
@mythrocks mythrocks changed the title Prep miscellaneous integ tests for Spark 4 Fix miscellaneous integ tests for Spark 4 Jun 25, 2024
@mythrocks mythrocks changed the title Fix miscellaneous integ tests for Spark 4 Fix miscellaneous integ tests for Spark 4 [databricks] Jun 25, 2024
@mythrocks
Copy link
Collaborator Author

Build

@mythrocks mythrocks requested a review from razajafri June 26, 2024 17:29
@sameerz sameerz added test Only impacts tests Spark 4.0+ Spark 4.0+ issues labels Jul 1, 2024
@mythrocks mythrocks requested a review from jlowe July 3, 2024 18:43
@mythrocks
Copy link
Collaborator Author

Build

@mythrocks mythrocks merged commit 6fa51ba into NVIDIA:branch-24.08 Jul 3, 2024
45 checks passed
@mythrocks
Copy link
Collaborator Author

This change has been merged. Thank you for reviewing again, @jlowe.

@mythrocks mythrocks deleted the spark4-misc-integ-tests branch July 3, 2024 23:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Spark 4.0+ Spark 4.0+ issues test Only impacts tests
Projects
None yet
3 participants