Skip to content

Commit

Permalink
Change test_single_sort_in_part to print source data frame on failure (
Browse files Browse the repository at this point in the history
…NVIDIA#2571)

* Change test_single_sort_in_part to print source data frame on failure

Signed-off-by: Alessandro Bellina <abellina@nvidia.com>

* Simplify test change

* Add a comment pointing to the github issue

* debug_df can be easily chained in place of a source dataframe
  • Loading branch information
abellina authored Jun 4, 2021
1 parent 2af2457 commit 1ba105e
Showing 1 changed file with 4 additions and 3 deletions.
7 changes: 4 additions & 3 deletions integration_tests/src/main/python/sort_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,10 +105,11 @@ def test_single_nested_orderby_with_limit(data_gen, order):
@pytest.mark.parametrize('data_gen', orderable_gens + orderable_not_null_gen, ids=idfn)
@pytest.mark.parametrize('order', [f.col('a').asc(), f.col('a').asc_nulls_last(), f.col('a').desc(), f.col('a').desc_nulls_first()], ids=idfn)
def test_single_sort_in_part(data_gen, order):
# This outputs the source data frame each time to debug intermittent test
# failures as documented here: https://github.com/NVIDIA/spark-rapids/issues/2477
assert_gpu_and_cpu_are_equal_collect(
lambda spark : unary_op_df(spark, data_gen).sortWithinPartitions(order),
conf = allow_negative_scale_of_decimal_conf)

lambda spark : debug_df(unary_op_df(spark, data_gen)).sortWithinPartitions(order),
conf = allow_negative_scale_of_decimal_conf)

@pytest.mark.parametrize('data_gen', [all_basic_struct_gen], ids=idfn)
@pytest.mark.parametrize('order', [
Expand Down

0 comments on commit 1ba105e

Please sign in to comment.