Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Column vector leak in conditionals_test.py #4541

Closed
andygrove opened this issue Jan 15, 2022 · 0 comments · Fixed by #4552
Closed

[BUG] Column vector leak in conditionals_test.py #4541

andygrove opened this issue Jan 15, 2022 · 0 comments · Fixed by #4552
Assignees
Labels
bug Something isn't working

Comments

@andygrove
Copy link
Contributor

Describe the bug

I am seeing this in branch-22.02

TEST_PARALLEL=1 ./run_pyspark_from_build.sh -k test_conditional_with_side_effects

...

../../src/main/python/conditionals_test.py::test_conditional_with_side_effects_col_col[Integer] PASSED [ 25%]
../../src/main/python/conditionals_test.py::test_conditional_with_side_effects_col_scalar[Integer] PASSED [ 50%]
../../src/main/python/conditionals_test.py::test_conditional_with_side_effects_cast[String] PASSED [ 75%]
../../src/main/python/conditionals_test.py::test_conditional_with_side_effects_case_when[String] PASSED [100%]

=============================== warnings summary ===============================
../../../../../../../opt/spark-3.0.1-bin-hadoop3.2/python/lib/pyspark.zip/pyspark/sql/context.py:75
  /opt/spark-3.0.1-bin-hadoop3.2/python/lib/pyspark.zip/pyspark/sql/context.py:75: DeprecationWarning: Deprecated in 3.0.0. Use SparkSession.builder.getOrCreate() instead.

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=============== 4 passed, 13803 deselected, 1 warning in 10.10s ================
22/01/15 03:16:39 ERROR ColumnVector: A DEVICE COLUMN VECTOR WAS LEAKED (ID: 6212 7f5e20059a30)
22/01/15 03:16:39 ERROR ColumnVector: A DEVICE COLUMN VECTOR WAS LEAKED (ID: 6660 7f5e2c049070)
22/01/15 03:16:39 ERROR ColumnVector: A DEVICE COLUMN VECTOR WAS LEAKED (ID: 5853 7f5d840681e0)
22/01/15 03:16:39 ERROR ColumnVector: A DEVICE COLUMN VECTOR WAS LEAKED (ID: 9029 7f5df01798c0)
22/01/15 03:16:39 ERROR ColumnVector: A DEVICE COLUMN VECTOR WAS LEAKED (ID: 6944 7f5db4056e70)
22/01/15 03:16:39 ERROR ColumnVector: A DEVICE COLUMN VECTOR WAS LEAKED (ID: 7809 7f5dc4067fc0)
22/01/15 03:16:39 ERROR ColumnVector: A DEVICE COLUMN VECTOR WAS LEAKED (ID: 7015 7f5dd804bcb0)

Steps/Code to reproduce bug
See above

Expected behavior
Should not leak.

Environment details (please complete the following information)
N/A

Additional context
None

@andygrove andygrove added bug Something isn't working ? - Needs Triage Need team to review and classify labels Jan 15, 2022
@andygrove andygrove added this to the Jan 10 - Jan 28 milestone Jan 15, 2022
@andygrove andygrove self-assigned this Jan 15, 2022
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label Jan 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants