Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Fix misleading "cannot run on GPU" warnings when AQE is enabled #607

Closed
andygrove opened this issue Aug 25, 2020 · 0 comments · Fixed by #647
Closed

[BUG] Fix misleading "cannot run on GPU" warnings when AQE is enabled #607

andygrove opened this issue Aug 25, 2020 · 0 comments · Fixed by #647
Assignees
Labels
bug Something isn't working

Comments

@andygrove
Copy link
Contributor

andygrove commented Aug 25, 2020

Describe the bug

With AQE enabled, the query plan gets optimized multiple times by the plugin, and once the plan contains query stages that have already been executed, we see the following warnings. This doesn't cause any incorrect behavior but is misleading since these query stages likely already executed on the GPU.

!NOT_FOUND <ShuffleQueryStageExec> cannot run on GPU because no GPU enabled version of operator class org.apache.spark.sql.execution.adaptive.ShuffleQueryStageExec could be found
!NOT_FOUND <BroadcastQueryStageExec> cannot run on GPU because no GPU enabled version of operator class org.apache.spark.sql.execution.adaptive.BroadcastQueryStageExec could be found
!NOT_FOUND <AdaptiveSparkPlanExec> cannot run on GPU because no GPU enabled version of operator class org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec could be found

Steps/Code to reproduce bug
Execute any query containing broadcast or shuffle exchanges with AQE enabled.

Expected behavior
We should not see these warnings.

Environment details (please complete the following information)
NA

Additional context
N/A

@andygrove andygrove added bug Something isn't working ? - Needs Triage Need team to review and classify labels Aug 25, 2020
@andygrove andygrove added this to the Aug 17 - Aug 28 milestone Aug 25, 2020
@andygrove andygrove self-assigned this Aug 25, 2020
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label Aug 25, 2020
tgravescs pushed a commit to tgravescs/spark-rapids that referenced this issue Nov 30, 2023
Signed-off-by: Matt Ahrens <matthewahrens@gmail.com>

Signed-off-by: Matt Ahrens <matthewahrens@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants