Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] GpuBatchScanExec only reports output row metrics #5615

Closed
jlowe opened this issue May 24, 2022 · 0 comments · Fixed by #5643
Closed

[BUG] GpuBatchScanExec only reports output row metrics #5615

jlowe opened this issue May 24, 2022 · 0 comments · Fixed by #5643
Assignees
Labels
bug Something isn't working

Comments

@jlowe
Copy link
Member

jlowe commented May 24, 2022

Describe the bug
GpuBatchScanExec is supposed to show the same types of GPU-related metrics as FileSourceScanExec (e.g.: buffer time, GPU decode time, output batches, etc.) but output rows are the only metrics shown even with spark.rapids.sql.metrics.level=DEBUG.

Steps/Code to reproduce bug

  • Set spark.rapids.sql.metrics.level to DEBUG
  • Set spark.sql.sources.useV1SourceList to ""
  • Run a simple query to scan a Parquet file (e.g.: spark.read.parquet("tests/src/test/resources/timestamp-date-test.parquet").collect()
  • Examine the SQL UI and note that only output rows are shown as a metric for the GPU batch scan, e.g.:

image

Expected behavior
All of the metrics setup in GpuBatchScanExecMetrics should appear in the SQL UI for the GpuBatchScan node rather than just output rows.

Environment details (please complete the following information)
Replicated the problem with Spark 3.1.2 and Spark 3.2.1

@jlowe jlowe added bug Something isn't working ? - Needs Triage Need team to review and classify labels May 24, 2022
@jlowe jlowe self-assigned this May 24, 2022
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label May 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants