Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] TPC-DS query 93 failing with UnsupportedOperationException #1280

Closed
jlowe opened this issue Dec 4, 2020 · 3 comments
Closed

[BUG] TPC-DS query 93 failing with UnsupportedOperationException #1280

jlowe opened this issue Dec 4, 2020 · 3 comments
Assignees
Labels
bug Something isn't working P0 Must have for release

Comments

@jlowe
Copy link
Member

jlowe commented Dec 4, 2020

Running TPC-DS query 93 with decimals on a recent 0.3-SNAPSHOT plugin build failed with this stacktrace:

Job aborted due to stage failure: Task 105 in stage 32.0 failed 4 times, most recent failure: Lost task 105.3 in stage 32.0 (TID 12228, 10.150.179.152, executor 4): java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainIntegerDictionary
	at org.apache.parquet.column.Dictionary.decodeToLong(Dictionary.java:49)
	at org.apache.spark.sql.execution.datasources.parquet.ParquetDictionary.decodeToLong(ParquetDictionary.java:36)
	at org.apache.spark.sql.execution.vectorized.OnHeapColumnVector.getLong(OnHeapColumnVector.java:364)
	at org.apache.spark.sql.execution.vectorized.WritableColumnVector.getDecimal(WritableColumnVector.java:367)
	at com.nvidia.spark.rapids.HostColumnarToGpu$.$anonfun$columnarCopy$16(HostColumnarToGpu.scala:130)
	at com.nvidia.spark.rapids.HostColumnarToGpu$.$anonfun$columnarCopy$16$adapted(HostColumnarToGpu.scala:126)
	at scala.collection.immutable.Range.foreach(Range.scala:158)
	at com.nvidia.spark.rapids.HostColumnarToGpu$.columnarCopy(HostColumnarToGpu.scala:126)
	at com.nvidia.spark.rapids.HostToGpuCoalesceIterator.$anonfun$addBatchToConcat$1(HostColumnarToGpu.scala:203)
	at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:158)
	at com.nvidia.spark.rapids.HostToGpuCoalesceIterator.addBatchToConcat(HostColumnarToGpu.scala:201)
	at com.nvidia.spark.rapids.AbstractGpuCoalesceIterator.addBatch(GpuCoalesceBatches.scala:370)
	at com.nvidia.spark.rapids.AbstractGpuCoalesceIterator.$anonfun$next$1(GpuCoalesceBatches.scala:285)
	at com.nvidia.spark.rapids.Arm.withResource(Arm.scala:28)
	at com.nvidia.spark.rapids.Arm.withResource$(Arm.scala:26)
	at com.nvidia.spark.rapids.AbstractGpuCoalesceIterator.withResource(GpuCoalesceBatches.scala:132)
	at com.nvidia.spark.rapids.AbstractGpuCoalesceIterator.next(GpuCoalesceBatches.scala:265)
	at com.nvidia.spark.rapids.AbstractGpuCoalesceIterator.next(GpuCoalesceBatches.scala:132)
	at scala.collection.Iterator$$anon$10.next(Iterator.scala:459)
	at com.nvidia.spark.rapids.ColumnarToRowIterator.loadNextBatch(GpuColumnarToRowExec.scala:177)
	at com.nvidia.spark.rapids.ColumnarToRowIterator.hasNext(GpuColumnarToRowExec.scala:210)
	at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:458)
	at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:458)
	at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:132)
	at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:52)
	at org.apache.spark.scheduler.Task.run(Task.scala:127)
	at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:446)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1377)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:449)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
@jlowe jlowe added bug Something isn't working ? - Needs Triage Need team to review and classify labels Dec 4, 2020
@jlowe
Copy link
Member Author

jlowe commented Dec 4, 2020

This looks similar to #1279, but not sure if it's the same root cause.

@sameerz
Copy link
Collaborator

sameerz commented Dec 8, 2020

Potentially a dupe of #1298

@sameerz sameerz added P0 Must have for release and removed ? - Needs Triage Need team to review and classify labels Dec 8, 2020
@sameerz sameerz added this to the Dec 7 - Dec 18 milestone Dec 8, 2020
@jlowe
Copy link
Member Author

jlowe commented Dec 9, 2020

This is fixed by #1278

@jlowe jlowe closed this as completed Dec 9, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working P0 Must have for release
Projects
None yet
Development

No branches or pull requests

2 participants