Qualification tool: Check for metadata being null #3285
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reported by a user, but haven't been able to reproduce:
21/08/20 09:55:31 WARN QualAppInfo: Got unexpected exception processing file: application_1626709192160_199049.lz4
java.lang.NullPointerException
at org.apache.spark.sql.rapids.tool.AppBase.getPlanMetaWithSchema(AppBase.scala:140)
at org.apache.spark.sql.rapids.tool.AppBase.checkMetadataForReadSchema(AppBase.scala:155)
at org.apache.spark.sql.rapids.tool.qualification.QualAppInfo.processSQLPlan(QualAppInfo.scala:218)
at org.apache.spark.sql.rapids.tool.qualification.QualEventProcessor.doSparkListenerSQLExecutionStart(QualEventProcessor.scala:90)
Signed-off-by: Thomas Graves tgraves@nvidia.com