Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate perfio.s3 reader [databricks] #10438

Merged
merged 1 commit into from
Feb 17, 2024

Conversation

gerashegalov
Copy link
Collaborator

@gerashegalov gerashegalov commented Feb 16, 2024

  • Provides an option spark.rapids.perfio.s3.enabled to improve S3 reads
  • Adds logging for spark-rapids-private version info properties
  • Removes duplicate logging of plugin and cudf versiion properties in checkCudfVersion

Signed-off-by: Gera Shegalov gera@apache.org

Signed-off-by: Gera Shegalov <gera@apache.org>
@gerashegalov gerashegalov self-assigned this Feb 16, 2024
@gerashegalov gerashegalov added the performance A performance related task/issue label Feb 16, 2024
@gerashegalov gerashegalov changed the title Integrate perfio.s3 reader Integrate perfio.s3 reader [databricks] Feb 17, 2024
@gerashegalov gerashegalov marked this pull request as ready for review February 17, 2024 09:30
@gerashegalov
Copy link
Collaborator Author

build

@gerashegalov gerashegalov merged commit 417d798 into NVIDIA:branch-24.04 Feb 17, 2024
40 of 41 checks passed
@gerashegalov gerashegalov deleted the enablePerfIO branch February 20, 2024 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance A performance related task/issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants