You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
As part of Remote store segment upload flow, we upload multiple files to remote store directory. Today this upload is sequential and no two files are uploaded parallel. Which could be increasing latency and impacting overall throughput.
Describe the solution you'd like
Explore if parallelization of segment files upload can help improve latency and throughout. Although we need to be mindful to not bombard s3 with requests get throttled. Run performance tests to figure out baseline limits for a cluster/node after this change
Describe alternatives you've considered
NA
Additional context
NA
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
As part of Remote store segment upload flow, we upload multiple files to remote store directory. Today this upload is sequential and no two files are uploaded parallel. Which could be increasing latency and impacting overall throughput.
Describe the solution you'd like
Explore if parallelization of segment files upload can help improve latency and throughout. Although we need to be mindful to not bombard s3 with requests get throttled. Run performance tests to figure out baseline limits for a cluster/node after this change
Describe alternatives you've considered
NA
Additional context
NA
The text was updated successfully, but these errors were encountered: