[Backport 2.x] Core and Plugin changes for query-level resource usages tracking #14085
Conversation
ansjcy
requested review from
anasalkouz,
andrross,
Bukhtawar,
CEHENKLE,
dblock,
dbwiddis,
dreamer-89,
gbbafna,
kotwanikunal,
mch2,
msfroh,
nknize,
owaiskazi19,
reta,
Rishikesh1159,
sachinpkale,
saratvemulapalli,
shwetathareja,
sohami,
tlfeng and
VachaShah
as code owners
|
❌ Gradle check result for d3c82b6: Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
ansjcy
changed the title
|
❌ Gradle check result for a862c82: Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
ansjcy
force-pushed
the
backport/backport-13172-to-2.x
branch
2 times, most recently
from
f213ca7
to
9cc51bc
Compare
|
❌ Gradle check result for f213ca7: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for 9cc51bc: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
ansjcy
force-pushed
the
backport/backport-13172-to-2.x
branch
from
9cc51bc
to
7b01702
Compare
|
❌ Gradle check result for 7b01702: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
ansjcy
force-pushed
the
backport/backport-13172-to-2.x
branch
from
7b01702
to
c56af99
Compare
|
❌ Gradle check result for c56af99: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
ansjcy
force-pushed
the
backport/backport-13172-to-2.x
branch
from
c56af99
to
aeca941
Compare
|
❕ Gradle check result for aeca941: UNSTABLE Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure. |
ansjcy
force-pushed
the
backport/backport-13172-to-2.x
branch
2 times, most recently
from
5cbe7fe
to
e2df8ad
Compare
|
Breaking changes is going to fail until #14143 is merged |
ansjcy
force-pushed
the
backport/backport-13172-to-2.x
branch
from
e2df8ad
to
0bfb26e
Compare
* Query-level resource usages tracking Signed-off-by: Chenyang Ji <cyji@amazon.com> * Moving TaskResourceTrackingService to clusterService Signed-off-by: Chenyang Ji <cyji@amazon.com> * use shard response header to piggyback task resource usages Signed-off-by: Chenyang Ji <cyji@amazon.com> * split changes for query insights plugin Signed-off-by: Chenyang Ji <cyji@amazon.com> * improve the supplier logic and other misc items Signed-off-by: Chenyang Ji <cyji@amazon.com> * track resource usage for failed requests Signed-off-by: Chenyang Ji <cyji@amazon.com> * move resource usages interactions into TaskResourceTrackingService Signed-off-by: Chenyang Ji <cyji@amazon.com> --------- Signed-off-by: Chenyang Ji <cyji@amazon.com> (cherry picked from commit 3d1fa98)
…h-project#13739) * consume query level cpu and memory usage in query insights Signed-off-by: Chenyang Ji <cyji@amazon.com> * handle failed requests metrics in query insights Signed-off-by: Chenyang Ji <cyji@amazon.com> * refactor the code to make it more maintainable Signed-off-by: Chenyang Ji <cyji@amazon.com> --------- Signed-off-by: Chenyang Ji <cyji@amazon.com> (cherry picked from commit 04a417a)
Signed-off-by: Chenyang Ji <cyji@amazon.com> (cherry picked from commit b403fdc)
ansjcy
force-pushed
the
backport/backport-13172-to-2.x
branch
from
0bfb26e
to
da4367e
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## 2.x #14085 +/- ##
============================================
+ Coverage 71.28% 71.40% +0.12%
- Complexity 60145 61836 +1691
============================================
Files 4957 5071 +114
Lines 282799 290401 +7602
Branches 41409 42338 +929
============================================
+ Hits 201591 207375 +5784
- Misses 64189 65572 +1383
- Partials 17019 17454 +435 ☔ View full report in Codecov by Sentry. |
…s tracking (opensearch-project#14085) * Query-level resource usages tracking (opensearch-project#13172) * Query-level resource usages tracking Signed-off-by: Chenyang Ji <cyji@amazon.com> * Moving TaskResourceTrackingService to clusterService Signed-off-by: Chenyang Ji <cyji@amazon.com> * use shard response header to piggyback task resource usages Signed-off-by: Chenyang Ji <cyji@amazon.com> * split changes for query insights plugin Signed-off-by: Chenyang Ji <cyji@amazon.com> * improve the supplier logic and other misc items Signed-off-by: Chenyang Ji <cyji@amazon.com> * track resource usage for failed requests Signed-off-by: Chenyang Ji <cyji@amazon.com> * move resource usages interactions into TaskResourceTrackingService Signed-off-by: Chenyang Ji <cyji@amazon.com> --------- Signed-off-by: Chenyang Ji <cyji@amazon.com> (cherry picked from commit 3d1fa98) * fix concurrent modification issue in thread context (opensearch-project#14084) Signed-off-by: Chenyang Ji <cyji@amazon.com> (cherry picked from commit c8f0b6d) * consume query level cpu and memory usage in query insights (opensearch-project#13739) * consume query level cpu and memory usage in query insights Signed-off-by: Chenyang Ji <cyji@amazon.com> * handle failed requests metrics in query insights Signed-off-by: Chenyang Ji <cyji@amazon.com> * refactor the code to make it more maintainable Signed-off-by: Chenyang Ji <cyji@amazon.com> --------- Signed-off-by: Chenyang Ji <cyji@amazon.com> (cherry picked from commit 04a417a) * fix japicmp check for threadContext Signed-off-by: Chenyang Ji <cyji@amazon.com> (cherry picked from commit b403fdc) Signed-off-by: kkewwei <kkewwei@163.com>
Description
This is a manual backport PR for Query-level resource usages tracking, and the query insights plugin support to consume the query level resource usage.
Related Issues
Original PRs:
#13172
#14084
#13739
#14147
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.