[DocDB] Add full compaction metrics for scheduled and admin-triggered #14886
Assignees
Labels
2.16 Backport Required
area/docdb
YugabyteDB core features
kind/bug
This issue is a bug
priority/medium
Medium priority issue
Comments
jmeehan16
added
area/docdb
yugabyte-ci
added
kind/bug
jmeehan16
added a commit
that referenced
this issue
Dec 5, 2022
Summary: Compaction task metrics currently only record task state metrics (`active`,`paused`,`queued`) for all compactions together rather than separating them by type. For easier debugging, this separates metrics as follows: Two state types: `active` and `nonactive` (`nonactive` is a composite of `paused` and `queued`). Three compaction types: `background` (i.e. minor), `full` (both admin-triggered and scheduler-triggered) and `postsplit` This diff also deprecates `paused`, `queued`, and `total` compaction metrics so that current UI graphs will not break, facilitating a transition to the new graphs. Also adds some compaction-related log messages for easier debugging. Test Plan: ./yb_build.sh --cxx-test integration-tests_compaction-test --gtest_filter CompactionTest.ManualCompactionTaskMetrics ./yb_build.sh --cxx-test integration-tests_compaction-test --gtest_filter CompactionTest.BackgroundCompactionTaskMetrics Reviewers: rthallam, timur Reviewed By: timur Subscribers: ybase, bogdan Differential Revision: https://phabricator.dev.yugabyte.com/D20893
jayant07-yb
pushed a commit
to jayant07-yb/yugabyte-db
that referenced
this issue
Dec 7, 2022
Summary: Compaction task metrics currently only record task state metrics (`active`,`paused`,`queued`) for all compactions together rather than separating them by type. For easier debugging, this separates metrics as follows: Two state types: `active` and `nonactive` (`nonactive` is a composite of `paused` and `queued`). Three compaction types: `background` (i.e. minor), `full` (both admin-triggered and scheduler-triggered) and `postsplit` This diff also deprecates `paused`, `queued`, and `total` compaction metrics so that current UI graphs will not break, facilitating a transition to the new graphs. Also adds some compaction-related log messages for easier debugging. Test Plan: ./yb_build.sh --cxx-test integration-tests_compaction-test --gtest_filter CompactionTest.ManualCompactionTaskMetrics ./yb_build.sh --cxx-test integration-tests_compaction-test --gtest_filter CompactionTest.BackgroundCompactionTaskMetrics Reviewers: rthallam, timur Reviewed By: timur Subscribers: ybase, bogdan Differential Revision: https://phabricator.dev.yugabyte.com/D20893
jmeehan16
added a commit
that referenced
this issue
Dec 16, 2022
…sk metrics Summary: Compaction task metrics currently only record task state metrics (`active`,`paused`,`queued`) for all compactions together rather than separating them by type. For easier debugging, this separates metrics as follows: Two state types: `active` and `nonactive` (`nonactive` is a composite of `paused` and `queued`). Three compaction types: `background` (i.e. minor), `full` (both admin-triggered and scheduler-triggered) and `postsplit` This diff also deprecates `paused`, `queued`, and `total` compaction metrics so that current UI graphs will not break, facilitating a transition to the new graphs. Also adds some compaction-related log messages for easier debugging. Original commit: 0f20d60 / D20893 Test Plan: Jenkins: rebase: 2.16 ./yb_build.sh --cxx-test integration-tests_compaction-test --gtest_filter CompactionTest.ManualCompactionTaskMetrics ./yb_build.sh --cxx-test integration-tests_compaction-test --gtest_filter CompactionTest.BackgroundCompactionTaskMetrics Reviewers: timur, rthallam Reviewed By: rthallam Subscribers: bogdan, ybase Differential Revision: https://phabricator.dev.yugabyte.com/D21571
|
Backported to 2.16, this is good to close. |
jmeehan16
added a commit
that referenced
this issue
Apr 5, 2023
…sk metrics Summary: Compaction task metrics currently only record task state metrics (`active`,`paused`,`queued`) for all compactions together rather than separating them by type. For easier debugging, this separates metrics as follows: Two state types: `active` and `nonactive` (`nonactive` is a composite of `paused` and `queued`). Three compaction types: `background` (i.e. minor), `full` (both admin-triggered and scheduler-triggered) and `postsplit` This diff also deprecates `paused`, `queued`, and `total` compaction metrics so that current UI graphs will not break, facilitating a transition to the new graphs. Also adds some compaction-related log messages for easier debugging. Original commit: 0f20d60 / D20893 Test Plan: Jenkins: rebase: 2.14 ./yb_build.sh --cxx-test integration-tests_compaction-test --gtest_filter CompactionTest.ManualCompactionTaskMetrics ./yb_build.sh --cxx-test integration-tests_compaction-test --gtest_filter CompactionTest.BackgroundCompactionTaskMetrics Reviewers: timur, arybochkin, rthallam Reviewed By: rthallam Subscribers: bogdan, ybase Differential Revision: https://phabricator.dev.yugabyte.com/D23982
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Jira Link: DB-4163
Description
We need metrics to track when full compactions are triggered via automatic scheduling or via admin triggering. These should be similar to the metrics we already use to track post-split compactions (
ts_post_split_compaction_added).A nice bonus would be to modify the "number of files per compaction" to include full compactions as well.
The text was updated successfully, but these errors were encountered: