Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Metricbeat] Add collecting AuroraDB metrics in rds metricset #16004

Merged
merged 5 commits into from
Feb 4, 2020
Merged

[Metricbeat] Add collecting AuroraDB metrics in rds metricset #16004

merged 5 commits into from
Feb 4, 2020

Conversation

kaiyan-sheng
Copy link
Contributor

What does this PR do?

This PR is to add collecting AuroraDB metrics into rds metricset: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.Monitoring.html

How to test this PR locally

  1. create an AuroraDB service in AWS
  2. enable aws module
  3. change aws.yml config to:
- module: aws
  period: 5m
  credential_profile_name: elastic-test
  metricsets:
    - rds
  1. You should see rds metrics from AuroraDB like:
    aws.rds.aurora_bin_log_replica_lag, aws.rds.aurora_replica.lag.ms, aurora_replica.lag_max.ms and etc.

Related issues

@kaiyan-sheng kaiyan-sheng requested a review from a team as a code owner January 31, 2020 21:26
@kaiyan-sheng kaiyan-sheng self-assigned this Jan 31, 2020
@kaiyan-sheng kaiyan-sheng added Metricbeat Metricbeat needs_backport PR is waiting to be backported to other branches. review Team:Platforms Label for the Integrations - Platforms team labels Jan 31, 2020
ChrsMark
ChrsMark reviewed Feb 3, 2020
View reviewed changes
x-pack/metricbeat/module/aws/rds/data.go Outdated
"cache_hit_ratio.result_set": c.Float("ResultSetCacheHitRatio"),

"engine_uptime.sec": c.Float("EngineUptime"),
"volume_read_io_per_sec": c.Float("VolumeReadIOPs"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe:

volume.io_per_sec.read and volume.io_per_sec.write?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about volume.read.iops and volume.write.iops? I think iops is the unit of these two metrics. That's why I put it at the end of the metric name.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

ChrsMark
ChrsMark reviewed Feb 3, 2020
View reviewed changes
Copy link
Member

@ChrsMark ChrsMark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any plans for new visualisations for these new metrics?

@kaiyan-sheng
Copy link
Contributor Author

Any plans for new visualisations for these new metrics?

Hmm I'm not planning to add one in this PR at least. rds metricset has it's predefined dashboard for general rds monitoring metrics(for all different DBs). I'm not sure how beneficial it will be to add a specific dashboard for AuroraDB only.

ChrsMark
ChrsMark approved these changes Feb 4, 2020
View reviewed changes
Copy link
Member

@ChrsMark ChrsMark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@kaiyan-sheng kaiyan-sheng merged commit 1e024a5 into elastic:master Feb 4, 2020
@kaiyan-sheng kaiyan-sheng deleted the auroradb branch February 4, 2020 15:55
@kaiyan-sheng kaiyan-sheng mentioned this pull request Feb 4, 2020
Merged
@kaiyan-sheng kaiyan-sheng added v7.7.0 and removed needs_backport PR is waiting to be backported to other branches. labels Feb 4, 2020
@kaiyan-sheng kaiyan-sheng mentioned this pull request Feb 4, 2020
Merged
kaiyan-sheng added a commit that referenced this pull request Feb 4, 2020
@kaiyan-sheng
Cherry-pick #16004 to 7.x: [Metricbeat] Add collecting AuroraDB metri…
75424ea 
…cs in rds metricset (#16071)

* [Metricbeat] Add collecting AuroraDB metrics in rds metricset (#16004)

* Add collecting AuroraDB metrics in rds metricset

* update changelog

* Rename to use iops as unit

* run mage fmt update

(cherry picked from commit 1e024a5)
(cherry picked from commit 61253f1)
@kaiyan-sheng kaiyan-sheng added the test-plan Add this PR to be manual test plan label Mar 23, 2020
@andresrc andresrc added the test-plan-added This PR has been added to the test plan label Mar 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ChrsMark ChrsMark ChrsMark approved these changes

Assignees

@kaiyan-sheng kaiyan-sheng

Labels
Metricbeat Metricbeat review Team:Platforms Label for the Integrations - Platforms team test-plan Add this PR to be manual test plan test-plan-added This PR has been added to the test plan v7.7.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Metricbeat] Add support for Amazon AuroraDB in rds metricset
3 participants
@kaiyan-sheng @ChrsMark @andresrc