Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cherry-pick #16420 to 7.x: Add filtering option for prometheus collector #16521

Merged
merged 1 commit into from
Feb 24, 2020

Conversation

ChrsMark
Copy link
Member

Cherry-pick of PR #16420 to 7.x branch. Original message:

Drafting what was discussed on #15493 to start implementation specific discussions.For the time being only filters are implemented, but mappings will be added if we agree on this.
@jsoriano I preferred to add the processing on collector layer to avoid touching GetFamilies since it is used by many other places like the state_* metricsets of k8s module. Not sure if we would like to have it in there (in the helper method).

What does this PR do?

This PR introduces filtering options for prometheus collector metricset.

Why is it important?

This is important when someone want to filter out metrics that come in a single response for instance in case of prometheus exporters.

This will be useful in light-modules like ibmmq module where currently we use script processor to filter out metrics as well as in OpenMetrics module where a user may want to explicitly define which metrics want to keep.

How to test this PR locally

  1. One can use a nodexporter as an example Prometheus app to collect from. For instance this forked project should be enough: https://github.com/ChrsMark/dockprom. Download and docker-compose up. Then metrics should be exported at http://localhost:9100/metrics.
  2. Configuration:
- module: prometheus
  period: 10s
  hosts: ["localhost:9092"]
  metrics_path: /metrics
  metrics_filters:
    include: ["node_filesystem_*"]
    exclude: ["node_filesystem_device_*", "node_filesystem_readonly"]

And expect to see only events that match node_filesystem_* but also not node_filesystem_device_* and are not node_filesystem_readonly.
3. Try to remove node_filesystem_readonly from exclude filters and see that this metric is included in events. One can try different combinations.

Related issues

@ChrsMark ChrsMark requested a review from a team as a code owner February 24, 2020 16:24
@ChrsMark ChrsMark self-assigned this Feb 24, 2020
@ChrsMark ChrsMark merged commit 510624d into elastic:7.x Feb 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants