Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enabling service to have multiple ports and serviceMonitor to scrape multiple endpoints #987

Closed
tmakamure opened this issue Aug 25, 2023 · 1 comment
Labels
enhancement New feature or request

Comments

@tmakamure
Copy link
Contributor

Is your feature request related to a problem?
Currently aws-for-fluentbit helm chart supports a kubernetes service with one port, and a serviceMonitor that exposes that service for scraping by Prometheus. The current supported configuration is as below:

. . . rest of chart . . .
serviceMonitor:
  service: 
    type: ClusterIP
    port: 2020
    targetPort: 2020
  enabled: false
  interval: 30s
  telemetryPath: /api/v1/metrics/prometheus
  labels:
    # app: example-application
    # teamname: example
  timeout: 10s
  relabelings: []
  targetLabels: []
  metricRelabelings: []

With fluentbit having the capacity for users to create more custom prometheus metrics, using plugins such as fliter:log_to_metrics -> output:prometheus_exporter and/or any other custom plugins capable of generating prometheus metrics, it is evidently desirable to have the ability to expose multiple ports of the kubernetes service for extra prometheus metrics, and for the servicemonitor to have the ability to monitor multiple endpoints for prometheus scraping.

An example configuration fitting the scenario could look like:

. . . rest of chart . . .
serviceMonitor:
  service: 
    type: ClusterIP
    port: 2020
    targetPort: 2020
    extraPorts:  # Any additional ports to be exposed by the service
        - port: 2021
          targetPort: 2021
          protocol: TCP
          name: custom-metrics
  enabled: true
  interval: 30s
  telemetryPath: /api/v1/metrics/prometheus
  labels:
    # app: example-application
    # teamname: example
  timeout: 10s
  relabelings: []
  targetLabels: []
  metricRelabelings: []
  extraEndpoints: # any additional endpoints of the service we want to monitor for scraping of metrics
      - port: custom-metrics
        path: /metrics
        interval: 10s
        scrapeTimeout: 10s
        scheme: http

Is your feature request related to a specific Helm chart, if yes mention name of the chart?
aws-for-fluentbit

Describe the solution you'd like
When the chart has the functionality for services to expose multiple ports, and the serviceMonitor to monitor mulitple endpoints of the service for prometheus scraping, it becomes easier to integrate custom prometheus metrics in fluentbit and scrape them with prometheus. A sample configuration could look like the above.

Describe alternatives you've considered
Alternatives could be to create an additional kubernetes service for the fluentbit pod, and have this service expose the port(s). You can then configure your promtheus instance to have a job that scrapes your additional service for metrics, or you can create another serviceMonitor CRD that monitors that additional service. It would however be more desirable and easier to manage if the chart itself enabled this functionality.

@tmakamure
Copy link
Contributor Author

Changes have been made and merged to master. Closing as completed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant