Skip to content

Commit

Permalink
Alloy Metric Module Imports (#543)
Browse files Browse the repository at this point in the history
* Added Support for Alloy Module Imports

Resolves #537

* Added extraConfig Import Example

* Added Loki Metrics to Allow List (#535)

* Added Loki Metrics to Allow List

* Regenerated Output

* Regenerated Docs

* Update Grafana Alloy (#546)

Signed-off-by: Pete Wall <pete.wall@grafana.com>

* Always call tpl outside of with blocks to preserve backwards compatibility (#554)

Signed-off-by: Pete Wall <pete.wall@grafana.com>

* Added schema

* Renamed modules.alias to modules.name

* Added Support for Journal Logs (#545)

* Added Support for Journal Logs

Resolves #544

* Fixed formatting

* Added Loki Metrics to Allow List (#535)

* Added Loki Metrics to Allow List

* Regenerated Output

* Regenerated Docs

* Added Support for Journal Logs

Resolves #544

* Added CI Test

* Fixed Formatting

* Updated Test Queries

* Regenerated Documentation

* Removed Journal from Test

Kind nodes do not have systemd

* Update Grafana Alloy (#546)

Signed-off-by: Pete Wall <pete.wall@grafana.com>

* Added Support for Journal Logs

Resolves #544

* Revert "Updated Test Queries"

This reverts commit 8de520c.

* Regenerated

---------

Signed-off-by: Pete Wall <pete.wall@grafana.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Added Support for Alloy Module Imports

Resolves #537

* Added extraConfig Import Example

* Added schema

* Renamed modules.alias to modules.name

* Generated Outputs

---------

Signed-off-by: Pete Wall <pete.wall@grafana.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Pete Wall <pete.wall@grafana.com>
  • Loading branch information
3 people committed Jun 8, 2024
1 parent ccaebc5 commit ead04cc
Show file tree
Hide file tree
Showing 20 changed files with 105,296 additions and 6 deletions.
14 changes: 8 additions & 6 deletions charts/k8s-monitoring/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -317,6 +317,7 @@ The Prometheus and Loki services may be hosted on the same cluster, or remotely
|-----|------|---------|-------------|
| logs.cluster_events.enabled | bool | `true` | Scrape Kubernetes cluster events |
| logs.cluster_events.extraConfig | string | `""` | Extra configuration that will be added to the Grafana Alloy for Cluster Events configuration file. This value is templated so that you can refer to other values from this file. This cannot be used to modify the generated configuration values, only append new components. See [Adding custom Flow configuration](#adding-custom-flow-configuration) for an example. |
| logs.cluster_events.extraStageBlocks | string | `""` | Stage blocks to be added to the loki.process component for cluster events. ([docs](https://grafana.com/docs/alloy/latest/reference/components/loki.process/#blocks)) This value is templated so that you can refer to other values from this file. |
| logs.cluster_events.logFormat | string | `"logfmt"` | Log format used to forward cluster events. Allowed values: `logfmt` (default), `json`. |
| logs.cluster_events.logToStdout | bool | `false` | Logs the cluster events to stdout. Useful for debugging. |
| logs.cluster_events.namespaces | list | `[]` | List of namespaces to watch for events (`[]` means all namespaces) |
Expand Down Expand Up @@ -389,6 +390,13 @@ The Prometheus and Loki services may be hosted on the same cluster, or remotely
| metrics.alloy.metricsTuning.useIntegrationAllowList | bool | `false` | Filter the list of metrics from Grafana Alloy to the minimal set required for Kubernetes Monitoring as well as the Grafana Alloy integration. |
| metrics.alloy.scrapeInterval | string | 60s | How frequently to scrape metrics from Grafana Alloy. Overrides metrics.scrapeInterval |

### Metrics Job: Alloy Modules

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| metrics.alloyModules.connections | list | `[]` | List of connection configurations used by modules. Configures the import.git component ([docs](https://grafana.com/docs/alloy/latest/reference/components/import.git/) <br>- `alias: ""` the alias of the connection <br>- `repository: ""` URL of the Git repository containing the module. <br>- `revision: ""` Branch, tag, or commit to be checked out. <br>- `pull_frequency: 15m` How often the module should check for updates. <br>- `default: true` If true, this connection is used as the default when none is specified. <br>- `basic_auth: {}` Credentials for basic authentication if needed. ([docs](https://grafana.com/docs/alloy/latest/reference/config-blocks/import.git/#basic_auth-block)) <br>- `ssh_key: {}` Provides SSH key details for secure connections. ([docs](https://grafana.com/docs/alloy/latest/reference/config-blocks/import.git/#ssh_key-block)) |
| metrics.alloyModules.modules | list | `[]` | List of Modules to import. Each module is expected to have a "kubernetes" module and a "scrape" component. Each module can have the following properties: <br>- `name: ""` the name to use for the module. *note:* this is automatically prefixed with module_ to avoid conflicts with core components <br>- `path: ""` the path to the alloy module <br>- `connection: ""` (optional) the alias of the connection to use, if not specified the default connection is used <br>- `targets: {}` (optional) Additional arguments to be passed to the modules kubernetes component <br>- `scrape: {}` (optional) Additional arguments to be passed to the modules scrape component <br>- `extraRelabelingRules: ""` additional relabeling rules for the discovery.relabel component <br>- `extraMetricRelabelingRules:` additional relabeling rules for the prometheus.relabel component |

### Metrics Job: ApiServer

| Key | Type | Default | Description |
Expand Down Expand Up @@ -722,12 +730,6 @@ The Prometheus and Loki services may be hosted on the same cluster, or remotely
| traces.receiver.filters | object | `{"span":[],"spanevent":[]}` | Apply a filter to traces received via the OTLP or OTLP HTTP receivers. ([docs](https://grafana.com/docs/alloy/latest/reference/components/otelcol.processor.filter/)) |
| traces.receiver.transforms | object | `{"resource":[],"span":[],"spanevent":[]}` | Apply a transformation to traces received via the OTLP or OTLP HTTP receivers. ([docs](https://grafana.com/docs/alloy/latest/reference/components/otelcol.processor.transform/)) |

### Other Values

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| logs.cluster_events.extraStageBlocks | string | `""` | Stage blocks to be added to the loki.process component for cluster events. ([docs](https://grafana.com/docs/alloy/latest/reference/components/loki.process/#blocks)) This value is templated so that you can refer to other values from this file. |

## Customizing the configuration

There are several options for customizing the configuration generated by this chart. This can be used to add extra
Expand Down
115 changes: 115 additions & 0 deletions charts/k8s-monitoring/schema-mods/enums-and-types.json
Original file line number Diff line number Diff line change
Expand Up @@ -233,6 +233,121 @@
"type": ["string", "integer", "null"]
}
}
},
"alloyModules": {
"type": "object",
"properties": {
"connections": {
"type": "array",
"items": {
"type": "object",
"properties": {
"alias": {
"type": "string",
"description": "The alias of the connection"
},
"repository": {
"type": "string",
"description": "URL of the Git repository containing the module"
},
"revision": {
"type": "string",
"description": "Branch, tag, or commit to be checked out"
},
"pull_frequency": {
"type": "string",
"description": "The frequency to pull the repository for updates in duration format."
},
"default": {
"type": "boolean",
"description": "If true, this connection is used as the default when none is specified"
},
"basic_auth": {
"type": "object",
"description": "Credentials for basic authentication if needed",
"properties": {
"username": {
"type": "string",
"description": "Username for basic authentication"
},
"password": {
"type": "string",
"description": "Password for basic authentication"
},
"password_file": {
"type": "string",
"description": "Path to a file containing the password"
}
}
},
"ssh_key": {
"type": "object",
"description": "Provides SSH key details for secure connections",
"properties": {
"username": {
"type": "string",
"description": "SSH username"
},
"key": {
"type": "string",
"description": "SSH private key"
},
"key_file": {
"type": "string",
"description": "Path to the SSH private key file"
},
"passphrase": {
"type": "string",
"description": "Passphrase for the SSH private key"
}
},
"required": ["username"]
}
},
"required": ["alias", "repository"]
}
},
"modules": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "The name to use for the module"
},
"path": {
"type": "string",
"description": "The path in the repository where the module is stored."
},
"connection": {
"type": "string",
"description": "The alias of the connection to use, if not specified the default connection is used"
},
"targets": {
"type": "object",
"description": "Additional arguments to be passed to the module's Kubernetes component",
"additionalProperties": true
},
"scrape": {
"type": "object",
"description": "Additional arguments to be passed to the module's scrape component",
"additionalProperties": true
},
"extraRelabelingRules": {
"type": "string",
"description": "Additional relabeling rules for the modules discovery.relabel component"
},
"extraMetricRelabelingRules": {
"type": "string",
"description": "Additional relabeling rules for the modules prometheus.relabel component"
}
},
"required": ["name", "path"]
}
}
},
"required": ["connections", "modules"]
}
}
},
Expand Down
4 changes: 4 additions & 0 deletions charts/k8s-monitoring/templates/_configs.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,10 @@
{{- include "alloy.config.service_monitors" . }}
{{- end }}

{{- if len .Values.metrics.alloyModules.modules }}
{{- include "alloy.config.alloyMetricModules" . }}
{{- end }}

{{- include "alloy.config.metricsService" . }}
{{- end }}

Expand Down
103 changes: 103 additions & 0 deletions charts/k8s-monitoring/templates/alloy_config/_alloy-modules.alloy.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
{{ define "alloy.config.alloyMetricModules" }}
// Alloy Modules
{{- $connections := .Values.metrics.alloyModules.connections }}
{{- /* Determine the default connection */}}
{{- $defaultConnection := dict }}
{{- range $connections }}
{{- if .default }}
{{- $defaultConnection = . }}
{{- end }}
{{- end }}
{{- $extraRelabelingRules := .Values.metrics.extraRelabelingRules | default "" }}
{{- $extraMetricRelabelingRules := .Values.metrics.extraMetricRelabelingRules | default "" }}
{{- $defaultMaxCacheSize := .Values.metrics.maxCacheSize }}
{{- $defaultScrapeInterval := .Values.metrics.scrapeInterval }}
{{- $clustering := .Values.alloy.alloy.clustering.enabled }}

{{- range $module := .Values.metrics.alloyModules.modules }}
{{- $moduleName := $module.name }}
{{- /* Find the connection for the module or use the default */}}
{{- $moduleConnection := $defaultConnection }}
{{- range $conn := $connections }}
{{- if and $module.connection (eq $conn.alias $module.connection) }}
{{- $moduleConnection = $conn }}
{{- end }}
{{- end }}

// {{ $moduleName }}: import the module
import.git "module_{{ $moduleName }}" {
repository = "{{ $moduleConnection.repository }}"
revision = "{{ $moduleConnection.revision }}"
path = "{{ $module.path }}"
pull_frequency = "{{ $moduleConnection.pull_frequency }}"

{{- with $moduleConnection.basic_auth }}
basic_auth {
{{ if .username }}username = "{{ .username }}"{{ end }}
{{ if .password }}password = "{{ .password }}"{{ end }}
{{ if .password_file }}password_file = "{{ .password_file }}"{{ end }}
}
{{- end }}

{{- with $moduleConnection.ssh_key }}
ssh_key {
{{ if .username }}username = "{{ .username }}"{{ end }}
{{ if .key }}key = "{{ .key }}"{{ end }}
{{ if .key_file }}key_file = "{{ .key_file }}"{{ end }}
{{ if .passphrase }}passphrase = "{{ .passphrase }}"{{ end }}
}
{{- end }}
}

// {{ $moduleName }}: call the targets component
module_{{ $moduleName }}.kubernetes "targets" {
{{- if $module.targets }}
{{- range $key, $value := $module.targets }}
{{ $key }} = {{ if kindIs "slice" $value }}[{{ range $index, $item := $value }}"{{ $item }}"{{ if lt $index (sub (len $value) 1) }}, {{ end }}{{ end }}]{{ else }}"{{ $value }}"{{ end }}
{{- end }}
{{- end }}
}

// {{ $moduleName }}: additional service discovery relabelings
discovery.relabel "module_{{ $moduleName }}" {
targets = module_{{ $moduleName }}.kubernetes.targets.output
// global discovery relabelings
{{- if $extraRelabelingRules }}
{{ $extraRelabelingRules | indent 2 }}
{{- end }}
// {{ $moduleName }}: additional discovery relabelings
{{- if $module.extraRelabelingRules }}
{{ $module.extraRelabelingRules | indent 2 }}
{{- end }}
}

// {{ $moduleName }}: call the scrape component
module_{{ $moduleName }}.scrape "metrics" {
targets = discovery.relabel.module_{{ $moduleName }}.output
forward_to = [prometheus.relabel.module_{{ $moduleName }}.receiver]
clustering = {{ $clustering }}
scrape_interval = "{{ if and (hasKey $module "scrape") (hasKey $module.scrape "scrape_interval") }}{{ index $module.scrape "scrape_interval" }}{{- else }}{{ $defaultScrapeInterval }}{{- end }}"
{{- if $module.scrape }}
{{- range $key, $value := $module.scrape }}
{{- if not (eq $key "scrape_interval") }}
{{ $key }} = {{ if kindIs "slice" $value }}[{{ range $index, $item := $value }}"{{ $item }}"{{ if lt $index (sub (len $value) 1) }}, {{ end }}{{ end }}]{{ else }}"{{ $value }}"{{ end }}
{{- end }}
{{- end }}
{{- end }}
}

// {{ $moduleName }}: additional metric relabelings
prometheus.relabel "module_{{ $moduleName }}" {
max_cache_size = {{ if and (hasKey $module "scrape") (hasKey $module.scrape "max_cache_size") }}{{ index $module.scrape "max_cache_size" | int }}{{- else }}{{ $defaultMaxCacheSize | int }}{{- end }}
// global metric relabelings
{{- if $extraMetricRelabelingRules }}
{{ $extraMetricRelabelingRules | indent 2 }}
{{- end }}
// additional {{ $moduleName }} module specific metric relabelings
{{- if $module.extraMetricRelabelingRules }}
{{ $module.extraMetricRelabelingRules | indent 2 }}
{{- end }}
forward_to = [prometheus.relabel.metrics_service.receiver]
}
{{- end }}
{{ end }}
Loading

0 comments on commit ead04cc

Please sign in to comment.