Skip to content

Commit

Permalink
[UII] Advanced agent monitoring options UI for HTTP endpoint and diag…
Browse files Browse the repository at this point in the history 
…nostics (elastic#193361)

## Summary

Resolves elastic#153950.

This PR implements a UI to configure advanced Elastic Agent monitoring
options under agent policy settings. These advanced options include
enabling HTTP monitoring endpoints and various options for agent
diagnostics. They are shown under an a toggle under the existing agent
monitoring logs and metrics collection options:

<img width="1326" alt="image"
src="https://github.com/user-attachments/assets/ac8cbe00-d838-4c9a-8a35-3dbf31222dc9">

If the base HTTP monitoring endpoint is not enabled, the rest of the
HTTP options are disabled:

<img width="1328" alt="image"
src="https://github.com/user-attachments/assets/2eac787c-3055-4862-b3eb-2566a39ee86c">

The following new fields are added to agent policy schema to support
this:
```
monitoring_http
monitoring_pprof_enabled
monitoring_diagnostics
```

This work supersedes the previous `HTTP monitoring endpoint` options
under `Advanced Settings` at the bottom of the page. Any previous
configuration under an agent policy's
`advanced_settings.agent_monitoring_http` saved object field are
migrated over to the new `monitoring_http` field and the old field is
deleted. See the migration fn `backfillAgentPolicyToV4`.

These new options are compiled to agent yaml like this:

```yml
agent:
  monitoring:
    enabled: true
    use_output: default
    logs: true
    metrics: true
    traces: true
    namespace: default
    pprof:
      enabled: true
    http:
      enabled: true
      host: localhost
      port: 6791
    diagnostics:
      limit:
        interval: 1m
        burst: 1
      uploader:
        max_retries: 10
        init_dur: 1s
        max_dur: 10m
```
Summarize your PR. If it involves visual changes include a screenshot or
gif.

### To-do
- [x] API integration tests
- [x] Full manual test of SO migration
- [x] Full manual test with agent using these settings

### Checklist

Delete any items that are not applicable to this PR.

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
  - elastic/ingest-docs#1333 
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
  • Loading branch information
@jen-huang @kibanamachine
jen-huang and kibanamachine authored Sep 22, 2024
1 parent a378064 commit 87cdc2d
Show file tree
Hide file tree
Showing 32 changed files with 1,671 additions and 136 deletions.
3 changes: 3 additions & 0 deletions packages/kbn-check-mappings-update-cli/current_fields.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -605,8 +605,11 @@
"is_preconfigured",
"is_protected",
"keep_monitoring_alive",
"monitoring_diagnostics",
"monitoring_enabled",
"monitoring_http",
"monitoring_output_id",
"monitoring_pprof_enabled",
"name",
"namespace",
"overrides",
Expand Down
12 changes: 12 additions & 0 deletions packages/kbn-check-mappings-update-cli/current_mappings.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1996,13 +1996,25 @@
"keep_monitoring_alive": {
"type": "boolean"
},
"monitoring_diagnostics": {
"index": false,
"type": "flattened"
},
"monitoring_enabled": {
"index": false,
"type": "keyword"
},
"monitoring_http": {
"index": false,
"type": "flattened"
},
"monitoring_output_id": {
"type": "keyword"
},
"monitoring_pprof_enabled": {
"index": false,
"type": "boolean"
},
"name": {
"type": "keyword"
},
Expand Down
1 change: 1 addition & 0 deletions packages/kbn-doc-links/src/get_doc_links.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -879,6 +879,7 @@ export const getDocLinks = ({ kibanaBranch, buildFlavor }: GetDocLinkOptions): D
roleAndPrivileges: `${FLEET_DOCS}fleet-roles-and-privileges.html`,
proxiesSettings: `${FLEET_DOCS}fleet-agent-proxy-support.html`,
unprivilegedMode: `${FLEET_DOCS}elastic-agent-unprivileged.html#unprivileged-change-mode`,
httpMonitoring: `${FLEET_DOCS}agent-policy.html#change-policy-enable-agent-monitoring`,
},
integrationDeveloper: {
upload: `${INTEGRATIONS_DEV_DOCS}upload-a-new-integration.html`,
Expand Down
1 change: 1 addition & 0 deletions packages/kbn-doc-links/src/types.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -564,6 +564,7 @@ export interface DocLinks {
roleAndPrivileges: string;
proxiesSettings: string;
unprivilegedMode: string;
httpMonitoring: string;
}>;
readonly integrationDeveloper: {
upload: string;
Expand Down
2 changes: 1 addition & 1 deletion src/core/server/integration_tests/ci_checks/saved_objects/check_registered_types.test.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ describe('checking migration metadata changes on all registered SO types', () =>
"infra-custom-dashboards": "1a5994f2e05bb8a1609825ddbf5012f77c5c67f3",
"infrastructure-monitoring-log-view": "5f86709d3c27aed7a8379153b08ee5d3d90d77f5",
"infrastructure-ui-source": "113182d6895764378dfe7fa9fa027244f3a457c4",
"ingest-agent-policies": "90625b4a5ded9d4867358fcccc14a57c0454fcee",
"ingest-agent-policies": "5e95e539826a40ad08fd0c1d161da0a4d86ffc6d",
"ingest-download-sources": "279a68147e62e4d8858c09ad1cf03bd5551ce58d",
"ingest-outputs": "daafff49255ab700e07491376fe89f04fc998b91",
"ingest-package-policies": "53a94064674835fdb35e5186233bcd7052eabd22",
Expand Down
137 changes: 137 additions & 0 deletions x-pack/plugins/fleet/common/openapi/bundled.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7592,6 +7592,56 @@
},
"description": "User defined data tags that are added to all of the inputs. The values can be strings or numbers."
}
},
"monitoring_pprof_enabled": {
"type": "boolean"
},
"monitoring_http": {
"type": "object",
"properties": {
"enabled": {
"type": "boolean"
},
"host": {
"type": "string"
},
"port": {
"type": "number"
}
},
"required": [
"enabled"
]
},
"monitoring_diagnostics": {
"type": "object",
"properties": {
"limit": {
"type": "object",
"properties": {
"interval": {
"type": "string"
},
"burst": {
"type": "number"
}
}
},
"uploader": {
"type": "object",
"properties": {
"max_retries": {
"type": "number"
},
"init_dur": {
"type": "string"
},
"max_dur": {
"type": "string"
}
}
}
}
}
},
"required": [
Expand Down Expand Up @@ -7874,6 +7924,93 @@
}
}
},
"monitoring": {
"type": "object",
"properties": {
"namespace": {
"type": "string"
},
"use_output": {
"type": "string"
},
"enabled": {
"type": "boolean"
},
"metrics": {
"type": "boolean"
},
"logs": {
"type": "boolean"
},
"traces": {
"type": "boolean"
},
"pprof": {
"type": "object",
"properties": {
"enabled": {
"type": "boolean"
}
},
"required": [
"enabled"
]
},
"http": {
"type": "object",
"properties": {
"enabled": {
"type": "boolean"
},
"host": {
"type": "string"
},
"port": {
"type": "number"
}
},
"required": [
"enabled"
]
},
"diagnostics": {
"type": "object",
"properties": {
"limit": {
"type": "object",
"properties": {
"interval": {
"type": "string"
},
"burst": {
"type": "number"
}
}
},
"uploader": {
"type": "object",
"properties": {
"max_retries": {
"type": "number"
},
"init_dur": {
"type": "string"
},
"max_dur": {
"type": "string"
}
}
}
}
}
},
"required": [
"enabled",
"metrics",
"logs",
"traces"
]
},
"fleet": {
"oneOf": [
{
Expand Down
89 changes: 89 additions & 0 deletions x-pack/plugins/fleet/common/openapi/bundled.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4861,6 +4861,38 @@ components:
description: >-
User defined data tags that are added to all of the inputs. The
values can be strings or numbers.
monitoring_pprof_enabled:
type: boolean
monitoring_http:
type: object
properties:
enabled:
type: boolean
host:
type: string
port:
type: number
required:
- enabled
monitoring_diagnostics:
type: object
properties:
limit:
type: object
properties:
interval:
type: string
burst:
type: number
uploader:
type: object
properties:
max_retries:
type: number
init_dur:
type: string
max_dur:
type: string
required:
- id
- status
Expand Down Expand Up @@ -5054,6 +5086,63 @@ components:
type: integer
data:
$ref: '#/components/schemas/full_agent_policy_output_permissions'
monitoring:
type: object
properties:
namespace:
type: string
use_output:
type: string
enabled:
type: boolean
metrics:
type: boolean
logs:
type: boolean
traces:
type: boolean
pprof:
type: object
properties:
enabled:
type: boolean
required:
- enabled
http:
type: object
properties:
enabled:
type: boolean
host:
type: string
port:
type: number
required:
- enabled
diagnostics:
type: object
properties:
limit:
type: object
properties:
interval:
type: string
burst:
type: number
uploader:
type: object
properties:
max_retries:
type: number
init_dur:
type: string
max_dur:
type: string
required:
- enabled
- metrics
- logs
- traces
fleet:
oneOf:
- type: object
Expand Down
32 changes: 32 additions & 0 deletions x-pack/plugins/fleet/common/openapi/components/schemas/agent_policy.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,38 @@ properties:
- type: string
- type: number
description: User defined data tags that are added to all of the inputs. The values can be strings or numbers.
monitoring_pprof_enabled:
type: boolean
monitoring_http:
type: object
properties:
enabled:
type: boolean
host:
type: string
port:
type: number
required:
- enabled
monitoring_diagnostics:
type: object
properties:
limit:
type: object
properties:
interval:
type: string
burst:
type: number
uploader:
type: object
properties:
max_retries:
type: number
init_dur:
type: string
max_dur:
type: string
required:
- id
- status
Expand Down
Loading

0 comments on commit 87cdc2d

Please sign in to comment.