Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable monitoring during fleetserver bootstrap process #27222

Merged

Conversation

michel-laterman
Copy link
Contributor

@michel-laterman michel-laterman commented Aug 3, 2021

What does this PR do?

Set the monitoring.enabled flag to false during the fleet-server
bootstrap process in order to ensure that no data is sent to an
elasticsearch instance running on the same host.

Why is it important?

In a self-hosted scenario fleet will show that some data-streams have stopped in certain cases (such as after the fleet-server restarts). However these datastreams are incorrectly sent by the agent during the bootstrapping processes when the default credentials for output match Elasticsearch's credentials.

Checklist

  • My code follows the style guidelines of this project
  • [] I have commented my code, particularly in hard-to-understand areas
  • [] I have made corresponding changes to the documentation
  • [] I have made corresponding change to the default configuration files
  • [] I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

Run ES + fleet locally (elastic-package can do this), view the datastreams tab in the fleet-ui, the elastic_agent.metricsbeat and elastic_agent.filebeat streams should not generate.

Related issues

Set the monitoring.enabled flag to false during the fleet-server
bootstrap process in order to ensure that no data is sent to an
elasticsearch instance running on the same host.
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Aug 3, 2021
@michel-laterman michel-laterman added the Team:Elastic-Agent Label for the Agent team label Aug 3, 2021
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Aug 3, 2021
cfg.Settings.MonitoringConfig.Enabled = false
} else {
cfg.Settings.MonitoringConfig = &monitoringCfg.MonitoringConfig{Enabled: false}
}
monitor, err := monitoring.NewMonitor(cfg.Settings)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I attempted to replace the monitor with the noop monitor, but the agent was not able to bootstap correctly.

@elasticmachine
Copy link
Collaborator

elasticmachine commented Aug 3, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-08-09T17:50:19.817+0000

  • Duration: 67 min 22 sec

  • Commit: d6a642e

Test stats 🧪

Test Results
Failed 0
Passed 7012
Skipped 16
Total 7028

Trends 🧪

Image of Build Times

Image of Tests

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 7012
Skipped 16
Total 7028

@michel-laterman michel-laterman marked this pull request as ready for review August 9, 2021 19:07
@elasticmachine
Copy link
Collaborator

Pinging @elastic/agent (Team:Agent)

Copy link
Contributor

@blakerouse blakerouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@michel-laterman michel-laterman merged commit 212dfa9 into elastic:master Aug 17, 2021
@michel-laterman michel-laterman deleted the disable-bootstrap-monitor branch August 17, 2021 16:14
michel-laterman added a commit to michel-laterman/beats that referenced this pull request Aug 17, 2021
* Disable monitoring during fleetserver bootstrap process

Set the monitoring.enabled flag to false during the fleet-server
bootstrap process in order to ensure that no data is sent to an
elasticsearch instance running on the same host.

* Add CHANGELOG entry
michel-laterman added a commit that referenced this pull request Aug 17, 2021
)

* Disable monitoring during fleetserver bootstrap process

Set the monitoring.enabled flag to false during the fleet-server
bootstrap process in order to ensure that no data is sent to an
elasticsearch instance running on the same host.

* Add CHANGELOG entry
@amolnater-qasource
Copy link

Hi @EricDavisX
We have revalidated this issue on 7.15.0 snapshot Kibana self-managed environment and found this elastic/fleet-server#376 issue fixed now.

Observations:

  • No elastic_agent.filebeat/metricbeat available.
  • Other expected data is available under Data Streams tab.
DATASET Type Before Reboot After Reboot
elastic_agent.fleet_server metrics Generating Normally Generating Normally
elastic_agent.elastic_agent metrics Generating Normally Generating Normally
elastic_agent.fleet_server logs Generated only Once in beginning of Fleet Server agent installation. Generated only Once in beginning of Fleet Server agent reboot.
elastic_agent logs Generated only Once in beginning of Fleet Server agent installation. Generated only Once in beginning of Fleet Server agent reboot.

Build details:

Build: 43546
Commit: 81e86207a4e276826fb0744e78239370f448c1ef
Artifact Link: https://snapshots.elastic.co/7.15.0-4b8a78a4/downloads/beats/elastic-agent/elastic-agent-7.15.0-SNAPSHOT-windows-x86_64.zip

Screenshot:
10

Please let us know if anything else is required from our end.
cc: @michel-laterman
Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Elastic-Agent Label for the Agent team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Self managed]: elastic_agent.metricbeat/filebeat datastreams generated on installing fleet-server agent.
4 participants