Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.0](backport #30694) Fix/change the initialization of management layer #30806

Merged
merged 3 commits into from
Mar 21, 2022

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Mar 14, 2022

This is an automatic backport of pull request #30694 done by Mergify.


Mergify commands and options

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

  • @Mergifyio refresh will re-evaluate the rules
  • @Mergifyio rebase will rebase this PR on its base branch
  • @Mergifyio update will merge the base branch into this PR
  • @Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

  • look at your merge queues
  • generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.com

* Ensure that libbeat manager is instantiated after the hooks.

This fix an issues on Filebeat that make the start sequence of filebeat
non-deterministic. It was possible that not all the hooks were
configured correctly before the managed was receiving a configuration
from the Elastic Agent.

This causes an inconsistency between the expected configuration state
and the actual running states, this includes the following symptoms:

- Having Filebeat runnings and not sending any data to Elasticsearch
- Having Filebeat partially configured, when only some inputs were
  sending data.
- Missing log from the Filebeat collector
- Having only metricsbeats running and sending logs.

This solves the issues by moving the `Start` and stop `Stop` of the
managed into the beats initialization process, each beats need to be
adjusted to support. This is indeed a breaking changes for beats author,
but the bootstrap process of beats and libbeat cannot easily be
extended to make the change into a unique place.

(cherry picked from commit 4c14f03)
@mergify mergify bot requested a review from a team as a code owner March 14, 2022 16:23
@mergify mergify bot added the backport label Mar 14, 2022
@mergify mergify bot assigned ph Mar 14, 2022
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Mar 14, 2022
@ph ph closed this Mar 14, 2022
@ph ph reopened this Mar 14, 2022
@elasticmachine
Copy link
Collaborator

elasticmachine commented Mar 14, 2022

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-03-21T12:26:28.659+0000

  • Duration: 112 min 38 sec

Test stats 🧪

Test Results
Failed 0
Passed 42509
Skipped 3853
Total 46362

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@ph
Copy link
Contributor

ph commented Mar 16, 2022

@Mergifyio update

@mergify
Copy link
Contributor Author

mergify bot commented Mar 16, 2022

update

✅ Branch has been successfully updated

@ph
Copy link
Contributor

ph commented Mar 16, 2022

Fix for metricbeat k8s #30870

@mergify
Copy link
Contributor Author

mergify bot commented Mar 17, 2022

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b mergify/bp/8.0/pr-30694 upstream/mergify/bp/8.0/pr-30694
git merge upstream/8.0
git push upstream mergify/bp/8.0/pr-30694

@mergify
Copy link
Contributor Author

mergify bot commented Mar 21, 2022

This pull request has not been merged yet. Could you please review and merge it @ph? 🙏

@ph
Copy link
Contributor

ph commented Mar 21, 2022

@v1v Any reason why this PR is stuck on testing?

I've restarted it a few times.

@v1v
Copy link
Member

v1v commented Mar 21, 2022

@v1v Any reason why this PR is stuck on testing?

My guess, GitHub was a bit temperamental last week, so likely this is the reason for this (it was about 4 days ago), I see the build failed while interacting with GitHub

image

@v1v
Copy link
Member

v1v commented Mar 21, 2022

/test

@v1v
Copy link
Member

v1v commented Mar 21, 2022

run elasticsearch-ci/docs

@mtojek mtojek added the Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team label Mar 21, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Mar 21, 2022
@ph ph merged commit af6bfb8 into 8.0 Mar 21, 2022
@mergify mergify bot deleted the mergify/bp/8.0/pr-30694 branch March 21, 2022 15:15
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
…astic#30806)


This fix an issues on Filebeat that make the start sequence of filebeat
non-deterministic. It was possible that not all the hooks were
configured correctly before the managed was receiving a configuration
from the Elastic Agent.

This causes an inconsistency between the expected configuration state
and the actual running states, this includes the following symptoms:

- Having Filebeat runnings and not sending any data to Elasticsearch
- Having Filebeat partially configured, when only some inputs were
  sending data.
- Missing log from the Filebeat collector
- Having only metricsbeats running and sending logs.

This solves the issues by moving the `Start` and stop `Stop` of the
managed into the beats initialization process, each beats need to be
adjusted to support. This is indeed a breaking changes for beats author,
but the bootstrap process of beats and libbeat cannot easily be
extended to make the change into a unique place.

(cherry picked from commit 48da76f)

Co-authored-by: Pier-Hugues Pellerin <phpellerin@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants