Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cherry-pick #20387 to 7.x: [Elastic Agent] Add support for synthetics inputs that run heartbeat #20488

Closed
wants to merge 2 commits into from

Conversation

blakerouse
Copy link
Contributor

Cherry-pick of PR #20387 to 7.x branch. Original message:

What does this PR do?

This adds support for any input with synthetics/* to run heartbeat and send the inputs to heartbeat. This also bundles heartbeat with Elastic Agent in data/downloads.

Note: Currently heartbeat doesn't support the top-level inputs key. Once it does this will just work.

Why is it important?

To support running heartbeat with synthetics inputs.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

…lastic#20387)

* Add support for synthetics inputs that run heartbeat.

* Run mage fmt.

* Add changelog.

* Update test configs to use host instead of hosts.

* Enable support for top-level inputs from Elastic Agent.

(cherry picked from commit b153044)
@blakerouse blakerouse requested a review from a team as a code owner August 6, 2020 20:14
@elasticmachine
Copy link
Collaborator

Pinging @elastic/uptime (Team:Uptime)

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Aug 6, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/ingest-management (Team:Ingest Management)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Aug 6, 2020
@blakerouse blakerouse self-assigned this Aug 6, 2020
@elasticmachine
Copy link
Collaborator

elasticmachine commented Aug 6, 2020

❕ Build Aborted

Either there was a build timeout or someone aborted the build.'}

Pipeline View Test View Changes Artifacts

Expand to view the summary

Build stats

  • Build Cause: [Pull request #20488 updated]

  • Start Time: 2020-08-07T12:44:57.158+0000

  • Duration: 124 min 20 sec

Test stats 🧪

Test Results
Failed 0
Passed 16930
Skipped 1640
Total 18570

Steps errors

Expand to view the steps failures

  • Name: Error signal

    • Description:

    • Duration: 0 min 0 sec

    • Start Time: 2020-08-07T14:48:47.286+0000

    • log

  • Name: Notifies GitHub of the status of a Pull Request

    • Description: untar: step failed with error null

    • Duration: 0 min 2 sec

    • Start Time: 2020-08-07T14:49:04.432+0000

    • log

Log output

Expand to view the last 100 lines of log output

[2020-08-07T14:49:02.187Z]  Experimental:      false
[2020-08-07T14:49:02.187Z] 
[2020-08-07T14:49:02.187Z] Server: Docker Engine - Community
[2020-08-07T14:49:02.187Z]  Engine:
[2020-08-07T14:49:02.187Z]   Version:          19.03.12
[2020-08-07T14:49:02.187Z]   API version:      1.40 (minimum version 1.12)
[2020-08-07T14:49:02.187Z]   Go version:       go1.13.10
[2020-08-07T14:49:02.187Z]   Git commit:       48a66213fe
[2020-08-07T14:49:02.187Z]   Built:            Mon Jun 22 15:44:07 2020
[2020-08-07T14:49:02.187Z]   OS/Arch:          linux/amd64
[2020-08-07T14:49:02.187Z]   Experimental:     false
[2020-08-07T14:49:02.187Z]  containerd:
[2020-08-07T14:49:02.187Z]   Version:          1.2.13
[2020-08-07T14:49:02.187Z]   GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
[2020-08-07T14:49:02.187Z]  runc:
[2020-08-07T14:49:02.187Z]   Version:          1.0.0-rc10
[2020-08-07T14:49:02.187Z]   GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
[2020-08-07T14:49:02.187Z]  docker-init:
[2020-08-07T14:49:02.187Z]   Version:          0.18.0
[2020-08-07T14:49:02.187Z]   GitCommit:        fec3683
[2020-08-07T14:49:02.187Z] Unable to find image 'alpine:3.4' locally
[2020-08-07T14:49:02.761Z] 3.4: Pulling from library/alpine
[2020-08-07T14:49:03.023Z] c1e54eec4b57: Pulling fs layer
[2020-08-07T14:49:03.063Z] + rm source.tgz
[2020-08-07T14:49:03.285Z] c1e54eec4b57: Verifying Checksum
[2020-08-07T14:49:03.285Z] c1e54eec4b57: Download complete
[2020-08-07T14:49:03.549Z] c1e54eec4b57: Pull complete
[2020-08-07T14:49:03.549Z] Digest: sha256:b733d4a32c4da6a00a84df2ca32791bb03df95400243648d8c539e7b4cce329c
[2020-08-07T14:49:03.549Z] Status: Downloaded newer image for alpine:3.4
[2020-08-07T14:49:05.250Z] + python .ci/scripts/pre_archive_test.py
[2020-08-07T14:49:05.250Z] Copy ./x-pack/libbeat/build into build/x-pack/libbeat/build
[2020-08-07T14:49:05.269Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/build
[2020-08-07T14:49:05.297Z] Recording test results
[2020-08-07T14:49:06.342Z] Failed in branch Metricbeat Mac OS X
[2020-08-07T14:49:06.454Z] Stashed 2 file(s)
[2020-08-07T14:49:06.466Z] Archiving artifacts
[2020-08-07T14:49:06.863Z] + python .ci/scripts/search_system_tests.py
[2020-08-07T14:49:06.923Z] [INFO] system-tests=''. If no empty then let's create a tarball
[2020-08-07T14:49:08.194Z] Failed in branch Libbeat x-pack
[2020-08-07T14:49:08.336Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats
[2020-08-07T14:49:08.649Z] + find . -type f -name TEST*.xml -path */build/* -delete
[2020-08-07T14:49:08.664Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Lint
[2020-08-07T14:49:08.764Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Elastic-Agent-Mac-OS-X
[2020-08-07T14:49:08.849Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Elastic-Agent-x-pack
[2020-08-07T14:49:08.937Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Auditbeat-x-pack-Mac-OS-X
[2020-08-07T14:49:09.025Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Auditbeat-crosscompile
[2020-08-07T14:49:09.117Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Filebeat-x-pack-Mac-OS-X
[2020-08-07T14:49:09.210Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Dockerlogbeat
[2020-08-07T14:49:09.303Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Winlogbeat-oss
[2020-08-07T14:49:09.389Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Journalbeat-oss
[2020-08-07T14:49:09.480Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Filebeat-Mac-OS-X
[2020-08-07T14:49:09.565Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Metricbeat-x-pack-Mac-OS-X
[2020-08-07T14:49:09.653Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Auditbeat-oss-Mac-OS-X
[2020-08-07T14:49:09.738Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Functionbeat-x-pack
[2020-08-07T14:49:09.820Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Generators-Metricbeat-Linux
[2020-08-07T14:49:09.900Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Elastic-Agent-x-pack-Windows
[2020-08-07T14:49:09.984Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Packetbeat-Linux
[2020-08-07T14:49:10.074Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Metricbeat-OSS-Unit-tests
[2020-08-07T14:49:10.156Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Heartbeat-oss
[2020-08-07T14:49:10.240Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Metricbeat-crosscompile
[2020-08-07T14:49:10.326Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Auditbeat-x-pack
[2020-08-07T14:49:10.411Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Auditbeat-oss-Linux
[2020-08-07T14:49:10.504Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Auditbeat-oss-Windows
[2020-08-07T14:49:10.599Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Auditbeat-x-pack-Windows
[2020-08-07T14:49:10.685Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Heartbeat-Mac-OS-X
[2020-08-07T14:49:10.770Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Filebeat-x-pack-Windows
[2020-08-07T14:49:10.855Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Winlogbeat-Windows-x-pack
[2020-08-07T14:49:10.937Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Functionbeat-Mac-OS-X-x-pack
[2020-08-07T14:49:11.020Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Packetbeat-Mac-OS-X
[2020-08-07T14:49:11.110Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Filebeat-Windows
[2020-08-07T14:49:11.194Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Winlogbeat-Windows
[2020-08-07T14:49:11.289Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Metricbeat-Windows
[2020-08-07T14:49:11.376Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Metricbeat-x-pack-Windows
[2020-08-07T14:49:11.458Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Generators-Beat-Linux
[2020-08-07T14:49:11.538Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Heartbeat-Windows
[2020-08-07T14:49:11.620Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Filebeat-x-pack
[2020-08-07T14:49:11.703Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Functionbeat-Windows
[2020-08-07T14:49:11.793Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Filebeat-oss
[2020-08-07T14:49:11.875Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Packetbeat-Windows
[2020-08-07T14:49:11.956Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Metricbeat-OSS-Go-Integration-tests
[2020-08-07T14:49:12.040Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Libbeat-oss
[2020-08-07T14:49:12.121Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Metricbeat-OSS-Python-Integration-tests
[2020-08-07T14:49:12.208Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Generators-Metricbeat-Mac-OS-X
[2020-08-07T14:49:12.286Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Libbeat-crosscompile
[2020-08-07T14:49:12.366Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Libbeat-stress-tests
[2020-08-07T14:49:12.445Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Generators-Beat-Mac-OS-X
[2020-08-07T14:49:12.529Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Metricbeat-x-pack
[2020-08-07T14:49:12.610Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20488/src/github.com/elastic/beats/Libbeat-x-pack
[2020-08-07T14:49:12.983Z] + cat
[2020-08-07T14:49:12.983Z] + /usr/local/bin/runbld ./runbld-script
[2020-08-07T14:49:12.983Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-08-07T14:49:15.003Z] Body did not finish within grace period; terminating with extreme prejudice
[2020-08-07T14:49:15.048Z] ERROR: runbld post build action failed.
[2020-08-07T14:49:15.048Z] Timeout has been exceeded
[2020-08-07T14:49:15.776Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-20488
[2020-08-07T14:49:16.014Z] [INFO] getVaultSecret: Getting secrets
[2020-08-07T14:49:16.104Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-08-07T14:49:17.164Z] + chmod 755 generate-build-data.sh
[2020-08-07T14:49:17.164Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20488/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20488/runs/2 ABORTED 7459740
[2020-08-07T14:49:17.164Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20488/runs/2/steps/?limit=10000 -o steps-info.json

Copy link
Contributor

@michalpristas michalpristas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cherrypick looks ok

@blakerouse
Copy link
Contributor Author

#20498 is include in this backport.

@jsoriano jsoriano mentioned this pull request Aug 12, 2020
6 tasks
@andrewvc andrewvc mentioned this pull request Aug 17, 2020
4 tasks
@andrewvc
Copy link
Contributor

Closing, backport went through in #20633

@andrewvc andrewvc closed this Aug 19, 2020
andrewvc added a commit to andrewvc/beats that referenced this pull request Oct 6, 2020
This keeps most of the changes in
elastic#20488 but unbundles heartbeat
since it's not ready for prime time yet. We may as well keep the work
that has been done already IMHO.
andrewvc added a commit that referenced this pull request Oct 14, 2020
This keeps most of the changes in
#20488 but unbundles heartbeat
since it's not ready for prime time yet. We may as well keep the work
that has been done already IMHO.
andrewvc added a commit to andrewvc/beats that referenced this pull request Oct 14, 2020
This keeps most of the changes in
elastic#20488 but unbundles heartbeat
since it's not ready for prime time yet. We may as well keep the work
that has been done already IMHO.

(cherry picked from commit 39b1db8)
andrewvc added a commit to andrewvc/beats that referenced this pull request Oct 14, 2020
This keeps most of the changes in
elastic#20488 but unbundles heartbeat
since it's not ready for prime time yet. We may as well keep the work
that has been done already IMHO.

(cherry picked from commit 39b1db8)
andrewvc added a commit that referenced this pull request Oct 14, 2020
This keeps most of the changes in
#20488 but unbundles heartbeat
since it's not ready for prime time yet. We may as well keep the work
that has been done already IMHO.

(cherry picked from commit 39b1db8)
andrewvc added a commit that referenced this pull request Oct 14, 2020
This keeps most of the changes in
#20488 but unbundles heartbeat
since it's not ready for prime time yet. We may as well keep the work
that has been done already IMHO.

(cherry picked from commit 39b1db8)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport Team:obs-ds-hosted-services Label for the Observability Hosted Services team [zube]: In Review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants