chore: update GCP bucket calculation #306

mdelapenya · 2020-09-16T14:49:43Z

What does this PR do?

As a result of elastic/beats#20903, we detected the URLs to get the snapshots needed to be updated. This PR is addressing this need, including some refactors:

move code specific to the elastic-agent to the downloadAgentBinary method, instead of overloading the generic one that uses the artifactory to get the artifacts (separation of concerns). This change will enable using a default value for the version, because snapshots for PRs hardcode the target version in the file name (i.e. /pull-requests/pr-21100/elastic-agent-8.0.0-SNAPSHOT-x86_64.rpm).
fix the download process from a GCP bucket: we only used the first page in the response, being now possible to paginate over the different pages in the response following the nextPageToken attribute in the response. If this attribute is present, and we did not find the object in the current page, we continue paginating using the token as a query string parameter. This way, the backoff strategy will fail (and retry) if:
- the endpoint URL for the GCP bucket is not available
- it's not possible to parse response into a JSON object
- we reached the end of the current page and the object is not present, having more pages (controlled by the nextPageToken attribute)
- we reached the end of the current page and the object is not present, not having more pages (the nextPageToken attribute is not present)

If the object is present, then we return it's media-link, which is the public URL for the object in the bucket.

Why is it important?

This change will allow us to enable the execution of the test suite alongside any PR, simply setting the environment with a few values to use the CI snapshots for branches and pull requests:

for branches, we must just set ELASTIC_AGENT_USE_CI_SNAPSHOTS=true. It will use snapshots for the current branch
for pull requests, we must just set ELASTIC_AGENT_USE_CI_SNAPSHOTS=true and ELASTIC_AGENT_VERSION=pr-21100. It will use snapshots for the PR identified by the agent version (i.e. pr-21100 represents the Github PR with 21100 as PR ID).

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have run the Unit tests for the CLI, and they are passing locally
I have run the End-2-End tests for the suite I'm working on, and they are passing locally
I have noticed new Go dependencies (run make notice in the proper directory)

Author's Checklist

Run tests without the snapshots variables: the test suite fetches what is described in the compose files and/or default values for agent in the current branch (Stack will honour compose files). This is the default behavior when developing on this project.
Run tests for CI snapshots without setting agent's version: the test suite fetches the snapshot for the current branch that was already pushed to GCP (Stack will honour compose files). This is the behavior for branches in the Beats repo, which will trigger the test suite right after the packaging job finishes.
Run tests for CI snapshots setting agent's version: the test suite fetches the snapshot for the current pull request that was already pushed to GCP (Stack will honour compose files). This is the behavior for PRs in the Beats repo, which will trigger the test suite right after the packaging job finishes.
Run tests for CI snapshots setting a non existent version for the agent: the test suite is not able to find the object in the GCP bucket, failing.

How to test this PR locally

Testing old behavior

$ OP_LOG_LEVEL=TRACE DEVELOPER_MODE=true godog -t "stand_alone_mode"
$ OP_LOG_LEVEL=TRACE DEVELOPER_MODE=true godog -t "fleet_mode"

It will use the elastic-agent:8.0.0-SNAPSHOT image, and the agent binaries from the official artifactory.

Testing an already packaged branch

$ OP_LOG_LEVEL=TRACE DEVELOPER_MODE=true ELASTIC_AGENT_USE_CI_SNAPSHOTS=true godog -t "stand_alone_mode"
$ OP_LOG_LEVEL=TRACE DEVELOPER_MODE=true ELASTIC_AGENT_USE_CI_SNAPSHOTS=true godog -t "fleet_mode"

It will use the elastic-agent image built for the branch, and the agent binaries snapshots from the GCP bucket.

Testing an already packaged pull request

$ OP_LOG_LEVEL=TRACE DEVELOPER_MODE=true ELASTIC_AGENT_USE_CI_SNAPSHOTS=true ELASTIC_AGENT_VERSION=pr-21100 godog -t "stand_alone_mode"
$ OP_LOG_LEVEL=TRACE DEVELOPER_MODE=true ELASTIC_AGENT_USE_CI_SNAPSHOTS=true ELASTIC_AGENT_VERSION=pr-21100 godog -t "fleet_mode"

It will use the elastic-agent image built for the PR, and the agent binaries snapshots from the GCP bucket.

Related issues

Use cases

Scenario: Run tests while developing the test project
When the test framework is changed
Then it uses versions defined in the compose files

Scenario: Run tests for Beats branches
Given a branch on Beats
When the packages (binaries and images) for the PR are published after a merge
Then the test project uses versions for the branch

Scenario: Run tests for Beats pull requests
Given a PR on Beats
When the packages (binaries and images) for the PR are published with the packaging job
Then the test project uses versions for the PR

Follow-ups

elasticmachine · 2020-09-16T15:05:39Z

💔 Tests Failed

Expand to view the summary

Build stats

Build Cause: [Pull request #306 updated]
Start Time: 2020-09-16T15:51:29.800+0000
Duration: 43 min 42 sec

Test stats 🧪

Test	Results
Failed	9
Passed	51
Skipped	20
Total	80

Test errors

Expand to view the tests failures

Name: Initializing / End-To-End Tests / ingest-manager_agent_endpoint_integration / Adding the Endpoint Integration to an Agent makes the host to show in Security App – Agent Endpoint Integration
- Age: 2
- Duration: 275.81924
- Error Details: Step the host name is shown in the Administration view in the Security App as "online": The host 0a43a6a7f6af is not listed in the Administration view in the Security App
Name: Initializing / End-To-End Tests / ingest-manager_agent_endpoint_integration / Deploying an Endpoint makes policies to appear in the Security App – Agent Endpoint Integration
- Age: 2
- Duration: 269.97897
- Error Details: Step the policy response will be shown in the Security App: The policy response is not listed as 'success' in the Administration view in the Security App yet
Name: Initializing / End-To-End Tests / ingest-manager_fleet_mode / Deploying the centos agent – Fleet Mode Agent
- Age: 2
- Duration: 269.03558
- Error Details: Step system package dashboards are listed in Fleet: There are no datastreams yet
Name: Initializing / End-To-End Tests / ingest-manager_fleet_mode / Deploying the debian agent – Fleet Mode Agent
- Age: 2
- Duration: 271.36026
- Error Details: Step system package dashboards are listed in Fleet: There are no datastreams yet
Name: Initializing / End-To-End Tests / ingest-manager_fleet_mode / Starting the centos agent starts backend processes – Fleet Mode Agent
- Age: 2
- Duration: 251.01248
- Error Details: Step the "metricbeat" process is in the "started" state on the host
Name: Initializing / End-To-End Tests / ingest-manager_fleet_mode / Starting the debian agent starts backend processes – Fleet Mode Agent
- Age: 2
- Duration: 253.87706
- Error Details: Step the "metricbeat" process is in the "started" state on the host
Name: Initializing / End-To-End Tests / ingest-manager_fleet_mode / Restarting the centos host with persistent agent restarts backend processes – Fleet Mode Agent
- Age: 2
- Duration: 263.11526
- Error Details: Step the "metricbeat" process is in the "started" state on the host
Name: Initializing / End-To-End Tests / ingest-manager_fleet_mode / Restarting the debian host with persistent agent restarts backend processes – Fleet Mode Agent
- Age: 2
- Duration: 247.66083
- Error Details: Step the "metricbeat" process is in the "started" state on the host
Name: Initializing / End-To-End Tests / ingest-manager_stand_alone_mode / Deploying a stand-alone agent – Stand-alone Agent
- Age: 3
- Duration: 179.12886
- Error Details: Step there is new data in the index from agent: Not enough hits in the index yet. Current: 0, Desired: 50

Steps errors

Expand to view the steps failures

Name: Run functional tests for ingest-manager:agent_endpoint_integration
- Description:
- Duration: 17 min 21 sec
- Start Time: 2020-09-16T15:56:19.231+0000
- log
Name: Error signal
- Description:
- Duration: 0 min 0 sec
- Start Time: 2020-09-16T16:13:40.377+0000
- log
Name: General Build Step
- Description: [2020-09-16T16:13:40.866Z] Archiving artifacts
  hudson.AbortException: script returned exit code 1
- Duration: 0 min 0 sec
- Start Time: 2020-09-16T16:13:40.861+0000
- log
Name: Run functional tests for ingest-manager:stand_alone_mode
- Description:
- Duration: 8 min 49 sec
- Start Time: 2020-09-16T15:56:16.739+0000
- log
Name: Error signal
- Description:
- Duration: 0 min 0 sec
- Start Time: 2020-09-16T16:04:05.485+0000
- log
Name: General Build Step
- Description: [2020-09-16T16:04:05.945Z] Archiving artifacts
  hudson.AbortException: script returned exit code 1
- Duration: 0 min 0 sec
- Start Time: 2020-09-16T16:04:05.940+0000
- log
Name: Run functional tests for ingest-manager:fleet_mode
- Description:
- Duration: 38 min 48 sec
- Start Time: 2020-09-16T15:56:20.662+0000
- log
Name: Error signal
- Description:
- Duration: 0 min 0 sec
- Start Time: 2020-09-16T16:34:09.192+0000
- log
Name: General Build Step
- Description: [2020-09-16T16:34:09.777Z] Archiving artifacts
  hudson.AbortException: script returned exit code 1
- Duration: 0 min 0 sec
- Start Time: 2020-09-16T16:34:09.767+0000
- log

Log output

Expand to view the last 100 lines of log output

[2020-09-16T16:33:46.192Z] Building dependency tree...
[2020-09-16T16:33:46.192Z] Reading state information...
[2020-09-16T16:33:46.192Z] The following NEW packages will be installed:
[2020-09-16T16:33:46.192Z]   elastic-agent
[2020-09-16T16:33:48.127Z] 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
[2020-09-16T16:33:48.127Z] Need to get 0 B/110 MB of archives.
[2020-09-16T16:33:48.127Z] After this operation, 131 MB of additional disk space will be used.
[2020-09-16T16:33:48.127Z] Get:1 /elastic-agent-8.0.0-SNAPSHOT-amd64.deb elastic-agent amd64 8.0.0 [110 MB]
[2020-09-16T16:33:48.390Z] debconf: delaying package configuration, since apt-utils is not installed
[2020-09-16T16:33:48.390Z] Selecting previously unselected package elastic-agent.
[2020-09-16T16:33:48.390Z] (Reading database ... 
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 7400 files and directories currently installed.)
[2020-09-16T16:33:48.390Z] Preparing to unpack .../elastic-agent-8.0.0-SNAPSHOT-amd64.deb ...
[2020-09-16T16:33:48.390Z] Unpacking elastic-agent (8.0.0) ...
[2020-09-16T16:33:49.347Z] Setting up elastic-agent (8.0.0) ...
[2020-09-16T16:33:49.617Z] Processing triggers for systemd (232-25+deb9u12) ...
[2020-09-16T16:33:49.887Z] time="2020-09-16T16:33:49Z" level=debug msg="Docker compose executed." cmd="[exec -T debian-systemd apt install /elastic-agent-8.0.0-SNAPSHOT-amd64.deb -y]" composeFilePaths="[/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-306/.op/compose/profiles/ingest-manager/docker-compose.yml /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-306/.op/compose/services/debian-systemd/docker-compose.yml]" env="map[centos_systemdAgentBinarySrcPath:/tmp/elastic-agent-8.0.0-SNAPSHOT-x86_64.rpm879130885 centos_systemdAgentBinaryTargetPath:/elastic-agent-8.0.0-SNAPSHOT-x86_64.rpm centos_systemdContainerName:ingest-manager_centos-systemd_centos-systemd_2 centos_systemdTag:latest debian_systemdAgentBinarySrcPath:/tmp/elastic-agent-8.0.0-SNAPSHOT-amd64.deb711839904 debian_systemdAgentBinaryTargetPath:/elastic-agent-8.0.0-SNAPSHOT-amd64.deb debian_systemdContainerName:ingest-manager_debian-systemd_debian-systemd_2 debian_systemdTag:stretch kibanaConfigPath:/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-306/src/github.com/elastic/e2e-testing/e2e/_suites/ingest-manager/configurations/kibana.config.yml stackVersion:8.0.0-SNAPSHOT]" profile=ingest-manager
[2020-09-16T16:33:50.463Z] Synchronizing state of elastic-agent.service with SysV service script with /lib/systemd/systemd-sysv-install.
[2020-09-16T16:33:50.463Z] Executing: /lib/systemd/systemd-sysv-install enable elastic-agent
[2020-09-16T16:33:50.738Z] time="2020-09-16T16:33:50Z" level=debug msg="Docker compose executed." cmd="[exec -T debian-systemd systemctl enable elastic-agent]" composeFilePaths="[/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-306/.op/compose/profiles/ingest-manager/docker-compose.yml /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-306/.op/compose/services/debian-systemd/docker-compose.yml]" env="map[centos_systemdAgentBinarySrcPath:/tmp/elastic-agent-8.0.0-SNAPSHOT-x86_64.rpm879130885 centos_systemdAgentBinaryTargetPath:/elastic-agent-8.0.0-SNAPSHOT-x86_64.rpm centos_systemdContainerName:ingest-manager_centos-systemd_centos-systemd_2 centos_systemdTag:latest debian_systemdAgentBinarySrcPath:/tmp/elastic-agent-8.0.0-SNAPSHOT-amd64.deb711839904 debian_systemdAgentBinaryTargetPath:/elastic-agent-8.0.0-SNAPSHOT-amd64.deb debian_systemdContainerName:ingest-manager_debian-systemd_debian-systemd_2 debian_systemdTag:stretch kibanaConfigPath:/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-306/src/github.com/elastic/e2e-testing/e2e/_suites/ingest-manager/configurations/kibana.config.yml stackVersion:8.0.0-SNAPSHOT]" profile=ingest-manager
[2020-09-16T16:33:51.692Z] The Elastic Agent is currently in BETA and should not be used in production
[2020-09-16T16:33:52.001Z] 2020-09-16T16:33:51.932Z	DEBUG	kibana/client.go:170	Request method: POST, path: /api/ingest_manager/fleet/agents/enroll
[2020-09-16T16:33:52.278Z] fail to enroll: fail to execute request to Kibana: Status code: 401, Kibana returned an error: Unauthorized, message: [security_exception] missing authentication credentials for REST request [/_security/_authenticate], with { header={ WWW-Authenticate={ 0="ApiKey" & 1="Basic realm=\"security\" charset=\"UTF-8\"" } } }
[2020-09-16T16:33:52.278Z] time="2020-09-16T16:33:52Z" level=error msg="Could not execute command in container" command="[elastic-agent enroll http://kibana:5601 LXJuRGwzUUJNWnhxZUNSUHFuODI6ZDc5ZWtITm9UMUtlam14aW05OWxJdw== -f --insecure]" error="Could not run compose file: [/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-306/.op/compose/profiles/ingest-manager/docker-compose.yml /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-306/.op/compose/services/debian-systemd/docker-compose.yml] - Local Docker compose exited abnormally whilst running docker-compose: [exec -T debian-systemd elastic-agent enroll http://kibana:5601 LXJuRGwzUUJNWnhxZUNSUHFuODI6ZDc5ZWtITm9UMUtlam14aW05OWxJdw== -f --insecure]. exit status 1" service=debian-systemd
[2020-09-16T16:33:52.279Z] time="2020-09-16T16:33:52Z" level=error msg="Could not enroll the agent with the token" command="[elastic-agent enroll http://kibana:5601 LXJuRGwzUUJNWnhxZUNSUHFuODI6ZDc5ZWtITm9UMUtlam14aW05OWxJdw== -f --insecure]" error="Could not run compose file: [/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-306/.op/compose/profiles/ingest-manager/docker-compose.yml /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-306/.op/compose/services/debian-systemd/docker-compose.yml] - Local Docker compose exited abnormally whilst running docker-compose: [exec -T debian-systemd elastic-agent enroll http://kibana:5601 LXJuRGwzUUJNWnhxZUNSUHFuODI6ZDc5ZWtITm9UMUtlam14aW05OWxJdw== -f --insecure]. exit status 1" image=debian-systemd service=debian-systemd tag=stretch token="LXJuRGwzUUJNWnhxZUNSUHFuODI6ZDc5ZWtITm9UMUtlam14aW05OWxJdw=="
[2020-09-16T16:33:52.279Z] time="2020-09-16T16:33:52Z" level=debug msg="As expected, it's not possible to enroll an agent with a revoked token" err="Could not run compose file: [/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-306/.op/compose/profiles/ingest-manager/docker-compose.yml /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-306/.op/compose/services/debian-systemd/docker-compose.yml] - Local Docker compose exited abnormally whilst running docker-compose: [exec -T debian-systemd elastic-agent enroll http://kibana:5601 LXJuRGwzUUJNWnhxZUNSUHFuODI6ZDc5ZWtITm9UMUtlam14aW05OWxJdw== -f --insecure]. exit status 1" token="LXJuRGwzUUJNWnhxZUNSUHFuODI6ZDc5ZWtITm9UMUtlam14aW05OWxJdw=="
[2020-09-16T16:33:52.279Z] time="2020-09-16T16:33:52Z" level=warning msg="Could not create command in container" command="[cat /etc/elastic-agent/.elastic-agent.active.commit]" container=ff741ae7a6d3 detach=false error="Error response from daemon: No such container: ff741ae7a6d3" tty=false
[2020-09-16T16:33:52.279Z] time="2020-09-16T16:33:52Z" level=error msg="Could not get agent hash in the container" containerName=ff741ae7a6d3 error="Error response from daemon: No such container: ff741ae7a6d3"
[2020-09-16T16:33:52.279Z] time="2020-09-16T16:33:52Z" level=debug msg="Un-enrolling agent in Fleet" agentID=77718db2-8150-4f18-8f16-fb45fa8fb617 hostname=ff741ae7a6d3
[2020-09-16T16:33:53.225Z] time="2020-09-16T16:33:53Z" level=debug msg="Fleet agent was unenrolled" agentID=77718db2-8150-4f18-8f16-fb45fa8fb617
[2020-09-16T16:33:54.189Z] Stopping ingest-manager_debian-systemd_debian-systemd_2 ... 
[2020-09-16T16:33:55.138Z] 
Stopping ingest-manager_debian-systemd_debian-systemd_2 ... done
Removing ingest-manager_debian-systemd_debian-systemd_2 ... 
[2020-09-16T16:33:55.401Z] 
Removing ingest-manager_debian-systemd_debian-systemd_2 ... done
Going to remove ingest-manager_debian-systemd_debian-systemd_2
[2020-09-16T16:33:55.402Z] time="2020-09-16T16:33:55Z" level=debug msg="Docker compose executed." cmd="[rm -fvs debian-systemd]" composeFilePaths="[/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-306/.op/compose/profiles/ingest-manager/docker-compose.yml /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-306/.op/compose/services/debian-systemd/docker-compose.yml]" env="map[centos_systemdAgentBinarySrcPath:/tmp/elastic-agent-8.0.0-SNAPSHOT-x86_64.rpm879130885 centos_systemdAgentBinaryTargetPath:/elastic-agent-8.0.0-SNAPSHOT-x86_64.rpm centos_systemdContainerName:ingest-manager_centos-systemd_centos-systemd_2 centos_systemdTag:latest debian_systemdAgentBinarySrcPath:/tmp/elastic-agent-8.0.0-SNAPSHOT-amd64.deb711839904 debian_systemdAgentBinaryTargetPath:/elastic-agent-8.0.0-SNAPSHOT-amd64.deb debian_systemdContainerName:ingest-manager_debian-systemd_debian-systemd_2 debian_systemdTag:stretch kibanaConfigPath:/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-306/src/github.com/elastic/e2e-testing/e2e/_suites/ingest-manager/configurations/kibana.config.yml stackVersion:8.0.0-SNAPSHOT]" profile=ingest-manager
[2020-09-16T16:33:55.402Z] time="2020-09-16T16:33:55Z" level=debug msg="Service removed from compose" profile=ingest-manager service=debian-systemd
[2020-09-16T16:33:56.349Z] time="2020-09-16T16:33:56Z" level=debug msg="The token was deleted" tokenID=6f9d3955-6e48-4e11-ba17-b12f41b4989d
[2020-09-16T16:33:56.349Z] time="2020-09-16T16:33:56Z" level=info msg="Integration deleted from the configuration" integration= packageConfigId= policyID=566654c0-f83a-11ea-8025-f70c52269b82 version=
[2020-09-16T16:34:02.965Z] time="2020-09-16T16:34:02Z" level=debug msg="The policy was deleted" policyID=566654c0-f83a-11ea-8025-f70c52269b82
[2020-09-16T16:34:02.965Z] time="2020-09-16T16:34:02Z" level=debug msg="Destroying ingest-manager runtime dependencies"
[2020-09-16T16:34:02.965Z] Stopping ingest-manager_kibana_1           ... 
[2020-09-16T16:34:02.965Z] Stopping ingest-manager_elasticsearch_1    ... 
[2020-09-16T16:34:02.965Z] Stopping ingest-manager_package-registry_1 ... 
[2020-09-16T16:34:08.870Z] 
Stopping ingest-manager_kibana_1           ... done

Stopping ingest-manager_package-registry_1 ... done

Stopping ingest-manager_elasticsearch_1    ... done
Removing ingest-manager_kibana_1           ... 
[2020-09-16T16:34:08.870Z] Removing ingest-manager_elasticsearch_1    ... 
[2020-09-16T16:34:08.870Z] Removing ingest-manager_package-registry_1 ... 
[2020-09-16T16:34:08.870Z] 
Removing ingest-manager_package-registry_1 ... done

Removing ingest-manager_elasticsearch_1    ... done

Removing ingest-manager_kibana_1           ... done
Removing network ingest-manager_default
[2020-09-16T16:34:09.133Z] time="2020-09-16T16:34:08Z" level=debug msg="Docker compose executed." cmd="[down]" composeFilePaths="[/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-306/.op/compose/profiles/ingest-manager/docker-compose.yml]" env="map[centos_systemdAgentBinarySrcPath:/tmp/elastic-agent-8.0.0-SNAPSHOT-x86_64.rpm879130885 centos_systemdAgentBinaryTargetPath:/elastic-agent-8.0.0-SNAPSHOT-x86_64.rpm centos_systemdContainerName:ingest-manager_centos-systemd_centos-systemd_2 centos_systemdTag:latest debian_systemdAgentBinarySrcPath:/tmp/elastic-agent-8.0.0-SNAPSHOT-amd64.deb711839904 debian_systemdAgentBinaryTargetPath:/elastic-agent-8.0.0-SNAPSHOT-amd64.deb debian_systemdContainerName:ingest-manager_debian-systemd_debian-systemd_2 debian_systemdTag:stretch kibanaConfigPath:/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-306/src/github.com/elastic/e2e-testing/e2e/_suites/ingest-manager/configurations/kibana.config.yml stackVersion:8.0.0-SNAPSHOT]" profile=ingest-manager
[2020-09-16T16:34:09.133Z] time="2020-09-16T16:34:08Z" level=debug msg="Elastic Agent binary was removed." installer=centos-systemd path=/tmp/elastic-agent-8.0.0-SNAPSHOT-x86_64.rpm879130885
[2020-09-16T16:34:09.133Z] time="2020-09-16T16:34:08Z" level=debug msg="Elastic Agent binary was removed." installer=debian-systemd path=/tmp/elastic-agent-8.0.0-SNAPSHOT-amd64.deb711839904
[2020-09-16T16:34:09.133Z] <?xml version="1.0" encoding="UTF-8"?>
[2020-09-16T16:34:09.133Z] <testsuites name="main" tests="14" skipped="0" failures="6" errors="0" time="2231.897145573">
[2020-09-16T16:34:09.133Z]   <testsuite name="Fleet Mode Agent" tests="14" skipped="0" failures="6" errors="0" time="2025.221692524">
[2020-09-16T16:34:09.133Z]     <testcase name="Deploying the centos agent" status="failed" time="269.035578145">
[2020-09-16T16:34:09.133Z]       <failure message="Step system package dashboards are listed in Fleet: There are no datastreams yet"></failure>
[2020-09-16T16:34:09.133Z]     </testcase>
[2020-09-16T16:34:09.133Z]     <testcase name="Deploying the debian agent" status="failed" time="271.360273215">
[2020-09-16T16:34:09.133Z]       <failure message="Step system package dashboards are listed in Fleet: There are no datastreams yet"></failure>
[2020-09-16T16:34:09.133Z]     </testcase>
[2020-09-16T16:34:09.133Z]     <testcase name="Starting the centos agent starts backend processes" status="failed" time="251.012482082">
[2020-09-16T16:34:09.133Z]       <failure message="Step the &#34;filebeat&#34; process is in the &#34;started&#34; state on the host: filebeat process is not running in the container yet"></failure>
[2020-09-16T16:34:09.133Z]       <error message="Step the &#34;metricbeat&#34; process is in the &#34;started&#34; state on the host" type="skipped"></error>
[2020-09-16T16:34:09.133Z]     </testcase>
[2020-09-16T16:34:09.133Z]     <testcase name="Starting the debian agent starts backend processes" status="failed" time="253.877058365">
[2020-09-16T16:34:09.133Z]       <failure message="Step the &#34;filebeat&#34; process is in the &#34;started&#34; state on the host: filebeat process is not running in the container yet"></failure>
[2020-09-16T16:34:09.133Z]       <error message="Step the &#34;metricbeat&#34; process is in the &#34;started&#34; state on the host" type="skipped"></error>
[2020-09-16T16:34:09.133Z]     </testcase>
[2020-09-16T16:34:09.133Z]     <testcase name="Stopping the centos agent stops backend processes" status="passed" time="12.561832652"></testcase>
[2020-09-16T16:34:09.133Z]     <testcase name="Stopping the debian agent stops backend processes" status="passed" time="14.587074624"></testcase>
[2020-09-16T16:34:09.133Z]     <testcase name="Restarting the centos host with persistent agent restarts backend processes" status="failed" time="263.115278039">
[2020-09-16T16:34:09.133Z]       <failure message="Step the &#34;filebeat&#34; process is in the &#34;started&#34; state on the host: filebeat process is not running in the container yet"></failure>
[2020-09-16T16:34:09.133Z]       <error message="Step the &#34;metricbeat&#34; process is in the &#34;started&#34; state on the host" type="skipped"></error>
[2020-09-16T16:34:09.133Z]     </testcase>
[2020-09-16T16:34:09.133Z]     <testcase name="Restarting the debian host with persistent agent restarts backend processes" status="failed" time="247.660825871">
[2020-09-16T16:34:09.133Z]       <failure message="Step the &#34;filebeat&#34; process is in the &#34;started&#34; state on the host: filebeat process is not running in the container yet"></failure>
[2020-09-16T16:34:09.133Z]       <error message="Step the &#34;metricbeat&#34; process is in the &#34;started&#34; state on the host" type="skipped"></error>
[2020-09-16T16:34:09.134Z]     </testcase>
[2020-09-16T16:34:09.134Z]     <testcase name="Un-enrolling the centos agent" status="passed" time="15.341217013"></testcase>
[2020-09-16T16:34:09.134Z]     <testcase name="Un-enrolling the debian agent" status="passed" time="12.467095534"></testcase>
[2020-09-16T16:34:09.134Z]     <testcase name="Re-enrolling the centos agent" status="passed" time="40.205129765"></testcase>
[2020-09-16T16:34:09.134Z]     <testcase name="Re-enrolling the debian agent" status="passed" time="27.449298983"></testcase>
[2020-09-16T16:34:09.134Z]     <testcase name="Revoking the enrollment token for the centos agent" status="passed" time="30.390348134"></testcase>
[2020-09-16T16:34:09.134Z]     <testcase name="Revoking the enrollment token for the debian agent" status="passed" time="23.171643867"></testcase>
[2020-09-16T16:34:09.134Z]   </testsuite>
[2020-09-16T16:34:09.134Z] </testsuites>make: *** [functional-test] Error 1
[2020-09-16T16:34:09.134Z] Makefile:45: recipe for target 'functional-test' failed
[2020-09-16T16:34:09.134Z] + echo 'ERROR: functional-test failed'
[2020-09-16T16:34:09.134Z] ERROR: functional-test failed
[2020-09-16T16:34:09.134Z] + exit_status=1
[2020-09-16T16:34:09.134Z] + sed -e 's/^[ \t]*//; s#>.*failed$#>#g' outputs/TEST-ingest-manager-fleet_mode
[2020-09-16T16:34:09.134Z] + grep -E '^<.*>$'
[2020-09-16T16:34:09.134Z] + exit 1
[2020-09-16T16:34:09.206Z] Recording test results
[2020-09-16T16:34:09.777Z] Archiving artifacts
[2020-09-16T16:34:09.901Z] Failed in branch ingest-manager_fleet_mode
[2020-09-16T16:34:10.900Z] Stage "Release" skipped due to earlier failure(s)
[2020-09-16T16:34:11.482Z] Running on Jenkins in /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-306@2
[2020-09-16T16:34:11.531Z] [INFO] getVaultSecret: Getting secrets
[2020-09-16T16:34:11.611Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-09-16T16:34:12.309Z] + chmod 755 generate-build-data.sh
[2020-09-16T16:34:12.309Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/e2e-tests/e2e-testing-mbp/PR-306/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/e2e-tests/e2e-testing-mbp/PR-306/runs/3 FAILURE 2562249
[2020-09-16T16:34:12.310Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/e2e-tests/e2e-testing-mbp/PR-306/runs/3/steps/?limit=10000 -o steps-info.json

EricDavisX

YES!

mdelapenya · 2020-09-16T15:51:06Z

Errors seem to be the same as in upstream:

backend process not being preset
index appearing and disappearing

I suggest merging

mdelapenya · 2020-09-16T16:36:35Z

The failing tests are always the same:

backend processes not starting
no datastreams (flaky)
security app

* fix: normalise GCP bucket structure * chore: extract agent download logic from the helper method * chore: simplify * feat: support pagination when retrieving objects from a GCP bucket * chore: explain we reached the end of the list # Conflicts: # e2e/utils.go

ph · 2020-09-16T17:51:11Z

<3 @mdelapenya Thanks

* fix: normalise GCP bucket structure * chore: extract agent download logic from the helper method * chore: simplify * feat: support pagination when retrieving objects from a GCP bucket * chore: explain we reached the end of the list # Conflicts: # e2e/utils.go

mdelapenya added 4 commits September 16, 2020 13:57

fix: normalise GCP bucket structure

3e6f42d

chore: extract agent download logic from the helper method

22c6ce0

chore: simplify

d0fda2b

feat: support pagination when retrieving objects from a GCP bucket

9755ed0

mdelapenya self-assigned this Sep 16, 2020

mdelapenya requested review from a team and EricDavisX September 16, 2020 14:49

chore: explain we reached the end of the list

eb8a744

mdelapenya marked this pull request as ready for review September 16, 2020 15:19

mdelapenya changed the title ~~chore: add a check for the elastic-agent process in the main scenario~~ chore: update GGCP bucket calculation Sep 16, 2020

mdelapenya changed the title ~~chore: update GGCP bucket calculation~~ chore: update GCP bucket calculation Sep 16, 2020

mdelapenya mentioned this pull request Sep 16, 2020

feat: add a new step to run the e2e tests for certain parts of Beats elastic/beats#21100

Merged

EricDavisX approved these changes Sep 16, 2020

View reviewed changes

Merge branch 'master' into fix-bucket-paths-for-ci-snapshots

156951f

kuisathaverat approved these changes Sep 16, 2020

View reviewed changes

mdelapenya merged commit 062edab into elastic:master Sep 16, 2020

mdelapenya mentioned this pull request Sep 16, 2020

chore: update slack channel (#308) | chore: update GCP bucket calculation (#306) backport for 7.9.x #311

Merged

mdelapenya mentioned this pull request Sep 22, 2020

fix: calculate bucket path properly #318

Merged

9 tasks

mdelapenya deleted the fix-bucket-paths-for-ci-snapshots branch September 23, 2020 07:10

mdelapenya mentioned this pull request Nov 30, 2020

fix: honor ELASTIC_AGENT_VERSION when downloading the agent #480

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: update GCP bucket calculation #306

chore: update GCP bucket calculation #306

mdelapenya commented Sep 16, 2020 •

edited

Loading

elasticmachine commented Sep 16, 2020 •

edited

Loading

Build stats

Test stats 🧪

EricDavisX left a comment

mdelapenya commented Sep 16, 2020

mdelapenya commented Sep 16, 2020

ph commented Sep 16, 2020

chore: update GCP bucket calculation #306

chore: update GCP bucket calculation #306

Conversation

mdelapenya commented Sep 16, 2020 • edited Loading

What does this PR do?

Why is it important?

Checklist

Author's Checklist

How to test this PR locally

Testing old behavior

Testing an already packaged branch

Testing an already packaged pull request

Related issues

Use cases

Follow-ups

elasticmachine commented Sep 16, 2020 • edited Loading

💔 Tests Failed

Build stats

Test stats 🧪

Test errors

Steps errors

Log output

EricDavisX left a comment

Choose a reason for hiding this comment

mdelapenya commented Sep 16, 2020

mdelapenya commented Sep 16, 2020

ph commented Sep 16, 2020

mdelapenya commented Sep 16, 2020 •

edited

Loading

elasticmachine commented Sep 16, 2020 •

edited

Loading