From 00230cfe0c23ccbdbbe217f549f64a45f24429a6 Mon Sep 17 00:00:00 2001 From: Alex Szabo Date: Wed, 4 Sep 2024 12:12:27 +0200 Subject: [PATCH] [CI] use kibana_install_dir from built artifacts (#191909) ## Summary I noticed recently, that the step "Copying ES Snapshot cache" got cancelled due to a lost agent. On closer inspection, it leads to the changed file (`.buildkite/scripts/steps/functional/osquery_cypress.sh`) where with a note, there's a kibana rebuild. Unless there's something special in that rebuild, we should be using the artifacts built in the build step. So let's follow the TODO notes, and use the downloaded artifacts - this should save ~6m per step where the osquery tests were warmed up. ## Before From [on-merge#49584](https://buildkite.com/elastic/kibana-on-merge/builds/49584), running the pre-PR version: - You can see, after the copying of es snapshot cache, the command to build kibana's plugins is ran, taking 6+ minutes Screenshot 2024-09-02 at 16 25 43 ## After This is on the [PR build](https://buildkite.com/elastic/kibana-pull-request/builds/231455#0191b2cc-f8be-410d-860d-59ab9f984477): - You can see, the es snapshot cache update is only 6 sec, after that, we can get going with the testing Screenshot 2024-09-02 at 16 28 10 ## Results On a primitive comparison between the PR build (https://buildkite.com/elastic/kibana-pull-request/builds/231505) and an on-merge run (https://buildkite.com/elastic/kibana-on-merge/builds/49598) we've **saved on average 9 minutes per runs** of `Osquery Cypress Tests`. We can only say more adequate savings once we compare on-merge with on-merge, but it can be estimated this would be **16 x 9 minutes = 144 minutes** (16 = 8 osquery cypress + 8 osquery serverless cypress) per on-merges. --- .buildkite/scripts/steps/functional/osquery_cypress.sh | 6 +----- .buildkite/scripts/steps/functional/osquery_cypress_burn.sh | 5 ++--- .../scripts/steps/functional/security_serverless_osquery.sh | 6 +----- 3 files changed, 4 insertions(+), 13 deletions(-) diff --git a/.buildkite/scripts/steps/functional/osquery_cypress.sh b/.buildkite/scripts/steps/functional/osquery_cypress.sh index 9ba7cabc4d1cb5..0ae47edd8c8e7f 100755 --- a/.buildkite/scripts/steps/functional/osquery_cypress.sh +++ b/.buildkite/scripts/steps/functional/osquery_cypress.sh @@ -4,11 +4,7 @@ set -euo pipefail source .buildkite/scripts/steps/functional/common.sh -# TODO: remove the line below to use build artifacts for tests. -# in addition to remove the line, we will have to expose the kibana install dir into the downloaded build location -# by exporting a var like: -# export KIBANA_INSTALL_DIR=${KIBANA_BUILD_LOCATION} -node scripts/build_kibana_platform_plugins.js +export KIBANA_INSTALL_DIR=${KIBANA_BUILD_LOCATION} export JOB=kibana-osquery-cypress diff --git a/.buildkite/scripts/steps/functional/osquery_cypress_burn.sh b/.buildkite/scripts/steps/functional/osquery_cypress_burn.sh index 8068dccac64f32..bebd391d952546 100755 --- a/.buildkite/scripts/steps/functional/osquery_cypress_burn.sh +++ b/.buildkite/scripts/steps/functional/osquery_cypress_burn.sh @@ -2,10 +2,9 @@ set -euo pipefail -source .buildkite/scripts/common/util.sh +source .buildkite/scripts/steps/functional/common.sh -.buildkite/scripts/bootstrap.sh -node scripts/build_kibana_platform_plugins.js +export KIBANA_INSTALL_DIR=${KIBANA_BUILD_LOCATION} export JOB=kibana-osquery-cypress diff --git a/.buildkite/scripts/steps/functional/security_serverless_osquery.sh b/.buildkite/scripts/steps/functional/security_serverless_osquery.sh index 27ed0c79871ee3..9844222e7f8698 100755 --- a/.buildkite/scripts/steps/functional/security_serverless_osquery.sh +++ b/.buildkite/scripts/steps/functional/security_serverless_osquery.sh @@ -4,11 +4,7 @@ set -euo pipefail source .buildkite/scripts/steps/functional/common.sh -# TODO: remove the line below to use build artifacts for tests. -# in addition to remove the line, we will have to expose the kibana install dir into the downloaded build location -# by exporting a var like: -# export KIBANA_INSTALL_DIR=${KIBANA_BUILD_LOCATION} -node scripts/build_kibana_platform_plugins.js +export KIBANA_INSTALL_DIR=${KIBANA_BUILD_LOCATION} export JOB=kibana-osquery-cypress-serverless