Avoid rebuilding operator and checking ES twice #187

rsevilla87 · 2020-05-28T15:57:29Z

Would fix #186 along with avoid rebuilding and pushing ripsaw in each test.

Also simplifies the function wait_clean since deleting a namespace waits for the deletion of all objects within it by default.

Rather than doing all those "sed" invocations, we might use the image argument of the benchmark YAMLs to overwrite the image. However I'm not sure about that idea since we would have to reference each test benchmark CRD.
Thoughts?

rsevilla87 · 2020-05-29T10:06:23Z

/rerun all

rht-perf-ci · 2020-05-29T12:16:54Z

Results for SNAFU CI Test

Test	Result	Runtime
fio_wrapper	PASS	00:10:25
fs_drift_wrapper	PASS	00:05:44
hammerdb	PASS	00:06:05
iperf	PASS	00:02:53
pgbench_wrapper	PASS	00:03:18
smallfile_wrapper	PASS	00:05:40
sysbench	PASS	00:01:51
uperf_wrapper	PASS	00:16:34
ycsb_wrapper	PASS	00:22:46

rsevilla87 · 2020-06-01T09:49:38Z

/rerun all

rht-perf-ci · 2020-06-01T10:58:37Z

Results for SNAFU CI Test

Test	Result	Runtime
fio_wrapper	PASS	00:11:07
fs_drift_wrapper	PASS	00:05:00
hammerdb	FAIL	00:03:05
iperf	PASS	00:02:55
pgbench_wrapper	PASS	00:03:10
smallfile_wrapper	PASS	00:05:47
sysbench	PASS	00:03:02
uperf_wrapper	PASS	00:16:33
ycsb_wrapper	PASS	00:07:18

rsevilla87 · 2020-06-01T11:56:00Z

/rerun all

rht-perf-ci · 2020-06-01T13:00:18Z

Results for SNAFU CI Test

Test	Result	Runtime
fio_wrapper	PASS	00:09:28
fs_drift_wrapper	PASS	00:05:33
hammerdb	PASS	00:05:48
iperf	PASS	00:02:28
pgbench_wrapper	PASS	00:02:43
smallfile_wrapper	PASS	00:05:03
sysbench	PASS	00:02:12
uperf_wrapper	PASS	00:16:45
ycsb_wrapper	FAIL	00:09:48

rsevilla87 · 2020-06-01T22:28:17Z

/rerun all

rht-perf-ci · 2020-06-01T22:55:21Z

Results for SNAFU CI Test

Test	Result	Runtime
fio_wrapper	PASS	00:09:22
fs_drift_wrapper	PASS	00:05:01
hammerdb	FAIL	00:02:48
iperf	PASS	00:02:37

rsevilla87 · 2020-06-02T10:15:31Z

/rerun all

rht-perf-ci · 2020-06-02T11:42:10Z

Results for SNAFU CI Test

Test	Result	Runtime
fio_wrapper	PASS	00:09:33
fs_drift_wrapper	PASS	00:04:46
hammerdb	PASS	00:06:19
iperf	PASS	00:02:50
pgbench_wrapper	PASS	00:02:49
smallfile_wrapper	PASS	00:05:29
sysbench	PASS	00:01:41
uperf_wrapper	PASS	00:16:37
ycsb_wrapper	PASS	00:07:49

jtaleric

IMHO We should remove the es_check in Ripsaw. It should be only checked via SNAFU (removing the double check).

rsevilla87 · 2020-06-03T09:00:39Z

IMHO We should remove the es_check in Ripsaw. It should be only checked via SNAFU (removing the double check).

Yeah, I get your point. However snafu's CI depends on ripsaw's CI and not the opposite. Checking ES in ripsaw's CI is the only way now to ensure changes in snafu and ripsaw don't break indexing.

jtaleric · 2020-06-03T11:14:41Z

IMHO We should remove the es_check in Ripsaw. It should be only checked via SNAFU (removing the double check).

Yeah, I get your point. However snafu's CI depends on ripsaw's CI and not the opposite. Checking ES in ripsaw's CI is the only way now to ensure changes in snafu and ripsaw don't break indexing.

If there is a circular dependency, I wonder if we can set a var to skip indexing? by default the indexing is enabled, but if running w/ Ripsaw CI, we disable SNAFU es_check?

rht-perf-ci · 2020-06-05T10:03:38Z

Results for SNAFU CI Test

Test	Result	Runtime
fio_wrapper	PASS	00:09:50
fs_drift_wrapper	PASS	00:05:16
hammerdb	PASS	00:05:34
iperf	PASS	00:02:37
pgbench_wrapper	PASS	00:02:42
smallfile_wrapper	PASS	00:05:50
sysbench	PASS	00:01:57
uperf_wrapper	PASS	00:16:58
ycsb_wrapper	PASS	00:06:55

dry923 · 2020-06-05T17:33:24Z

IMHO We should remove the es_check in Ripsaw. It should be only checked via SNAFU (removing the double check).

Yeah, I get your point. However snafu's CI depends on ripsaw's CI and not the opposite. Checking ES in ripsaw's CI is the only way now to ensure changes in snafu and ripsaw don't break indexing.

If there is a circular dependency, I wonder if we can set a var to skip indexing? by default the indexing is enabled, but if running w/ Ripsaw CI, we disable SNAFU es_check?

I think a variable for this is redundant. We are not removing the check_es script from SNAFU, we are just not calling it for the functions that are using ripsaw. If we add a new workload, foo, that is outside of ripsaw we can simply call the check_es script from foo's ci_test.sh file. Thoughts?

comet-perf-ci · 2020-07-16T14:22:00Z

Results for SNAFU CI Test

Test	Result	Runtime

comet-perf-ci · 2020-07-16T14:54:37Z

!!! Couldn't read commit file !!!

rsevilla87 · 2020-07-17T20:59:06Z

/rerun all

comet-perf-ci · 2020-07-17T22:34:56Z

Results for SNAFU CI Test

Test	Result	Runtime
fio_wrapper	PASS	00:12:01
fs_drift_wrapper	PASS	00:06:43
hammerdb	PASS	00:06:05
iperf	PASS	00:04:40
pgbench_wrapper	PASS	00:03:33
smallfile_wrapper	PASS	00:07:13
sysbench	PASS	00:02:29
uperf_wrapper	PASS	00:22:09
vegeta_wrapper	PASS	00:07:28
ycsb_wrapper	PASS	00:06:25

rsevilla87 · 2020-07-20T09:28:12Z

@jtaleric @dry923

IMHO We should remove the es_check in Ripsaw. It should be only checked via SNAFU (removing the double check).

Yeah, I get your point. However snafu's CI depends on ripsaw's CI and not the opposite. Checking ES in ripsaw's CI is the only way now to ensure changes in snafu and ripsaw don't break indexing.

If there is a circular dependency, I wonder if we can set a var to skip indexing? by default the indexing is enabled, but if running w/ Ripsaw CI, we disable SNAFU es_check?

I think a variable for this is redundant. We are not removing the check_es script from SNAFU, we are just not calling it for the functions that are using ripsaw. If we add a new workload, foo, that is outside of ripsaw we can simply call the check_es script from foo's ci_test.sh file. Thoughts?

+1 This PR simplifies the overall CI besides of preventing double-checking of ES indices.

dry923 · 2020-07-20T12:23:34Z

@rsevilla87 CI from the last run is showing an error when building the benchmark-operator:snafu_ci

00:00:02.446 + image_spec=quay.io/rht_perf_ci/benchmark-operator:snafu_ci
00:00:02.446 + operator-sdk build quay.io/rht_perf_ci/benchmark-operator:snafu_ci --image-builder podman
00:00:02.511 time="2020-07-17T21:15:49Z" level=info msg="Building OCI image quay.io/rht_perf_ci/benchmark-operator:snafu_ci"
00:00:02.704 STEP 1: FROM quay.io/operator-framework/ansible-operator:v0.17.1
00:00:02.771 STEP 2: USER root
00:00:02.775 --> Using cache 38af728f984f1717973bd2c0bb81eea46d753c57947ed6d371ee4c2edec81132
00:00:02.796 STEP 3: COPY requirements.yml ${HOME}/requirements.yml
00:00:04.750 --> d570bdbb4787bdd21961591e4194cdbbe08405c3142e6877b94b8c8a6e082169
00:00:04.770 STEP 4: RUN ansible-galaxy collection install -r ${HOME}/requirements.yml  && chmod -R ug+rwx ${HOME}/.ansible
00:00:06.348 Process install dependency map
00:00:06.348 |�/�-�\�|�/�-�\�|�/�-�\�|�/�-�\�|�/�-�\�|�/�-�\�|�/�-�\�|�/�-�\�|�/�-�\�|�/�-�\�|�/�-�\�|�/�-�\�|�/�-�\�|�/�-�\�|�/�-�\�|�/�-�\�|�/�-�\�|�/�-�\�|�/�-�\�|�/�-�\�|�/�-�\�|�/�-�\�|�/�-�\�|�/�-�ERROR! Unknown error when attempting to call Galaxy at 'https://galaxy.ansible.com/api/': <urlopen error [Errno -2] Name or service not known>
00:00:16.105 Error: error building at STEP "RUN ansible-galaxy collection install -r ${HOME}/requirements.yml  && chmod -R ug+rwx ${HOME}/.ansible": error while running runtime: exit status 1
00:00:16.111 Error: failed to output build image quay.io/rht_perf_ci/benchmark-operator:snafu_ci: (failed to exec []string{"podman", "build", "-f", "build/Dockerfile", "-t", "quay.io/rht_perf_ci/benchmark-operator:snafu_ci", "."}: exit status 125)
00:00:16.111 Usage:
00:00:16.111   operator-sdk build <image> [flags]
00:00:16.111 
00:00:16.111 Flags:
00:00:16.111       --go-build-args string      Extra Go build arguments as one string such as "-ldflags -X=main.xyz=abc"
00:00:16.111   -h, --help                      help for build
00:00:16.111       --image-build-args string   Extra image build arguments as one string such as "--build-arg https_proxy=$https_proxy"
00:00:16.111       --image-builder string      Tool to build OCI images. One of: [docker, podman, buildah] (default "docker")
00:00:16.111 
00:00:16.111 Global Flags:
00:00:16.111       --verbose   Enable verbose logging
00:00:16.111

rsevilla87 · 2020-07-20T12:26:36Z

/rerun all

comet-perf-ci · 2020-07-20T13:58:45Z

Results for SNAFU CI Test

Test	Result	Runtime
fio_wrapper	PASS	00:11:35
fs_drift_wrapper	PASS	00:06:53
hammerdb	PASS	00:06:13
iperf	PASS	00:04:58
pgbench_wrapper	PASS	00:05:31
smallfile_wrapper	PASS	00:08:28
sysbench	PASS	00:03:06
uperf_wrapper	PASS	00:25:28
vegeta_wrapper	PASS	00:10:14
ycsb_wrapper	PASS	00:07:57

dry923

LGTM!

rsevilla87 · 2020-07-21T13:17:27Z

/rerun all

comet-perf-ci · 2020-07-21T15:00:21Z

Results for SNAFU CI Test

Test	Result	Runtime
src/fio_wrapper	PASS	00:13:45
src/fs_drift_wrapper	PASS	00:07:56
src/hammerdb	PASS	00:07:08
src/iperf	PASS	00:04:02
src/pgbench_wrapper	PASS	00:04:02
src/smallfile_wrapper	PASS	00:08:31
src/sysbench	PASS	00:02:27
src/uperf_wrapper	PASS	00:26:25
src/vegeta_wrapper	FAIL	00:11:20
src/ycsb_wrapper	PASS	00:14:21

comet-perf-ci · 2020-07-21T16:50:35Z

Results for SNAFU CI Test

Test	Result	Runtime
src/fio_wrapper	FAIL	00:08:38
src/fs_drift_wrapper	PASS	00:08:12
src/hammerdb	PASS	00:06:20
src/iperf	PASS	00:05:25
src/pgbench_wrapper	PASS	00:04:45
src/smallfile_wrapper	PASS	00:08:57
src/sysbench	PASS	00:02:25
src/uperf_wrapper	PASS	00:24:39
src/vegeta_wrapper	FAIL	00:10:55
src/ycsb_wrapper	PASS	00:08:15

rsevilla87 · 2020-07-22T07:24:20Z

/rerun all

comet-perf-ci · 2020-07-22T08:50:36Z

Results for SNAFU CI Test

Test	Result	Runtime
src/fio_wrapper	PASS	00:11:42
src/fs_drift_wrapper	PASS	00:07:21
src/hammerdb	PASS	00:06:26
src/iperf	PASS	00:03:57
src/pgbench_wrapper	PASS	00:03:47
src/smallfile_wrapper	PASS	00:07:35
src/sysbench	PASS	00:02:23
src/uperf_wrapper	PASS	00:21:57
src/vegeta_wrapper	FAIL	00:10:46
src/ycsb_wrapper	PASS	00:06:39

rsevilla87 · 2020-07-22T15:40:49Z

/rerun all

comet-perf-ci · 2020-07-22T17:10:03Z

Results for SNAFU CI Test

Test	Result	Runtime
src/fio_wrapper	PASS	00:12:29
src/fs_drift_wrapper	PASS	00:06:49
src/hammerdb	PASS	00:06:40
src/iperf	PASS	00:03:50
src/pgbench_wrapper	PASS	00:04:24
src/smallfile_wrapper	PASS	00:07:40
src/sysbench	PASS	00:02:22
src/uperf_wrapper	PASS	00:24:18
src/vegeta_wrapper	FAIL	00:10:56
src/ycsb_wrapper	PASS	00:07:32

rsevilla87 · 2020-07-22T17:14:49Z

/rerun all

comet-perf-ci · 2020-07-22T18:39:35Z

Results for SNAFU CI Test

Test	Result	Runtime
src/fio_wrapper	PASS	00:12:37
src/fs_drift_wrapper	PASS	00:06:48
src/hammerdb	PASS	00:06:52
src/iperf	PASS	00:03:43
src/pgbench_wrapper	PASS	00:04:22
src/smallfile_wrapper	PASS	00:07:52
src/sysbench	PASS	00:02:26
src/uperf_wrapper	PASS	00:23:49
src/vegeta_wrapper	PASS	00:07:27
src/ycsb_wrapper	PASS	00:07:30

ci/common.sh

Signed-off-by: Raul Sevilla <rsevilla@redhat.com>

jtaleric

LGTM

comet-perf-ci · 2020-08-04T16:29:49Z

Results for SNAFU CI Test

Test	Result	Runtime
src/fio_wrapper	PASS	00:11:59
src/fs_drift_wrapper	PASS	00:07:03
src/hammerdb	PASS	00:06:14
src/iperf	PASS	00:04:06
src/pgbench_wrapper	PASS	00:04:06
src/smallfile_wrapper	PASS	00:07:22
src/sysbench	PASS	00:02:23
src/uperf_wrapper	PASS	00:22:04
src/vegeta_wrapper	PASS	00:07:00
src/ycsb_wrapper	PASS	00:07:20

rsevilla87 added the CI CI related label May 28, 2020

rsevilla87 force-pushed the ripsaw-rebuild branch from 5292aff to 4536df6 Compare May 29, 2020 10:05

rsevilla87 added the ok to test Kick off our CI framework label May 29, 2020

rsevilla87 marked this pull request as ready for review May 29, 2020 10:05

rsevilla87 requested review from dry923 and aakarshg May 29, 2020 10:20

rsevilla87 force-pushed the ripsaw-rebuild branch 2 times, most recently from 6ac5861 to a24e408 Compare May 29, 2020 10:55

rsevilla87 force-pushed the ripsaw-rebuild branch from a24e408 to 3c51d93 Compare June 1, 2020 09:54

jtaleric requested changes Jun 2, 2020

View reviewed changes

rsevilla87 force-pushed the ripsaw-rebuild branch from 3c51d93 to 5ce3135 Compare June 5, 2020 08:28

rsevilla87 force-pushed the ripsaw-rebuild branch from 5ce3135 to e05ef5e Compare July 16, 2020 14:42

rsevilla87 requested review from jtaleric and removed request for aakarshg July 16, 2020 14:42

dry923 approved these changes Jul 20, 2020

View reviewed changes

rsevilla87 force-pushed the ripsaw-rebuild branch 2 times, most recently from 7317715 to b40dabc Compare July 21, 2020 10:46

rsevilla87 force-pushed the ripsaw-rebuild branch from b40dabc to 09903f4 Compare July 21, 2020 15:18

cloud-bulldozer deleted a comment from comet-perf-ci Jul 22, 2020

jtaleric reviewed Aug 4, 2020

View reviewed changes

ci/common.sh Outdated Show resolved Hide resolved

Avoid rebuilding operator and checking ES twice

dd8d9dc

Signed-off-by: Raul Sevilla <rsevilla@redhat.com>

rsevilla87 force-pushed the ripsaw-rebuild branch from 09903f4 to dd8d9dc Compare August 4, 2020 15:08

jtaleric approved these changes Aug 4, 2020

View reviewed changes

jtaleric merged commit e0927ee into cloud-bulldozer:master Aug 4, 2020

rsevilla87 deleted the ripsaw-rebuild branch August 4, 2020 17:56

Avoid rebuilding operator and checking ES twice #187

Avoid rebuilding operator and checking ES twice #187

Conversation

rsevilla87 commented May 28, 2020 • edited Loading

rsevilla87 commented May 29, 2020

rht-perf-ci commented May 29, 2020

rsevilla87 commented Jun 1, 2020

rht-perf-ci commented Jun 1, 2020

rsevilla87 commented Jun 1, 2020

rht-perf-ci commented Jun 1, 2020

rsevilla87 commented Jun 1, 2020

rht-perf-ci commented Jun 1, 2020

rsevilla87 commented Jun 2, 2020

rht-perf-ci commented Jun 2, 2020

jtaleric left a comment

Choose a reason for hiding this comment

rsevilla87 commented Jun 3, 2020

jtaleric commented Jun 3, 2020

rht-perf-ci commented Jun 5, 2020

dry923 commented Jun 5, 2020

comet-perf-ci commented Jul 16, 2020

comet-perf-ci commented Jul 16, 2020

rsevilla87 commented Jul 17, 2020

comet-perf-ci commented Jul 17, 2020

rsevilla87 commented Jul 20, 2020

dry923 commented Jul 20, 2020

rsevilla87 commented Jul 20, 2020

comet-perf-ci commented Jul 20, 2020

dry923 left a comment

Choose a reason for hiding this comment

rsevilla87 commented Jul 21, 2020

comet-perf-ci commented Jul 21, 2020

comet-perf-ci commented Jul 21, 2020

rsevilla87 commented Jul 22, 2020

comet-perf-ci commented Jul 22, 2020

rsevilla87 commented Jul 22, 2020

comet-perf-ci commented Jul 22, 2020

rsevilla87 commented Jul 22, 2020

comet-perf-ci commented Jul 22, 2020

jtaleric left a comment

Choose a reason for hiding this comment

comet-perf-ci commented Aug 4, 2020

rsevilla87 commented May 28, 2020 •

edited

Loading