Enhancement for HammerDB doc generation/indexing #120

mkarg75 · 2020-01-15T13:57:00Z

hopefully the last one

dry923 · 2020-01-15T14:02:34Z

/rerun all

rht-perf-ci · 2020-01-15T14:29:09Z

Results for SNAFU CI Test

Test	Result	Runtime

dry923 · 2020-01-15T14:29:34Z

/rerun all

dry923 · 2020-01-15T15:29:04Z

/rerun all

rht-perf-ci · 2020-01-15T15:56:25Z

Results for SNAFU CI Test

Test	Result	Runtime
fio_wrapper	PASS	00:06:48
fs_drift_wrapper	PASS	00:07:30
hammerdb	FAIL	00:04:38

dry923 · 2020-01-15T15:56:31Z

/rerun all

bengland2 · 2020-01-15T16:03:30Z

@mkarg75 , I'm concerned that the results you get from minishift might not match the results other people get from OCP4 or from minikube. Does that make sense? If so, any chance you can try this with version of minikube recommended by Russ or whoever in outcome of ripsaw issue 272 ?

bengland2 · 2020-01-15T16:10:14Z

also would like to see this patch in snafu/hammerdb/hammerd_wrapper.py:

@@ -170,6 +170,8 @@ def main():
     if es_server != "" :
         if len(documents) > 0 :
             _index_result("ripsaw-hammerdb-results", es_server, es_port, documents)
+        else:
+            raise Exception('failed to produce hammerdb results document')

or something equivalent to it, where benchmark doesn't just silently fail.

…e ES document generation fails

rht-perf-ci · 2020-01-15T16:57:54Z

Results for SNAFU CI Test

Test	Result	Runtime
fio_wrapper	PASS	00:06:54
fs_drift_wrapper	PASS	00:07:39
hammerdb	FAIL	00:04:22
iperf	PASS	00:06:04
pgbench-wrapper	PASS	00:07:44

dry923 · 2020-01-15T16:58:04Z

/rerun all

rht-perf-ci · 2020-01-15T18:02:46Z

Results for SNAFU CI Test

Test	Result	Runtime
fio_wrapper	PASS	00:06:54
fs_drift_wrapper	PASS	00:07:42
hammerdb	FAIL	00:04:28
iperf	PASS	00:06:16
pgbench-wrapper	PASS	00:07:27
smallfile_wrapper	PASS	00:06:47
sysbench	PASS	00:05:48
uperf-wrapper	PASS	00:07:29
ycsb-wrapper	PASS	00:05:55

aakarshg · 2020-01-16T10:11:56Z

/rerun all

rht-perf-ci · 2020-01-16T10:14:23Z

Results for SNAFU CI Test

Test	Result	Runtime

aakarshg · 2020-01-16T11:01:15Z

hammerdb/Dockerfile

-COPY . /opt/snafu
+# clone the snafu repo 
+#RUN cd /opt/; git clone https://github.com/mkarg75/snafu.git
+RUN cd /opt/; git clone https://github.com/cloud-bulldozer/snafu.git


This change is not required, as snafu is in the build context of the image.

rht-perf-ci · 2020-01-16T11:40:46Z

Results for SNAFU CI Test

Test	Result	Runtime

dry923 · 2020-01-16T11:46:34Z

/rerun all

rht-perf-ci · 2020-01-16T11:47:25Z

Results for SNAFU CI Test

Test	Result	Runtime

aakarshg · 2020-01-16T11:57:16Z

/rerun all

rht-perf-ci · 2020-01-16T12:12:13Z

Results for SNAFU CI Test

Test	Result	Runtime
hammerd	FAIL	00:12:39

rht-perf-ci · 2020-01-16T16:27:55Z

Results for SNAFU CI Test

Test	Result	Runtime
hammerd	FAIL	00:01:28

rht-perf-ci · 2020-01-16T16:42:16Z

Results for SNAFU CI Test

Test	Result	Runtime
hammerd	FAIL	00:09:06

aakarshg · 2020-01-16T17:14:07Z

/rerun all

rht-perf-ci · 2020-01-16T17:26:36Z

Results for SNAFU CI Test

Test	Result	Runtime
hammerd	FAIL	00:08:33

dry923 · 2020-01-16T18:19:57Z

@mkarg75 looks like hammer was successful however it failed ci because it couldn't find any results in ES.

++ kubectl logs hammerdb-workload-723e8b18-j89x8 --namespace my-ripsaw
++ grep 'SEQUENCE COMPLETE'
++ echo 'Hammerdb test: Success'
Hammerdb test: Success
...
+ python3 ci/check_es.py -s marquez.perf.lab.eng.rdu2.redhat.com -p 9200 -u 723e8b18-644f-5352-aafe-c18c679ca138 -i ripsaw-hammerdb-results
No result found in ES

rht-perf-ci · 2020-01-17T10:23:46Z

Results for SNAFU CI Test

Test	Result	Runtime
hammerd	FAIL	00:08:50

dry923 · 2020-01-17T11:55:18Z

@mkarg75 why the revert from the ubi8 Dockerfile?

rht-perf-ci · 2020-01-17T13:19:36Z

Results for SNAFU CI Test

Test	Result	Runtime
hammerd	FAIL	00:07:34

dry923 · 2020-01-20T19:23:38Z

/rerun all

rht-perf-ci · 2020-01-20T19:36:24Z

Results for SNAFU CI Test

Test	Result	Runtime
hammerd	FAIL	00:01:52

rht-perf-ci · 2020-01-28T08:03:41Z

Results for SNAFU CI Test

Test	Result	Runtime
hammerd	PASS	00:15:19

rht-perf-ci · 2020-01-28T13:40:11Z

Results for SNAFU CI Test

Test	Result	Runtime
hammerd	PASS	00:15:23

aakarshg

There are a bunch of tcl files that are also added but I don't see them being used in the dockerfile (and no change to ripsaw was merged yet) ? So are they really required @mkarg75 ? If they're not can only the files/changes required be in this PR?

aakarshg · 2020-01-28T14:35:35Z

hammerdb/build_container.sh

@@ -0,0 +1,7 @@
+#!/bin/bash


can you please remove this file?

aakarshg · 2020-01-28T14:36:44Z

hammerdb/entrypoint

@@ -0,0 +1,12 @@
+#!/usr/bin/env bash


Wouldn't this file make uid_entrypoint file redundant ? in that case can we remove one from the other ?

I don't see this added to the dockerfile as well?

hammerdb/hammerd_wrapper.py

rht-perf-ci · 2020-01-28T16:32:51Z

Results for SNAFU CI Test

Test	Result	Runtime
hammerd	PASS	00:12:44

aakarshg

LGTM.

dry923

LGTM

Marko Karg added 2 commits January 15, 2020 14:56

stuff added. and things

1ded583

cloud-bulldozer now

9422641

dry923 added the ok to test Kick off our CI framework label Jan 15, 2020

dry923 self-requested a review January 15, 2020 14:02

added paths to the COPY directives

3c4dd5c

added the patch Ben proposed to prevent from silently failing when th…

b31d518

…e ES document generation fails

aakarshg reviewed Jan 16, 2020

View reviewed changes

reverting to the old way snafu gets copied in

abe282c

Marko Karg added 2 commits January 17, 2020 08:25

cosmetics for the Dockerfile

06cb2d0

added missing env parameters

ebe12d7

merging

0f6c20e

fixed a loop variable

147c1e3

changes to make elasticsearch entries work. hopefully.

a8da271

Merge branch 'master' into master

34d9013

aakarshg suggested changes Jan 28, 2020

View reviewed changes

Marko Karg added 3 commits January 28, 2020 15:45

removed superfluous files

b963d87

Merge branch 'master' of github.com:mkarg75/snafu

c6b0033

another one bites the dust

555cdd1

aakarshg reviewed Jan 28, 2020

View reviewed changes

hammerdb/hammerd_wrapper.py Show resolved Hide resolved

aakarshg requested a review from jtaleric January 28, 2020 16:41

aakarshg changed the title ~~stuff added. and things~~ Enhancement for HammerDB doc generation/indexing Jan 28, 2020

aakarshg approved these changes Jan 28, 2020

View reviewed changes

dry923 approved these changes Jan 28, 2020

View reviewed changes

aakarshg merged commit ea6f81b into cloud-bulldozer:master Jan 28, 2020

Enhancement for HammerDB doc generation/indexing #120

Enhancement for HammerDB doc generation/indexing #120

Conversation

mkarg75 commented Jan 15, 2020

dry923 commented Jan 15, 2020

rht-perf-ci commented Jan 15, 2020

dry923 commented Jan 15, 2020

dry923 commented Jan 15, 2020

rht-perf-ci commented Jan 15, 2020

dry923 commented Jan 15, 2020

bengland2 commented Jan 15, 2020

bengland2 commented Jan 15, 2020

rht-perf-ci commented Jan 15, 2020

dry923 commented Jan 15, 2020

rht-perf-ci commented Jan 15, 2020

aakarshg commented Jan 16, 2020

rht-perf-ci commented Jan 16, 2020

aakarshg Jan 16, 2020

Choose a reason for hiding this comment

rht-perf-ci commented Jan 16, 2020

dry923 commented Jan 16, 2020

rht-perf-ci commented Jan 16, 2020

aakarshg commented Jan 16, 2020

rht-perf-ci commented Jan 16, 2020

rht-perf-ci commented Jan 16, 2020

rht-perf-ci commented Jan 16, 2020

aakarshg commented Jan 16, 2020

rht-perf-ci commented Jan 16, 2020

dry923 commented Jan 16, 2020

rht-perf-ci commented Jan 17, 2020

dry923 commented Jan 17, 2020

rht-perf-ci commented Jan 17, 2020

dry923 commented Jan 20, 2020

rht-perf-ci commented Jan 20, 2020

rht-perf-ci commented Jan 28, 2020

rht-perf-ci commented Jan 28, 2020

aakarshg left a comment

Choose a reason for hiding this comment

aakarshg Jan 28, 2020

Choose a reason for hiding this comment

aakarshg Jan 28, 2020

Choose a reason for hiding this comment

aakarshg Jan 28, 2020

Choose a reason for hiding this comment

rht-perf-ci commented Jan 28, 2020

aakarshg left a comment

Choose a reason for hiding this comment

dry923 left a comment

Choose a reason for hiding this comment