Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump es version #302

Merged
merged 2 commits into from
Jul 27, 2021
Merged

Conversation

learnitall
Copy link
Collaborator

Description

This PR bumps elasticsearch client version within our setup.cfg to use elasticsearch>=7.0.0,<8.0.0, (as per documentation spec at https://elasticsearch-py.readthedocs.io/en/v7.14.0a1/), in order to enable Python 3.9 support. Joe and I discussed that we would need to test this change against our ES instances to make sure bumping the client version doesn't break our data ingest.

Fixes

Currently we have elasticsearch client pinned as elasticsearch>=6.0.0,<=7.0.2, however these versions are not compatible with Python 3.9. The following ImportError is observed when trying to invoke snafu on Python 3.9:

snafu/run_snafu.py:20: in <module>
    import elasticsearch
.tox/py39/lib/python3.9/site-packages/elasticsearch/__init__.py:24: in <module>
    from .client import Elasticsearch
.tox/py39/lib/python3.9/site-packages/elasticsearch/client/__init__.py:4: in <module>
    from ..transport import Transport
.tox/py39/lib/python3.9/site-packages/elasticsearch/transport.py:4: in <module>
    from .connection import Urllib3HttpConnection
.tox/py39/lib/python3.9/site-packages/elasticsearch/connection/__init__.py:2: in <module>
    from .http_requests import RequestsHttpConnection
.tox/py39/lib/python3.9/site-packages/elasticsearch/connection/http_requests.py:3: in <module>
    from base64 import decodestring
E   ImportError: cannot import name 'decodestring' from 'base64' (/opt/hostedtoolcache/Python/3.9.6/x64/lib/python3.9/base64.py)

Additionally, I removed the elasticsearch1 module import, as we only have ES >= 7 in our environment, which is supported within the given elasticsearch module itself.

@learnitall learnitall added bug Something isn't working ok to test Kick off our CI framework labels Jul 19, 2021
learnitall added a commit to learnitall/benchmark-wrapper that referenced this pull request Jul 19, 2021
Not adding 3.9 for now, please see cloud-bulldozer#302
@comet-perf-ci
Copy link
Collaborator

Results for SNAFU CI Test

Test Result Runtime
snafu/hammerdb PASS 00:06:40
snafu/smallfile_wrapper PASS 00:06:11
snafu/flent_wrapper PASS 00:09:47
snafu/scale_openshift_wrapper PASS 00:10:43
snafu/sysbench PASS 00:01:27
snafu/benchmarks/uperf PASS 00:26:10
snafu/image_pull_wrapper PASS 00:03:59
snafu/ycsb_wrapper PASS 00:05:32
snafu/stressng_wrapper PASS 00:03:21
snafu/fio_wrapper PASS 00:19:00
snafu/log_generator_wrapper PASS 00:03:58
snafu/upgrade_openshift_wrapper PASS 00:00:00
snafu/iperf PASS 00:05:56
snafu/vegeta_wrapper PASS 00:05:28
snafu/fs_drift_wrapper PASS 00:05:55
snafu/pgbench_wrapper PASS 00:04:19

@learnitall learnitall changed the title Bug bump es version Bump es version Jul 20, 2021
@whitleykeith whitleykeith self-requested a review July 27, 2021 16:40
Copy link

@whitleykeith whitleykeith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Collaborator

@mohit-sheth mohit-sheth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mohit-sheth mohit-sheth merged commit 506d2ac into cloud-bulldozer:master Jul 27, 2021
@learnitall learnitall deleted the bug-bump-es-version branch September 25, 2021 00:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working ok to test Kick off our CI framework
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants