Skip to content
This repository has been archived by the owner on Jul 31, 2024. It is now read-only.

Update Python to 3.7.6 and update dependencies #1020

Merged
merged 2 commits into from
Jan 6, 2020
Merged

Update Python to 3.7.6 and update dependencies #1020

merged 2 commits into from
Jan 6, 2020

Conversation

willkg
Copy link
Member

@willkg willkg commented Jan 3, 2020

kombu: 4.6.6 -> 4.6.7

https://github.com/celery/kombu/blob/master/Changelog.rst

boto3: 1.10.7 -> 1.10.46
botocore 1.13.7 -> 1.13.46

https://github.com/boto/boto3/blob/master/CHANGELOG.rst
https://github.com/boto/botocore/blob/develop/CHANGELOG.rst

  • lots of api changes for things we don't use
  • several bugfixes for s3
    • bugfix:s3: Add stricter validation to s3 control account id parameter.
    • bugfix:s3: Fixed an issue where the request path was set incorrectly
      if access point name was present in key path.

celery[redis]: 4.3.0 -> 4.4.0

http://docs.celeryproject.org/en/master/whatsnew-4.4.html#whatsnew-4-4

  • Bunch of version updates with dependencies.
  • Some new features, but nothing we use.

certifi: 2019.9.11 -> 2019.11.28

No changelog.

datadog: 0.31.0 -> 0.33.0

https://github.com/DataDog/datadogpy/blob/master/CHANGELOG.md#0330--2019-12-12

gunicorn: 19.9.0 -> 20.0.4

https://docs.gunicorn.org/en/stable/news.html

pytest: 5.2.2 -> 5.3.2

https://docs.pytest.org/en/latest/changelog.html

Lots of bug fixes, new features, and some deprecations.

Sphinx: 2.2.1 -> 2.3.1

https://www.sphinx-doc.org/en/master/changes.html

alembic: 1.3.0 -> 1.3.2

https://alembic.sqlalchemy.org/en/latest/changelog.html

  • [bug] [mssql] Fixed bug in MSSQL dialect where the drop constraint
    execution steps used to remove server default or implicit foreign key
    constraint failed to take into account the schema name of the target
    table. References: Scheduled daily dependency update on sunday #621
  • [bug] [api] [autogenerate] Fixed regression introduced by geosubmit not work (latest docker image) #579 where
    server default rendering functions began to require a dialect
    implementation, however the render_python_code() convenience function
    did not include one, thus causing the function to fail when used in a
    server default context. The function now accepts a migration context
    argument and also creates one against the default dialect if one is not
    provided. References: Scheduled daily dependency update on sunday #635

Cython: 0.29.13 -> 0.29.14

http://docs.cython.org/en/latest/src/changes.html

  • The generated code failed to initialise the tp_print slot in CPython
    3.8. Patches by Pablo Galindo and Orivej Desh (Github issues #3171,
    #3201).
  • ? for bool was missing from the supported NumPy dtypes. Patch by Max
    Klein. (Github issue #2675)
  • await was not allowed inside of f-strings. Patch by Dmitro Getz.
    (Github issue #2877)
  • Coverage analysis failed for projects where the code resides in
    separate source sub-directories. Patch by Antonio Valentino. (Github
    issue Bump node from 19.8.1-slim to 20.3.1-slim in /docker/node #1985)
  • An incorrect compiler warning was fixed in automatic C++ string
    conversions. Patch by Gerion Entrup. (Github issue #3108)
  • Error reports in the Jupyter notebook showed unhelpful stack traces.
    Patch by Matthew Edwards (Github issue #3196).
  • Python.h is now also included explicitly from public header files.
    (Github issue #3133).
  • Distutils builds with --parallel did not work when using Cython’s
    deprecated build_ext command. Patch by Alphadelta14 (Github issue
    #3187).
  • The PyMemoryView_*() C-API is available in cpython.memoryview. Patch
    by Nathan Manville. (Github issue #2541)

geoip2: 2.9.0 -> 3.0.0

https://github.com/maxmind/GeoIP2-python/blob/master/HISTORY.rst

  • BREAKING CHANGE: The geoip2.record.* classes have been refactored to
    improve performance. This refactoring may break classes that inherit
    from them. The public API should otherwise be compatible.
  • The network attribute was added to geoip2.record.Traits,
    geoip2.model.AnonymousIP, geoip2.model.ASN, geoip2.model.ConnectionType,
    geoip2.model.Domain, and geoip2.model.ISP. This is an
    ipaddress.IPv4Network or an ipaddress.IPv6Network. This is the largest
    network where all of the fields besides ip_address have the same value.
    GitHub Handle too long wifi submissions #79.
  • Python 3.3 and 3.4 are no longer supported.
  • Updated documentation of anonymizer attributes - is_anonymous_vpn and
    is_hosting_provider - to be more descriptive.
  • Added support for the user_count trait for the GeoIP2 Precision
    webservice.
  • Added the static_ip_score attribute to geoip2.record.Traits for GeoIP2
    Precision Insights. This is a float which indicates how static or
    dynamic an IP address is.

maxminddb: 1.5.1 -> 1.5.2

https://github.com/maxmind/MaxMind-DB-Reader-python/blob/master/HISTORY.rst

  • Minor performance improvements in the pure Python reader.

numpy: 1.17.3 -> 1.18.0

https://github.com/numpy/numpy/tree/master/doc/changelog

Rtree: 0.8.3 -> 0.9.3

https://rtree.readthedocs.io/en/latest/changes.html

scipy: 1.3.1 -> 1.4.1

https://docs.scipy.org/doc/scipy/reference/release.html

SQLAlchemy: 1.3.10 -> 1.3.12

https://docs.sqlalchemy.org/en/13/changelog/changelog_13.html

kombu: 4.6.6 -> 4.6.7

https://github.com/celery/kombu/blob/master/Changelog.rst

* Use importlib.metadata from the standard library on Python 3.8+ (#1086).
* Add peek lock settings to be changed using transport options (#1119).
* Fix redis health checks (#1122).
* Reset ready before execute callback (#1126).
* Add missing parameter queue_args in kombu.connection.SimpleBuffer (#1128)

boto3: 1.10.7 -> 1.10.46
botocore 1.13.7 -> 1.13.46

https://github.com/boto/boto3/blob/master/CHANGELOG.rst
https://github.com/boto/botocore/blob/develop/CHANGELOG.rst

* lots of api changes for things we don't use
* several bugfixes for s3
  * bugfix:s3: Add stricter validation to s3 control account id parameter.
  * bugfix:s3: Fixed an issue where the request path was set incorrectly
    if access point name was present in key path.

celery[redis]: 4.3.0 -> 4.4.0

http://docs.celeryproject.org/en/master/whatsnew-4.4.html#whatsnew-4-4

* Bunch of version updates with dependencies.
* Some new features, but nothing we use.

certifi: 2019.9.11 -> 2019.11.28

No changelog.

datadog: 0.31.0 -> 0.33.0

https://github.com/DataDog/datadogpy/blob/master/CHANGELOG.md#0330--2019-12-12

gunicorn: 19.9.0 -> 20.0.4

https://docs.gunicorn.org/en/stable/news.html

pytest: 5.2.2 -> 5.3.2

https://docs.pytest.org/en/latest/changelog.html

Lots of bug fixes, new features, and some deprecations.

Sphinx: 2.2.1 -> 2.3.1

https://www.sphinx-doc.org/en/master/changes.html

alembic: 1.3.0 -> 1.3.2

https://alembic.sqlalchemy.org/en/latest/changelog.html

* [bug] [mssql] Fixed bug in MSSQL dialect where the drop constraint
  execution steps used to remove server default or implicit foreign key
  constraint failed to take into account the schema name of the target
  table. References: #621
* [bug] [api] [autogenerate] Fixed regression introduced by #579 where
  server default rendering functions began to require a dialect
  implementation, however the render_python_code() convenience function
  did not include one, thus causing the function to fail when used in a
  server default context. The function now accepts a migration context
  argument and also creates one against the default dialect if one is not
  provided. References: #635

Cython: 0.29.13 ->  0.29.14

http://docs.cython.org/en/latest/src/changes.html

* The generated code failed to initialise the tp_print slot in CPython
  3.8. Patches by Pablo Galindo and Orivej Desh (Github issues #3171,
  #3201).
* ? for bool was missing from the supported NumPy dtypes. Patch by Max
  Klein. (Github issue #2675)
* await was not allowed inside of f-strings. Patch by Dmitro Getz.
  (Github issue #2877)
* Coverage analysis failed for projects where the code resides in
  separate source sub-directories. Patch by Antonio Valentino. (Github
  issue #1985)
* An incorrect compiler warning was fixed in automatic C++ string
  conversions. Patch by Gerion Entrup. (Github issue #3108)
* Error reports in the Jupyter notebook showed unhelpful stack traces.
  Patch by Matthew Edwards (Github issue #3196).
* Python.h is now also included explicitly from public header files.
  (Github issue #3133).
* Distutils builds with --parallel did not work when using Cython’s
  deprecated build_ext command. Patch by Alphadelta14 (Github issue
  #3187).
* The PyMemoryView_*() C-API is available in cpython.memoryview. Patch
  by Nathan Manville. (Github issue #2541)

geoip2: 2.9.0 -> 3.0.0

https://github.com/maxmind/GeoIP2-python/blob/master/HISTORY.rst

* BREAKING CHANGE: The geoip2.record.* classes have been refactored to
  improve performance. This refactoring may break classes that inherit
  from them. The public API should otherwise be compatible.
* The network attribute was added to geoip2.record.Traits,
  geoip2.model.AnonymousIP, geoip2.model.ASN, geoip2.model.ConnectionType,
  geoip2.model.Domain, and geoip2.model.ISP. This is an
  ipaddress.IPv4Network or an ipaddress.IPv6Network. This is the largest
  network where all of the fields besides ip_address have the same value.
  GitHub #79.
* Python 3.3 and 3.4 are no longer supported.
* Updated documentation of anonymizer attributes - is_anonymous_vpn and
  is_hosting_provider - to be more descriptive.
* Added support for the user_count trait for the GeoIP2 Precision
  webservice.
* Added the static_ip_score attribute to geoip2.record.Traits for GeoIP2
  Precision Insights. This is a float which indicates how static or
  dynamic an IP address is.

maxminddb: 1.5.1 -> 1.5.2

https://github.com/maxmind/MaxMind-DB-Reader-python/blob/master/HISTORY.rst

* Minor performance improvements in the pure Python reader.

numpy: 1.17.3 -> 1.18.0

https://github.com/numpy/numpy/tree/master/doc/changelog

Rtree: 0.8.3 -> 0.9.3

https://rtree.readthedocs.io/en/latest/changes.html

* Add Index.GetResultSetOffset()
* Add Index.contains() method for object and id (requires
  libspatialindex 1.9.3+) #116
* Add Index.Flush() #107
* Add TPRTree index support (thanks @sdhiscocks #117 )
* Return container sizes without returning objects #90
* Add set_result_limit and set_result_offset for Index paging
  44ad21aecd3f7b49314b9be12f3334d8bae7e827
* Better exceptions in cases where stream functions throw #80
* Migrated CI platform to Azure Pipelines
  https://dev.azure.com/hobuinc/rtree/_build?definitionId=5
* Minor test enhancements and fixups. Both libspatialindex 1.8.5 and
  libspatialindex 1.9.3 are tested with CI
* Refactored tests to be based on unittest
  Toblerity/rtree#129
* Update libspatialindex library loading code to adapt previous behavior
  Toblerity/rtree#128
* Empty data streams throw exceptions and do not partially construct
  indexes Toblerity/rtree#127
* find_library and libspatialindex library loading
  Toblerity/rtree#131

scipy: 1.3.1 -> 1.4.1

https://docs.scipy.org/doc/scipy/reference/release.html

SQLAlchemy: 1.3.10 -> 1.3.12

https://docs.sqlalchemy.org/en/13/changelog/changelog_13.html
This updates to Python 3.7.6 and also pins the Docker image.
@willkg willkg requested a review from jwhitlock January 3, 2020 01:37
@willkg
Copy link
Member Author

willkg commented Jan 3, 2020

@jwhitlock: I don't think it makes sense to go through all the changes and double-check them. But does this update any known-troublesome dependencies? Is updating to Python 3.7.6 problematic?

Copy link
Member

@jwhitlock jwhitlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @willkg, this looks good. I appreciate the links to the changelogs, and the relevant items when available.

geoip2==2.9.0 \
--hash=sha256:a37ddac2d200ffb97c736da8b8ba9d5d8dc47da6ec0f162a461b681ecac53a14 \
--hash=sha256:f7ffe9d258e71a42cf622ce6350d976de1d0312b9f2fbce3975c7d838b57ecf0
geoip2==3.0.0 \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Breaking change in geoip2.record.* classes, but we just use configure_geoip

--hash=sha256:dd3b52e00f93fd1c86f2d78243dfb0d02743c94dd1d34ffea10055438e63b99d
Rtree==0.9.3 \
--hash=sha256:55c046a98e8d84235792de1f18635f680d7b4085264fbced6b073f28fcbe353a
scipy==1.4.1 \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Today I learned about 2018 CODATA

@@ -1,4 +1,4 @@
FROM python:3.7.4-slim
FROM python:3.7.6-slim@sha256:6cfba78e00ab3faf60d5d5ae875d639396d6ecb46a259da9880b40d0defe036c
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think a minor version update is an issue.

@willkg willkg merged commit 83b44b1 into mozilla:master Jan 6, 2020
@willkg
Copy link
Member Author

willkg commented Jan 6, 2020

Thank you!

@willkg willkg deleted the update-deps branch January 6, 2020 21:22
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants