[Python] Migrate to the next version of Python requests when released

[potiuk]

Hello Maintainers,

I am a PMC member of Apache Airflow, and I wanted to give you a bit of heads-up with rather important migration to the upcoming version of requests library in your Python release.

Since you are using requests library in your project, you are affected. It's not as bad for you as you seem to use requests library optionally (when http extra is used) and LGPL dependencies can be used if they are optional, however it would be safer if you use non-LGPL dependency also here).

As discussed at length in https://issues.apache.org/jira/browse/LEGAL-572 we found out that the chardet ibrary used by requests library was a mandatory dependency to requests and since it has LGPL licence, we should not release any Apache Software with it.

Since then (and since in Airflow we rely on requests heavily) we have been working with the requests maintainers and "charset-normalizer" maintainer to make it possible to replace chardet with MIT-licensed charset-normalizer instead so that requests library can be used in Python releases by Apache projects.

This was a bumpy road but finally the PR by @ashb has been merged: psf/requests#5797 and we hope soon a new version of requests library will be released.

This is just a heads-up. I will let you know when it is released, but I have a kind requests as well - I might ask the maintainers to release a release candidate of requests and maybe you could help to test it before it is released, that would be some re-assurance for the maintainers of requests who are very concerned about stability of their releases.

Let me know if you need any more information and whether you would like to help in testing the candidate when it is out.

[wu-sheng]

@kezhenxu94 Do our python use this in the runtime? I don't read our codes, but I know we use grpc to transport data. Is this a dependency of dependency?

[kezhenxu94]

@kezhenxu94 Do our python use this in the runtime? I don't read our codes, but I know we use grpc to transport data. Is this a dependency of dependency?

requests should be optional and not installed by default, but it was made mandatory in apache/skywalking-python#125 so I request @tom-pytel to make it optional again https://github.com/apache/skywalking-python/pull/125/files#r667484219 , luckily we haven't release this yet. It was tagged in 0.7.0

[kezhenxu94]

Hi @potiuk thanks for the heads up, just let me know when it's out, I'm glad to help to test it.

[wu-sheng]

@potiuk How could you find this in a dev version? Do you have a special tool? :P Just curious.

[potiuk]

@potiuk How could you find this in a dev version? Do you have a special tool? :P Just curious.

I looked at setup.py in main branch for all apache projects I identified as potentially affected. And then installed some via github URL and checked dependencies they brought with pipdeptree.

[wu-sheng]

I reopen this, in case we need to do the upgrade. @kezhenxu94 Your call.

[potiuk]

Requests 2.26.0 released without the mandatory chardet dependency:

https://pypi.org/project/requests/2.26.0/

Instead of chardet, use the MIT-licensed charset_normalizer for Python3 to remove license ambiguity for projects bundling requests. If chardet is already installed on your machine it will be used instead of charset_normalizer to keep backwards compatibility. (#5797)

You can also install chardet while installing requests by specifying [use_chardet_on_py3] extra as follows:

shell pip install "requests[use_chardet_on_py3]"

Python2 still depends upon the chardet module.