[docker-base-buster][docker-config-engine-buster] No longer install Python 2 #6162

jleveque · 2020-12-08T19:47:29Z

- Why I did it

As part of migrating SONiC codebase from Python 2 to Python 3

- How I did it

No longer install Python 2 in docker-base-buster or docker-config-engine-buster.
Install Python 2 and pip2 in the following containers until we can completely eliminate it there:
- docker-platform-monitor
- docker-sonic-mgmt-framework
- docker-sonic-vs
Pin pip2 version <21 where it is still temporarily needed, as pip version 21 will drop support for Python 2
Also preform some other cleanup, ensuring that pip3, setuptools and wheel packages are installed in docker-base-buster, and then removing any attempts to re-install them in derived containers

- How to verify it

Build and run an image

- Which release branch to backport (provide reason below if selected)

201811
201911
202006

lguohan · 2020-12-12T16:58:31Z

dockers/docker-base/Dockerfile.j2

@@ -52,6 +52,9 @@ COPY ["etc/rsyslog.conf", "/etc/rsyslog.conf"]
 COPY ["etc/rsyslog.d/*", "/etc/rsyslog.d/"]
 COPY ["root/.vimrc", "/root/.vimrc"]

+RUN pip install --upgrade 'pip<21'
+RUN apt-get purge -y python-pip


not sure why to touch it.

In January, pip version 21 will be released, which will drop support for Python 2. Therefore, we need to ensure we install a version < 21 for Python 2 purposes.

lguohan · 2020-12-12T16:59:14Z

dockers/docker-config-engine-stretch/Dockerfile.j2

-        python-dev            \
-        python-setuptools
-
-RUN pip install --upgrade pip


not sure why we need to modify stretch docker.

This is simply to align the way we install/upgrade pip everywhere

lguohan · 2020-12-12T16:59:34Z

dockers/docker-config-engine/Dockerfile.j2

-RUN pip install --upgrade pip
-
-RUN apt-get purge -y python-pip
+RUN apt-get install -y build-essential python-dev


not sure why we modify this file.

This is simply to align the way we install/upgrade pip everywhere

dockers/docker-orchagent/Dockerfile.j2

lguohan · 2020-12-12T17:00:59Z

sonic-slave-buster/Dockerfile.j2

@@ -341,7 +341,7 @@ RUN export VERSION=1.14.2 \
 && rm go$VERSION.linux-*.tar.gz

 RUN pip3 install --upgrade pip
-RUN pip2 install --upgrade pip
+RUN pip2 install --upgrade 'pip<21'


not sure why need this

In January, pip version 21 will be released, which will drop support for Python 2. Therefore, we need to ensure we install a version < 21 for Python 2 purposes.

lguohan · 2020-12-12T17:01:05Z

sonic-slave-stretch/Dockerfile.j2

@@ -329,7 +329,7 @@ RUN export VERSION=1.14.2 \
 && rm go$VERSION.linux-*.tar.gz

 RUN pip3 install --upgrade pip
-RUN pip2 install --upgrade pip
+RUN pip2 install --upgrade 'pip<21'


not sure why need this

In January, pip version 21 will be released, which will drop support for Python 2. Therefore, we need to ensure we install a version < 21 for Python 2 purposes.

dockers/docker-orchagent/Dockerfile.j2

zhenggen-xu · 2020-12-13T06:06:02Z

dockers/docker-orchagent/Dockerfile.j2

+         scapy==2.4.4 \
+         pyroute2==0.5.14 \
+         netifaces==0.10.9 \
+         monotonic==1.5


Do we have python3 < 3.3 supported? Wondering if we should use time lib directly on python 3.
https://pypi.org/project/monotonic/

Having this installed is fine to support python3 across the versions. but just to check.

As stated above, I simply added this here to make the Python 3 environment in the container match the Python 2 environment, which I though would help you in your efforts to convert restore_neighbors.py to Python 3. With Python 3, we have Python 3.7 installed. I don't see any reason for us to need to support Python 3 < 3.7.

Please feel free to modify the dependencies as part of your restore_neighbors.py Python 3 pull request.

lguohan · 2020-12-13T07:14:23Z

retest vsimage please

jleveque · 2020-12-19T20:01:09Z

Check builds are stuck in the state "Expected — Waiting for status to be reported". Force push will not fix. Closing and reopening PR in an attempt to hopefully fix.

svc-acs · 2020-12-19T20:14:23Z

test comment

lguohan · 2020-12-19T20:18:49Z

test

lguohan · 2020-12-20T19:51:25Z

retest vsimage please

lguohan · 2020-12-20T19:51:58Z

retest mellanox please

lguohan · 2020-12-21T13:16:41Z

retest vsimage please

jleveque · 2020-12-22T02:02:40Z

vsimage test failing, as it relies on sonic-net/sonic-swss#1542, which will be merged as part of this submodule update: #6270

lguohan · 2020-12-24T09:51:30Z

retst vsimage please

…l in base containers

…ython 3

…r needed

…ramework

…o be Python 3-compliant

lguohan · 2020-12-25T07:40:53Z

is it still draft?

jleveque added Cleanup 💧 Python 3 labels Dec 8, 2020

jleveque self-assigned this Dec 8, 2020

lguohan reviewed Dec 12, 2020

View reviewed changes

dockers/docker-orchagent/Dockerfile.j2 Outdated Show resolved Hide resolved

lguohan reviewed Dec 12, 2020

View reviewed changes

zhenggen-xu reviewed Dec 13, 2020

View reviewed changes

zhenggen-xu mentioned this pull request Dec 13, 2020

[restore_neighbors] python3 support for restore_neighbors.py sonic-net/sonic-swss#1542

Merged

jleveque closed this Dec 19, 2020

jleveque reopened this Dec 19, 2020

jleveque added 8 commits December 24, 2020 18:40

[docker-base-buster] No longer install Python 2

42c792b

Remove Python 2 from docker-config-engine-buster

ebf10a4

Install pip2 in PMon container

cfb9452

Minor tweaks to SNMP Dockerfile

b3fe099

Install pip2 in docker-sonic-vs until we can remove entirely

67ffd8a

Install Python 3 version of sonic-cfggen in all containers, Py2 in PMon

ebf67a0

No longer install Python 2 in docker-orchagent

b1f0b1d

Install build-essential in PMon

6bad421

jleveque added 9 commits December 24, 2020 18:40

Additional cleanup; Install/upgrade pip, intstall setuptools and whee…

1c88eaf

…l in base containers

Add Python 2 back to docker-orchagent until restore_neighbors.py is P…

3207bdd

…ython 3

Add quotes around pip version

2728f3f

Add Python 2 back to docker-sonic-mgmt-framework until it is no longe…

a156956

…r needed

Install python3-distutils in docker-base-buster

7859787

Install Python 2 setuptools and wheel packages in docker-sonic-mgmt-f…

82b64b6

…ramework

Fix new merge conflicts

5b84690

[docker-database-init.sh] Call python3 explicitly

b0f53d0

[docker-database/supervisord.conf.j2] Change 'iteritems' to 'items' t…

6e8679b

…o be Python 3-compliant

lguohan approved these changes Dec 25, 2020

View reviewed changes

lguohan force-pushed the master branch from 3690c1a to 512eb6b Compare December 25, 2020 18:34

jleveque marked this pull request as ready for review December 25, 2020 19:32

jleveque merged commit d40c9a1 into sonic-net:master Dec 26, 2020

jleveque deleted the remove_py2_containers branch December 26, 2020 05:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[docker-base-buster][docker-config-engine-buster] No longer install Python 2 #6162

[docker-base-buster][docker-config-engine-buster] No longer install Python 2 #6162

jleveque commented Dec 8, 2020 •

edited

Loading

lguohan Dec 12, 2020

jleveque Dec 12, 2020 •

edited

Loading

lguohan Dec 12, 2020

jleveque Dec 12, 2020 •

edited

Loading

lguohan Dec 12, 2020

jleveque Dec 12, 2020 •

edited

Loading

lguohan Dec 12, 2020

jleveque Dec 12, 2020

lguohan Dec 12, 2020

jleveque Dec 12, 2020 •

edited

Loading

zhenggen-xu Dec 13, 2020

jleveque Dec 14, 2020

lguohan commented Dec 13, 2020

jleveque commented Dec 19, 2020

svc-acs commented Dec 19, 2020

lguohan commented Dec 19, 2020

lguohan commented Dec 20, 2020

lguohan commented Dec 20, 2020

lguohan commented Dec 21, 2020

jleveque commented Dec 22, 2020

lguohan commented Dec 24, 2020

lguohan commented Dec 25, 2020

[docker-base-buster][docker-config-engine-buster] No longer install Python 2 #6162

[docker-base-buster][docker-config-engine-buster] No longer install Python 2 #6162

Conversation

jleveque commented Dec 8, 2020 • edited Loading

Choose a reason for hiding this comment

jleveque Dec 12, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jleveque Dec 12, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jleveque Dec 12, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jleveque Dec 12, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lguohan commented Dec 13, 2020

jleveque commented Dec 19, 2020

svc-acs commented Dec 19, 2020

lguohan commented Dec 19, 2020

lguohan commented Dec 20, 2020

lguohan commented Dec 20, 2020

lguohan commented Dec 21, 2020

jleveque commented Dec 22, 2020

lguohan commented Dec 24, 2020

lguohan commented Dec 25, 2020

jleveque commented Dec 8, 2020 •

edited

Loading

jleveque Dec 12, 2020 •

edited

Loading

jleveque Dec 12, 2020 •

edited

Loading

jleveque Dec 12, 2020 •

edited

Loading

jleveque Dec 12, 2020 •

edited

Loading