Can't install docker using pip on Debian 12

[phiilu]

Using the following playbook worked just fine using Debian 11, but it seems to break on Debian 12:

- hosts: all
  become: true
  roles:
    - role: geerlingguy.pip
      vars:
      pip_install_packages:
        - name: docker

Error:

│ TASK [geerlingguy.pip : Ensure Pip is installed.] ******************************
│ changed: [49.X.X.X]
│ 
│ TASK [geerlingguy.pip : Ensure pip_install_packages are installed.] ************
│ failed: [49.X.X.X] (item={'name': 'docker'}) => {"ansible_loop_var": "item", "changed": false, "cmd": ["/usr/bin/pip3", "install",
│ "docker"], "item": {"name": "docker"}, "msg": "\n:stderr: error: externally-managed-environment\n\n× This environment is externally
│ managed\n╰─> To install Python packages system-wide, try apt install\n    python3-xyz, where xyz is the package you are trying to\n
│ install.\n    \n    If you wish to install a non-Debian-packaged Python package,\n    create a virtual environment using python3 -m
│ venv path/to/venv.\n    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make\n    sure you have python3-full installed.\n
│ \n    If you wish to install a non-Debian packaged Python application,\n    it may be easiest to use pipx install xyz, which will
│ manage a\n    virtual environment for you. Make sure you have pipx installed.\n    \n    See /usr/share/doc/python3.11/README.venv
│ for more information.\n\nnote: If you believe this is a mistake, please contact your Python installation or OS distribution provider.
│ You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.\nhint: See PEP 668
│ for the detailed specification.\n"}

Should I pass the option --break-system-packages as the error message suggests or should I use a virtualenv. I am not very familiar with the Python package manager or virtualenvs and I just need to be able to use Ansible to manage Docker.

Thanks for your Videos and Open Source efforts 🫶🏼

[geerlingguy]

See: https://github.com/geerlingguy/docker-debian12-ansible/blob/master/Dockerfile#L19-L20

That's the fix I used to use Pip in a Docker image, I think it might be easiest to just ensure that file is created on Debian 12 systems for now.

It's annoying when python virtualenvs are trying to be shoved down everyone's throats, when system python has been just fine for decades :P

[phiilu]

Thanks for the linked solution, that's better than adding --break-system-packages to every package I need to install :)

I mean yes they give a solution by telling us to install docker via apt install python3-docker, but as a non-python developer I don't have much experience with why that would be better than installing it using pip3 and it now breaks existing scripts/playbooks ^^

Should the role pip be adapted to remove this file automatically or maybe add a var to force the removal of the /usr/lib/python3.11/EXTERNALLY-MANAGED file?

[zloveless]

Just ran into this as well when checking out some bugs someone else on my team was having with my playbook.

Ended up working around it with 2 quick and dirty tasks...

  tasks:
    - name: get python3 version installed
      ansible.builtin.command: python3 --version
      register: py3ver
      when: ansible_distribution == "Debian" and ansible_distribution_version == "12"

    - name: remove EXTERNALLY-MANAGED if we're on debian12
      ansible.builtin.file:
        path: /usr/lib/python3.11/EXTERNALLY-MANAGED
        state: absent
      when: py3ver is defined and py3ver.stdout.find("3.11") != -1

edit: my use case is just installing a couple python packages on a vm/vps. I'm not using it in docker. Just to install the docker api so ansible's docker modules can work.

[lenrik1589]

Hello, just wanted to add my two cents, there appear to exist a special option: virtualenv that allows you to specify python virtual environment, i would prefer to see that utilized as opposed to removing EXTERNALLY-MANAGED as that was put there explicitly by many distribution to prevent users from breaking package managers (video), which is much worse than

now breaks existing scripts/playbooks ^^

@phiilu
(there is a reason, that the option is called "break system packages")

[lenrik1589]

the more I look into it, the more I think there is pretty much nothing we can do at this time, as current community.docker.docker_compose relies on docker-compose-v1 which has been EoL since may of 2021 (citation needed), currently there is this PR that tries to add support for docker-compose>=2.0

although… I managed to get something running, but it is infuriating to work with as i am not experienced with networking and sick and so every problem causes me stress.

looking at it, this is even the wrong repo…

[zloveless]

Just ran into this as well when checking out some bugs someone else on my team was having with my playbook.

Ended up working around it with 2 quick and dirty tasks...

  tasks:
    - name: get python3 version installed
      ansible.builtin.command: python3 --version
      register: py3ver
      when: ansible_distribution == "Debian" and ansible_distribution_version == "12"

    - name: remove EXTERNALLY-MANAGED if we're on debian12
      ansible.builtin.file:
        path: /usr/lib/python3.11/EXTERNALLY-MANAGED
        state: absent
      when: py3ver is defined and py3ver.stdout.find("3.11") != -1

edit: my use case is just installing a couple python packages on a vm/vps. I'm not using it in docker. Just to install the docker api so ansible's docker modules can work.

So I noticed this got included in #58 and that it's broken now. ansible_distribution_version needs to be changed to ansible_distribution_major_version at minimum.

[julianborghuis]

sudo rm /usr/lib/python3.11/EXTERNALLY-MANAGED

Then it works ;)

[socheatsok78]

It seem that this issue resurface on Ubuntu 24 as well python3.12.

[thomasvvugt]

Prepending this small role did the trick for me, regardless of Ubuntu/Debian shoving down venvs;

---
- name: Get Python 3.X version
  ansible.builtin.command: python3 --version
  register: pyver
  changed_when: false
  failed_when: pyver.rc != 0

- name: Remove Python 3.X EXTERNALLY-MANAGED file
  ansible.builtin.file:
    state: absent
    path: "/usr/lib/python{{ pyver.stdout.split()[1] | regex_search('([0-9]+\\.[0-9]+)') }}/EXTERNALLY-MANAGED"
  when: pyver.stdout | regex_search('3\.[0-9]+')