Offline mode doesn't work with ANSIBLE_COLLECTIONS_PATH

[nre-ableton]

Summary

The recently-fixed --offline mode no longer works when collections are expected to be found outside of ~/.cache/ansible-compat/<hash>.

Issue Type

• Bug Report

Ansible and Ansible Lint details

ansible --version
ansible [core 2.14.0]
  config file = None
  configured module search path = ['/home/nre/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/nre/.cache/virtualenv/ansible-role-nix/lib/python3.10/site-packages/ansible
  ansible collection location = /home/nre/Code/Ableton/ansible-role-nix/.ansible/collections
  executable location = /home/nre/.cache/virtualenv/ansible-role-nix/bin/ansible
  python version = 3.10.7 (main, Nov 14 2022, 13:35:39) [GCC 11.3.0] (/home/nre/.cache/virtualenv/ansible-role-nix/bin/python)
  jinja version = 3.1.2
  libyaml = True

ansible-lint --version
ansible-lint 6.8.6 using ansible 2.14.0

• ansible installation method: one of source, pip, OS package: pip
• ansible-lint installation method: one of source, pip, OS package: pip

OS / ENVIRONMENT

Ubuntu-ish Linux 22.04.

STEPS TO REPRODUCE

Running ansible-lint on our repository now fails, even when ANSIBLE_COLLECTIONS_PATH is set correctly:

WARNING  Retrying execution failure 1 of: ansible-galaxy collection install -v --offline -r requirements.yml -p /home/nre/.cache/ansible-compat/1a0050/collections
ERROR    No config file found; using defaults
Starting galaxy collection install process
Process install dependency map

ERROR    [WARNING]: The specified collections path '/home/nre/.cache/ansible-
compat/1a0050/collections' is not part of the configured Ansible collections
paths '/home/nre/Code/Ableton/ansible-role-nix/.ansible/collections'. The
installed collection will not be picked up in an Ansible run, unless within a
playbook-adjacent collections directory.
ERROR! Failed to resolve the requested dependencies map. Could not satisfy the following requirements:
* community.docker:3.2.1 (direct request)

Got 1 exit code while running: ansible-galaxy collection install -v --offline -r requirements.yml -p /home/nre/.cache/ansible-compat/1a0050/collections

Related to #2651, but I'm not 100% sure that it's the same problem, so I opened another issue. Apologies if this issue should be treated as a dupe.

[nre-ableton]

Just to be clear, we do install collections and roles before attempting to run ansible-lint.

[simontunnat]

Same problem here. Ansible lint 6.x seems pretty unstable right now. :(

[nre-ableton]

FYI this is still broken in ansible-lint 6.9.0.

[bluikko]

Is this supposed to work in 6.9.1?

Collections are not detected (/usr/share/ansible/collections, as is listed in the error message) and with --offline errors are printed, without --offline the same already installed collections are downloaded again:

$ ansible-lint --offline
WARNING  Retrying execution failure 1 of: ansible-galaxy collection install -v --offline -r collections/requirements.yml -p /home/ansible/.cache/ansible-compat/5dd666/collections
ERROR    Using /etc/ansible/ansible.cfg as config file
Starting galaxy collection install process
Process install dependency map

ERROR    [WARNING]: The specified collections path '/home/ansible/.cache/ansible-
compat/5dd666/collections' is not part of the configured Ansible collections
paths '/home/ansible/.ansible/collections:/usr/share/ansible/collections'. The
installed collection will not be picked up in an Ansible run, unless within a
playbook-adjacent collections directory.
ERROR! Failed to resolve the requested dependencies map. Could not satisfy the following requirements:
* ansible.netcommon:* (direct request)

Got 1 exit code while running: ansible-galaxy collection install -v --offline -r collections/requirements.yml -p /home/ansible/.cache/ansible-compat/5dd666/collections

ansible.netcommon is installed:

$ ansible-galaxy collection list

# /usr/share/ansible/collections/ansible_collections
Collection        Version
----------------- -------
ansible.netcommon 4.1.0
ansible.posix     1.4.0
ansible.utils     2.8.0
ansible.windows   1.12.0

ansible-lint 6.9.1 using ansible 2.14.0

It is impossible to download collections on every CI/CD pipeline run, it takes several minutes.

[ssbarnea]

@nre-ableton One of the scopes of ansible-lint is to check that the developer got all requirements in place, basically that he did not forget to mention some of them in his requirements.yml file and the code just happened to work on his machine, just to crash later in production.

In order to be able to test the presence of requirements, by default ansible-lint runs in insolation, meaning that it will tell ansible to not look for collections anywhere else than the special location used by ansible-lint. So it will prevent ansible from finding collections installed in default system location, or user default location.

Check

ansible-lint/src/ansiblelint/app.py

Line 48 in 884d098

self.runtime = Runtime(isolated=True)

-- which initialized ansible runtime with isolated mode.

At this moment we did not add an option to disable the isolation mode but that could be quite easy. Still, you need to be aware that by disabling the isolation mode it means that ansible-lint will install and potentially override collections that you might already have installed. Still, this can be avoided by adding the --offline argument.

Basically it is a feature, not a bug. A feature that ensures that the tool will produce the same results on all machines, regardless what goodies (collections, roles) the user installed in his own machine.

I hope this explains it. Let me know what you think and what should we do, maybe we should also open a discussion in order to decide which changes should be made to accomodate for various use cases.

[bluikko]

@ssbarnea that feature should perhaps be documented. I could not see it documented and on the contrary this documentation fragment seemed to be now then outdated:

For optimal performance, Ansible-lint creates caches with installed or mocked roles, collections, and modules in the {project_dir}/.cache folder. The location of {project_dir} is passed with a command line argument, determined by the location of the configuration file, git project top-level directory, or user home directory.

Regarding your explanation (thank you!) it seems you are only considering one use case, that of a development workstation.

Edit: the simplest workaround for this issue is to simply rename/delete requirements.yml for the duration of ansible-lint. We do not want random developers to be able to pull in random collections/roles, they are managed on a different level.

Edit 2: another misleading documentation at configuration documentation says

# Offline mode disables installation of requirements.yml

[rgl]

Please provide a way for not using the isolated runtime or make it implied with --offline. For my use case, I ship the depencies in the container image, and do not need to download them again.

[ssbarnea]

@rgl That is indeed a good point. I think that most reasons for --offline are towards not doing network access and assuming that the code is already running in production-like environment, making a LOT of sense to disable the isolation more.

Still, this opens another question: Should offline mode also disable the mocking support? Somehow I have the impression that we don't want to accidentally mock anything and ignore the missing dependencies when doing so.

If that would be against other use-cases of offline, we might have to add a new command line option that describes this mode.

[rgl]

@ssbarnea, sorry, but I do not known about ansible-lint mocking. I'm not familiar with mocking in this context (which I assume, you are referring to mock_modules and mock_roles as can be seen at the Ansible-lint configuration documentation, which I never used).

[nre-ableton]

@nre-ableton One of the scopes of ansible-lint is to check that the developer got all requirements in place, basically that he did not forget to mention some of them in his requirements.yml file and the code just happened to work on his machine, just to crash later in production.

In order to be able to test the presence of requirements, by default ansible-lint runs in insolation, meaning that it will tell ansible to not look for collections anywhere else than the special location used by ansible-lint. So it will prevent ansible from finding collections installed in default system location, or user default location.

@ssbarnea Fair enough, but as noted by others, this is only one use-case. The other use-case is obviously CI installations and similar. Actually, we only run our production playbooks from a CI-like server, because we precisely don't want to end up in a situation where something worked on a developer's machine because they installed a role/collection that wasn't added to requirements.yml.

Check

ansible-lint/src/ansiblelint/app.py

Line 48 in 884d098

self.runtime = Runtime(isolated=True)

-- which initialized ansible runtime with isolated mode.

At this moment we did not add an option to disable the isolation mode but that could be quite easy. Still, you need to be aware that by disabling the isolation mode it means that ansible-lint will install and potentially override collections that you might already have installed. Still, this can be avoided by adding the --offline argument.

It would be nice if there was an option (either from the CLI or in ansible-lint.cfg, preferably the latter to avoid bloating the CLI's --help).

Also, regarding ansible-lint role mocking, I wasn't aware of that feature, but I don't think it's a great option (at least in my use-case) because (a) we rely on a ton of roles, and (b) we also rely on collections, and I want ansible-lint to inspect code that uses said collections.

Rather than opening a new discussion, let's continue this conversation at #2786.