Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix collection module info quirk #68223

Closed

Conversation

abadger
Copy link
Contributor

@abadger abadger commented Mar 13, 2020

SUMMARY
Fix the detection of collection module_utils

Before, when you imported an identifier from a module_util with the same
name as the Python module it belongs in, the CollectionModuleInfo class
would think that the identifier was a Python module and return the
source for the Python module when requested.  This change fixes the code
so that the class will throw an error in this case.

The lack of error was causing an issue when constructing AnsiballZs.
For instance:

    from ..module_utils.ismount import ismount

The former code would create the file:
    ansible_collections/ansible/posix/plugins/module_utils/ismount/ismount.py

Instead of:
    ansible_collections/ansible/posix/plugins/module_utils/ismount.py
ISSUE TYPE
  • Bugfix Pull Request
COMPONENT NAME
ADDITIONAL INFORMATION
  • This also implements pkg_dir in the CollectionModuleInfo class. I'm not sure if there's more that needs to be done to make use of that.
  • In a similar vein, the FIXMEs around collections in Ansiballz should be reviewed to see if any of them are also resolved by this change.
  • Lastly.... As I wrote this I kept feeling like there was probably a better way to do this. It feels like there should be an API in pkg_utils or importlib/imp or something else that a python loader or python finder might implement that would do the same thing as my code. I couldn't find it, though, so maybe that's just something to keep in mind as a potential refactor when the collection loader is next worked on.

@ansibot ansibot added affects_2.10 This issue/PR affects Ansible v2.10 bug This issue/PR relates to a bug. core_review In order to be merged, this PR must follow the core review workflow. needs_triage Needs a first human triage before being processed. new_plugin This PR includes a new plugin. support:community This issue/PR relates to code supported by the Ansible community. support:core This issue/PR relates to code supported by the Ansible Engineering Team. labels Mar 13, 2020
@abadger abadger force-pushed the fix-collection-module-info-quirk branch from fa491f3 to 9c90139 Compare March 13, 2020 21:35
@abadger
Copy link
Contributor Author

abadger commented Mar 13, 2020

ci_complete

@abadger abadger force-pushed the fix-collection-module-info-quirk branch from 9c90139 to c1acab0 Compare March 13, 2020 21:39
@ansibot
Copy link
Contributor

ansibot commented Mar 13, 2020

The test ansible-test sanity --test pylint [explain] failed with 1 error:

test/integration/targets/collections/collection_root_user/ansible_collections/testns/testcoll/plugins/modules/test_quirk.py:16:0: relative-beyond-top-level: Attempted relative import beyond top-level package

The test ansible-test sanity --test future-import-boilerplate [explain] failed with 1 error:

test/integration/targets/collections/collection_root_user/ansible_collections/testns/testcoll/plugins/module_utils/quirk.py:0:0: missing: from __future__ import (absolute_import, division, print_function)

The test ansible-test sanity --test metaclass-boilerplate [explain] failed with 1 error:

test/integration/targets/collections/collection_root_user/ansible_collections/testns/testcoll/plugins/module_utils/quirk.py:0:0: missing: __metaclass__ = type

The test ansible-test sanity --test pep8 [explain] failed with 1 error:

lib/ansible/executor/module_common.py:691:5: E301: expected 1 blank line, found 0

click here for bot help

@ansibot ansibot added needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. and removed core_review In order to be merged, this PR must follow the core review workflow. labels Mar 13, 2020
@abadger abadger force-pushed the fix-collection-module-info-quirk branch 2 times, most recently from 247c37c to 34096a1 Compare March 13, 2020 22:42
@ansibot ansibot added core_review In order to be merged, this PR must follow the core review workflow. and removed needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. labels Mar 13, 2020
@abadger abadger force-pushed the fix-collection-module-info-quirk branch 2 times, most recently from e0c146a to eea920a Compare March 13, 2020 23:36
@ansibot ansibot added needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. and removed core_review In order to be merged, this PR must follow the core review workflow. labels Mar 13, 2020
@ansibot
Copy link
Contributor

ansibot commented Mar 14, 2020

The test ansible-test sanity --test pylint [explain] failed with 1 error:

test/integration/targets/collections/collection_root_user/ansible_collections/testns/testcoll/plugins/modules/test_quirk.py:16:0: relative-beyond-top-level: Attempted relative import beyond top-level package

click here for bot help

@abadger abadger force-pushed the fix-collection-module-info-quirk branch from eea920a to 9298e45 Compare March 14, 2020 15:38
Before, when you import an identifier from a module_util with the same
name as the Python module it belongs in, the CollectionModuleInfo class
would think that the identifier was a Python module and return the
source for the Python module when requested.  This change fixes the code
so that the class will throw an error in this case.

The lack of error was causing an issue when constructing AnsiballZs.
For instance:

    from ..module_utils.ismount import ismount

The former code would create the file:
    ansible_collections/ansible/posix/plugins/module_utils/ismount/ismount.py

Instead of:
    ansible_collections/ansible/posix/plugins/module_utils/ismount.py

* The fix also implements the pkg_dir flag for the CollectionModuleInfo
  class.
* Since the fix requires that we cache whether the Finder found the
  module but didn't load it, make use of that when deciding whether the
  module has already been found.

ci_complete
…strings

Convert to native string at the border.  We should add docstrings and
rename variables to n_* to show this more clearly.

ci_complate
@abadger abadger force-pushed the fix-collection-module-info-quirk branch from 9298e45 to fc45e43 Compare March 14, 2020 15:46
@ansibot ansibot added core_review In order to be merged, this PR must follow the core review workflow. and removed needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. core_review In order to be merged, this PR must follow the core review workflow. labels Mar 14, 2020
@ansibot ansibot added the needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. label Mar 14, 2020
@sivel sivel requested a review from nitzmahone March 17, 2020 15:13
@sivel sivel removed the needs_triage Needs a first human triage before being processed. label Mar 17, 2020
@ansibot ansibot added stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. has_issue labels Mar 26, 2020
felixfontein added a commit to felixfontein/community.general that referenced this pull request Mar 30, 2020
gundalow pushed a commit to ansible-collections/community.general that referenced this pull request Mar 30, 2020
* Remove meraki unit tests.

* Reenable disabled tests.

* Re-disable xfs_quota tests - depend on ansible/ansible#68223.

* Fix FQCNs.

* More FQCNs.

* Fix typo.
felixfontein added a commit to felixfontein/community.crypto that referenced this pull request Apr 5, 2020
@ansibot ansibot added core_review In order to be merged, this PR must follow the core review workflow. and removed needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. labels Apr 8, 2020
felixfontein added a commit to felixfontein/community.crypto that referenced this pull request Apr 14, 2020
@ansibot ansibot added the stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. label Apr 24, 2020
@gundalow
Copy link
Contributor

gundalow commented May 4, 2020

@felixfontein Hi, I believe you tested this PR did it improve the situation for you? Was it an issue in mount, or something else?

@abadger What do you think the next steps are for this?

@felixfontein
Copy link
Contributor

@gundalow I tested this PR just for another problem (which it didn't solve).

@abadger
Copy link
Contributor Author

abadger commented May 5, 2020

@gundalow nitzmahone said he Incorporated a better fix into his routing pr (@nitzmahone can you confirm?)

We can probably look at the routing pr to see if the tests i added here got added there as another way to confirm whether there's a fix in there

@nitzmahone nitzmahone mentioned this pull request May 5, 2020
19 tasks
@abadger
Copy link
Contributor Author

abadger commented May 5, 2020

I see a test for this case in the routing PR so I think that it is fixed in a different way there.

@abadger
Copy link
Contributor Author

abadger commented May 28, 2020

The routing PR (with test for this issue) has now been merged. felixfontein confirms that xfs_quota is no longer broken. Closing this issue.

@abadger abadger closed this May 28, 2020
@abadger abadger deleted the fix-collection-module-info-quirk branch May 28, 2020 19:04
@ansible ansible locked and limited conversation to collaborators Jun 25, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
affects_2.10 This issue/PR affects Ansible v2.10 bug This issue/PR relates to a bug. core_review In order to be merged, this PR must follow the core review workflow. has_issue new_plugin This PR includes a new plugin. stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. support:community This issue/PR relates to code supported by the Ansible community. support:core This issue/PR relates to code supported by the Ansible Engineering Team.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants