Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.0.5: pytest fails #10

Closed
kloczek opened this issue Feb 19, 2024 · 16 comments · Fixed by #13
Closed

1.0.5: pytest fails #10

kloczek opened this issue Feb 19, 2024 · 16 comments · Fixed by #13

Comments

@kloczek
Copy link

kloczek commented Feb 19, 2024

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

  • python3 -sBm build -w --no-isolation
  • because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
  • install .whl file in </install/prefix> using 'installer` module
  • run pytest with $PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
  • build is performed in env which is cut off from access to the public network (pytest is executed with -m "not network")
Here is pytest output:
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-lazy-fixtures-1.0.5-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-lazy-fixtures-1.0.5-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network'
Traceback (most recent call last):
  File "/usr/bin/pytest", line 8, in <module>
    sys.exit(console_main())
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 198, in console_main
    code = main()
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 156, in main
    config = _prepareconfig(args, plugins)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 338, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "/usr/lib/python3.8/site-packages/pluggy/_hooks.py", line 493, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
  File "/usr/lib/python3.8/site-packages/pluggy/_manager.py", line 115, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/usr/lib/python3.8/site-packages/pluggy/_callers.py", line 113, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/usr/lib/python3.8/site-packages/pluggy/_callers.py", line 96, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/usr/lib/python3.8/site-packages/_pytest/helpconfig.py", line 105, in pytest_cmdline_parse
    config = yield
  File "/usr/lib/python3.8/site-packages/pluggy/_callers.py", line 77, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1096, in pytest_cmdline_parse
    self.parse(args)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1449, in parse
    self._preparse(args, addopts=addopts)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1348, in _preparse
    self.hook.pytest_load_initial_conftests(
  File "/usr/lib/python3.8/site-packages/pluggy/_hooks.py", line 493, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
  File "/usr/lib/python3.8/site-packages/pluggy/_manager.py", line 115, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/usr/lib/python3.8/site-packages/pluggy/_callers.py", line 113, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/usr/lib/python3.8/site-packages/pluggy/_callers.py", line 96, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/usr/lib/python3.8/site-packages/_pytest/warnings.py", line 148, in pytest_load_initial_conftests
    return (yield)
  File "/usr/lib/python3.8/site-packages/pluggy/_callers.py", line 96, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/usr/lib/python3.8/site-packages/_pytest/capture.py", line 153, in pytest_load_initial_conftests
    yield
  File "/usr/lib/python3.8/site-packages/pluggy/_callers.py", line 77, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1174, in pytest_load_initial_conftests
    self.pluginmanager._set_initial_conftests(
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 576, in _set_initial_conftests
    self._try_load_conftest(anchor, importmode, rootpath)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 598, in _try_load_conftest
    self._loadconftestmodules(x, importmode, rootpath)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 621, in _loadconftestmodules
    mod = self._importconftest(conftestpath, importmode, rootpath)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 676, in _importconftest
    self.consider_conftest(mod, registration_name=conftestpath_plugin_name)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 760, in consider_conftest
    self.register(conftestmodule, name=registration_name)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 511, in register
    self.consider_module(plugin)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 768, in consider_module
    self._import_plugin_specs(getattr(mod, "pytest_plugins", []))
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 775, in _import_plugin_specs
    self.import_plugin(import_spec)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 812, in import_plugin
    self.register(mod, modname)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 500, in register
    plugin_name = super().register(plugin, name)
  File "/usr/lib/python3.8/site-packages/pluggy/_manager.py", line 141, in register
    raise ValueError(
ValueError: Plugin already registered under a different name: pytest_lazy_fixtures.plugin=<module 'pytest_lazy_fixtures.plugin' from '/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-lazy-fixtures-1.0.5-2.fc35.x86_64/usr/lib/python3.8/site-packages/pytest_lazy_fixtures/plugin.py'>
{'140719200418064': <_pytest.config.PytestPluginManager object at 0x7ffbbdf43910>, 'pytestconfig': <_pytest.config.Config object at 0x7ffbbce2d370>, 'mark': <module '_pytest.mark' from '/usr/lib/python3.8/site-packages/_pytest/mark/__init__.py'>, 'main': <module '_pytest.main' from '/usr/lib/python3.8/site-packages/_pytest/main.py'>, 'runner': <module '_pytest.runner' from '/usr/lib/python3.8/site-packages/_pytest/runner.py'>, 'fixtures': <module '_pytest.fixtures' from '/usr/lib/python3.8/site-packages/_pytest/fixtures.py'>, 'helpconfig': <module '_pytest.helpconfig' from '/usr/lib/python3.8/site-packages/_pytest/helpconfig.py'>, 'python': <module '_pytest.python' from '/usr/lib/python3.8/site-packages/_pytest/python.py'>, 'terminal': <module '_pytest.terminal' from '/usr/lib/python3.8/site-packages/_pytest/terminal.py'>, 'debugging': <module '_pytest.debugging' from '/usr/lib/python3.8/site-packages/_pytest/debugging.py'>, 'unittest': <module '_pytest.unittest' from '/usr/lib/python3.8/site-packages/_pytest/unittest.py'>, 'capture': <module '_pytest.capture' from '/usr/lib/python3.8/site-packages/_pytest/capture.py'>, 'skipping': <module '_pytest.skipping' from '/usr/lib/python3.8/site-packages/_pytest/skipping.py'>, 'legacypath': <module '_pytest.legacypath' from '/usr/lib/python3.8/site-packages/_pytest/legacypath.py'>, 'tmpdir': <module '_pytest.tmpdir' from '/usr/lib/python3.8/site-packages/_pytest/tmpdir.py'>, 'monkeypatch': <module '_pytest.monkeypatch' from '/usr/lib/python3.8/site-packages/_pytest/monkeypatch.py'>, 'recwarn': <module '_pytest.recwarn' from '/usr/lib/python3.8/site-packages/_pytest/recwarn.py'>, 'pastebin': <module '_pytest.pastebin' from '/usr/lib/python3.8/site-packages/_pytest/pastebin.py'>, 'nose': <module '_pytest.nose' from '/usr/lib/python3.8/site-packages/_pytest/nose.py'>, 'assertion': <module '_pytest.assertion' from '/usr/lib/python3.8/site-packages/_pytest/assertion/__init__.py'>, 'junitxml': <module '_pytest.junitxml' from '/usr/lib/python3.8/site-packages/_pytest/junitxml.py'>, 'doctest': <module '_pytest.doctest' from '/usr/lib/python3.8/site-packages/_pytest/doctest.py'>, 'cacheprovider': <module '_pytest.cacheprovider' from '/usr/lib/python3.8/site-packages/_pytest/cacheprovider.py'>, 'freeze_support': <module '_pytest.freeze_support' from '/usr/lib/python3.8/site-packages/_pytest/freeze_support.py'>, 'setuponly': <module '_pytest.setuponly' from '/usr/lib/python3.8/site-packages/_pytest/setuponly.py'>, 'setupplan': <module '_pytest.setupplan' from '/usr/lib/python3.8/site-packages/_pytest/setupplan.py'>, 'stepwise': <module '_pytest.stepwise' from '/usr/lib/python3.8/site-packages/_pytest/stepwise.py'>, 'warnings': <module '_pytest.warnings' from '/usr/lib/python3.8/site-packages/_pytest/warnings.py'>, 'logging': <module '_pytest.logging' from '/usr/lib/python3.8/site-packages/_pytest/logging.py'>, 'reports': <module '_pytest.reports' from '/usr/lib/python3.8/site-packages/_pytest/reports.py'>, 'python_path': <module '_pytest.python_path' from '/usr/lib/python3.8/site-packages/_pytest/python_path.py'>, 'unraisableexception': <module '_pytest.unraisableexception' from '/usr/lib/python3.8/site-packages/_pytest/unraisableexception.py'>, 'threadexception': <module '_pytest.threadexception' from '/usr/lib/python3.8/site-packages/_pytest/threadexception.py'>, 'faulthandler': <module '_pytest.faulthandler' from '/usr/lib/python3.8/site-packages/_pytest/faulthandler.py'>, 'pytest_lazyfixture': <module 'pytest_lazy_fixtures.plugin' from '/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-lazy-fixtures-1.0.5-2.fc35.x86_64/usr/lib/python3.8/site-packages/pytest_lazy_fixtures/plugin.py'>, 'capturemanager': <CaptureManager _method='fd' _global_capturing=<MultiCapture out=<FDCapture 1 oldfd=5 _state='started' tmpfile=<_io.TextIOWrapper name="<_io.FileIO name=6 mode='rb+' closefd=True>" mode='r+' encoding='utf-8'>> err=<FDCapture 2 oldfd=7 _state='started' tmpfile=<_io.TextIOWrapper name="<_io.FileIO name=8 mode='rb+' closefd=True>" mode='r+' encoding='utf-8'>> in_=<FDCapture 0 oldfd=3 _state='started' tmpfile=<_io.TextIOWrapper name='/dev/null' mode='r' encoding='utf-8'>> _state='started' _in_suspended=False> _capture_fixture=None>, '/home/tkloczko/rpmbuild/BUILD/pytest-lazy-fixtures-1.0.5/tests/conftest.py': <module 'tests.conftest' from '/home/tkloczko/rpmbuild/BUILD/pytest-lazy-fixtures-1.0.5/tests/conftest.py'>}
error: Bad exit status from /var/tmp/rpm-tmp.zlwQJg (%check)
List of installed modules in build env:
Package            Version
------------------ --------
build              1.0.3
cppclean           0.13
distro             1.9.0
dnf                4.19.0
exceptiongroup     1.1.3
gpg                1.23.2
importlib_metadata 7.0.1
iniconfig          2.0.0
installer          0.7.0
libdnf             0.73.0
packaging          23.2
pluggy             1.3.0
poetry-core        1.9.0
pyproject_hooks    1.0.0
pytest             8.0.1
python-dateutil    2.8.2
six                1.16.0
tomli              2.0.1
wheel              0.42.0
zipp               3.17.0

Please let me know if you need more details or want me to perform some diagnostics.

@mgorny
Copy link

mgorny commented Feb 20, 2024

@kloczek, you can work around that by setting PYTEST_DISABLE_PLUGIN_AUTOLOAD=1.

@kloczek
Copy link
Author

kloczek commented Feb 20, 2024

Why not change the code to make that default behaviour of the pytest execution? 🤔

@kloczek
Copy link
Author

kloczek commented Mar 13, 2024

Just tested 1.0.6 and pytest still fails.

Here is pytest output:
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-lazy-fixtures-1.0.6-2.fc36.x86_64/usr/lib64/python3.9/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-lazy-fixtures-1.0.6-2.fc36.x86_64/usr/lib/python3.9/site-packages
+ /usr/bin/pytest -ra -m 'not network'
============================= test session starts ==============================
platform linux -- Python 3.9.18, pytest-8.1.1, pluggy-1.4.0
rootdir: /home/tkloczko/rpmbuild/BUILD/pytest-lazy-fixtures-1.0.6
configfile: pyproject.toml
collected 19 items

tests/tests.py ..FFFFFFFFFFF......                                       [100%]

=================================== FAILURES ===================================
_____________________ test_lazy_fixture[first0-second0-4] ______________________

first = one, second = two, other = 4

    @pytest.mark.parametrize(("first", "second", "other"), [(lf("one"), lf("two"), 4)])
    def test_lazy_fixture(first, second, other):
>       assert first == 1
E       assert one == 1

tests/tests.py:19: AssertionError
_______________ test_lazy_fixture_data_types[actual0-expected0] ________________

actual = {'a': [one, two]}, expected = {'a': [1, 2]}

    @pytest.mark.parametrize(
        ("actual", "expected"),
        [
            ({"a": [lf("one"), lf("two")]}, {"a": [1, 2]}),
            ({"a": {lf("one"), lf("two")}}, {"a": {1, 2}}),
            ({lf("three"), lf("two"), lf("three")}, {3, 2}),
            ([lf("three"), lf("two")], [3, 2]),
            (
                {"first_level": {"second_level": (lf("one"), lf("one"), lf("two")), "not_lazy_fixture": 1}},
                {"first_level": {"second_level": (1, 1, 2), "not_lazy_fixture": 1}},
            ),
        ],
    )
    def test_lazy_fixture_data_types(actual, expected):
>       assert actual == expected
E       AssertionError: assert {'a': [one, two]} == {'a': [1, 2]}
E
E         Differing items:
E         {'a': [one, two]} != {'a': [1, 2]}
E         Use -v to get more diff

tests/tests.py:38: AssertionError
_______________ test_lazy_fixture_data_types[actual1-expected1] ________________

actual = {'a': {two, one}}, expected = {'a': {1, 2}}

    @pytest.mark.parametrize(
        ("actual", "expected"),
        [
            ({"a": [lf("one"), lf("two")]}, {"a": [1, 2]}),
            ({"a": {lf("one"), lf("two")}}, {"a": {1, 2}}),
            ({lf("three"), lf("two"), lf("three")}, {3, 2}),
            ([lf("three"), lf("two")], [3, 2]),
            (
                {"first_level": {"second_level": (lf("one"), lf("one"), lf("two")), "not_lazy_fixture": 1}},
                {"first_level": {"second_level": (1, 1, 2), "not_lazy_fixture": 1}},
            ),
        ],
    )
    def test_lazy_fixture_data_types(actual, expected):
>       assert actual == expected
E       AssertionError: assert {'a': {two, one}} == {'a': {1, 2}}
E
E         Differing items:
E         {'a': {two, one}} != {'a': {1, 2}}
E         Use -v to get more diff

tests/tests.py:38: AssertionError
_______________ test_lazy_fixture_data_types[actual2-expected2] ________________

actual = {two, three}, expected = {2, 3}

    @pytest.mark.parametrize(
        ("actual", "expected"),
        [
            ({"a": [lf("one"), lf("two")]}, {"a": [1, 2]}),
            ({"a": {lf("one"), lf("two")}}, {"a": {1, 2}}),
            ({lf("three"), lf("two"), lf("three")}, {3, 2}),
            ([lf("three"), lf("two")], [3, 2]),
            (
                {"first_level": {"second_level": (lf("one"), lf("one"), lf("two")), "not_lazy_fixture": 1}},
                {"first_level": {"second_level": (1, 1, 2), "not_lazy_fixture": 1}},
            ),
        ],
    )
    def test_lazy_fixture_data_types(actual, expected):
>       assert actual == expected
E       assert {two, three} == {2, 3}
E
E         Extra items in the left set:
E         two
E         three
E         Extra items in the right set:
E         2
E         3
E         Use -v to get more diff

tests/tests.py:38: AssertionError
_______________ test_lazy_fixture_data_types[actual3-expected3] ________________

actual = [three, two], expected = [3, 2]

    @pytest.mark.parametrize(
        ("actual", "expected"),
        [
            ({"a": [lf("one"), lf("two")]}, {"a": [1, 2]}),
            ({"a": {lf("one"), lf("two")}}, {"a": {1, 2}}),
            ({lf("three"), lf("two"), lf("three")}, {3, 2}),
            ([lf("three"), lf("two")], [3, 2]),
            (
                {"first_level": {"second_level": (lf("one"), lf("one"), lf("two")), "not_lazy_fixture": 1}},
                {"first_level": {"second_level": (1, 1, 2), "not_lazy_fixture": 1}},
            ),
        ],
    )
    def test_lazy_fixture_data_types(actual, expected):
>       assert actual == expected
E       assert [three, two] == [3, 2]
E
E         At index 0 diff: three != 3
E         Use -v to get more diff

tests/tests.py:38: AssertionError
_______________ test_lazy_fixture_data_types[actual4-expected4] ________________

actual = {'first_level': {'not_lazy_fixture': 1, 'second_level': (one, one, two)}}
expected = {'first_level': {'not_lazy_fixture': 1, 'second_level': (1, 1, 2)}}

    @pytest.mark.parametrize(
        ("actual", "expected"),
        [
            ({"a": [lf("one"), lf("two")]}, {"a": [1, 2]}),
            ({"a": {lf("one"), lf("two")}}, {"a": {1, 2}}),
            ({lf("three"), lf("two"), lf("three")}, {3, 2}),
            ([lf("three"), lf("two")], [3, 2]),
            (
                {"first_level": {"second_level": (lf("one"), lf("one"), lf("two")), "not_lazy_fixture": 1}},
                {"first_level": {"second_level": (1, 1, 2), "not_lazy_fixture": 1}},
            ),
        ],
    )
    def test_lazy_fixture_data_types(actual, expected):
>       assert actual == expected
E       AssertionError: assert {'first_level...e, one, two)}} == {'first_level...': (1, 1, 2)}}
E
E         Differing items:
E         {'first_level': {'not_lazy_fixture': 1, 'second_level': (one, one, two)}} != {'first_level': {'not_lazy_fixture': 1, 'second_level': (1, 1, 2)}}
E         Use -v to get more diff

tests/tests.py:38: AssertionError
____________________ test_lazy_fixture_with_fixtures[item0] ____________________

item = four

    @pytest.mark.parametrize("item", [lf("four")])
    def test_lazy_fixture_with_fixtures(item):
>       assert item == 4
E       assert four == 4

tests/tests.py:43: AssertionError
___________ test_lazy_fixture_with_attrs[only_entity0-entity_value0] ___________

only_entity = entity, entity_value = entity.value

    @pytest.mark.parametrize("only_entity,entity_value", [(lf("entity"), lf("entity.value"))])
    def test_lazy_fixture_with_attrs(only_entity, entity_value):
>       assert isinstance(only_entity, Entity)
E       assert False
E        +  where False = isinstance(entity, Entity)

tests/tests.py:48: AssertionError
________________ test_lazy_fixture_callable_with_func[message0] ________________

message = format

    @pytest.mark.parametrize(
        "message",
        [
            lfc(
                "There is two lazy fixture args, {} and {}! And one kwarg {field}! And also simple value {simple}".format,
                lf("one"),
                lf("two"),
                field=lf("three"),
                simple="value",
            ),
        ],
    )
    def test_lazy_fixture_callable_with_func(message):
>       assert message == "There is two lazy fixture args, 1 and 2! And one kwarg 3! And also simple value value"
E       AssertionError: assert format == 'There is two lazy fixture args, 1 and 2! And one kwarg 3! And also simple value value'

tests/tests.py:65: AssertionError
________________ test_lazy_fixture_callable_with_lf[formatted0] ________________

formatted = entity_format, entity = Entity(value=1)

    @pytest.mark.parametrize("formatted", [lfc("entity_format", lf("entity"))])
    def test_lazy_fixture_callable_with_lf(formatted, entity):
>       assert formatted == {"value": entity.value}
E       AssertionError: assert entity_format == {'value': 1}

tests/tests.py:70: AssertionError
_______________ test_lazy_fixture_callable_with_attr_lf[result0] _______________

result = entity.sum

    @pytest.mark.parametrize("result", [lfc("entity.sum", lf("two"))])
    def test_lazy_fixture_callable_with_attr_lf(result):
>       assert result == 3
E       assert entity.sum == 3

tests/tests.py:75: AssertionError
=========================== short test summary info ============================
FAILED tests/tests.py::test_lazy_fixture[first0-second0-4] - assert one == 1
FAILED tests/tests.py::test_lazy_fixture_data_types[actual0-expected0] - Asse...
FAILED tests/tests.py::test_lazy_fixture_data_types[actual1-expected1] - Asse...
FAILED tests/tests.py::test_lazy_fixture_data_types[actual2-expected2] - asse...
FAILED tests/tests.py::test_lazy_fixture_data_types[actual3-expected3] - asse...
FAILED tests/tests.py::test_lazy_fixture_data_types[actual4-expected4] - Asse...
FAILED tests/tests.py::test_lazy_fixture_with_fixtures[item0] - assert four == 4
FAILED tests/tests.py::test_lazy_fixture_with_attrs[only_entity0-entity_value0]
FAILED tests/tests.py::test_lazy_fixture_callable_with_func[message0] - Asser...
FAILED tests/tests.py::test_lazy_fixture_callable_with_lf[formatted0] - Asser...
FAILED tests/tests.py::test_lazy_fixture_callable_with_attr_lf[result0] - ass...
========================= 11 failed, 8 passed in 0.09s =========================

@mtelka
Copy link

mtelka commented Mar 13, 2024

I tested 1.0.6 on OpenIndiana and all tests pass:

============================= test session starts ==============================
platform sunos5 -- Python 3.9.18, pytest-8.1.1, pluggy-1.4.0 -- /usr/bin/python3.9
cachedir: .pytest_cache
rootdir: /data/builds/oi-userland/components/python/pytest-lazy-fixtures/build/amd64-3.9
configfile: pyproject.toml
plugins: lazy-fixtures-1.0.6
collecting ... collected 24 items

tests/tests.py::test_lazy_fixture_repr PASSED                            [  4%]
tests/tests.py::test_lazy_fixture_callable_repr PASSED                   [  8%]
tests/tests.py::test_lazy_fixture[one-two-4] PASSED                      [ 12%]
tests/tests.py::test_lazy_fixture_data_types[actual0-expected0] PASSED   [ 16%]
tests/tests.py::test_lazy_fixture_data_types[actual1-expected1] PASSED   [ 20%]
tests/tests.py::test_lazy_fixture_data_types[actual2-expected2] PASSED   [ 25%]
tests/tests.py::test_lazy_fixture_data_types[actual3-expected3] PASSED   [ 29%]
tests/tests.py::test_lazy_fixture_data_types[actual4-expected4] PASSED   [ 33%]
tests/tests.py::test_lazy_fixture_with_fixtures[four] PASSED             [ 37%]
tests/tests.py::test_lazy_fixture_with_attrs[entity-entity.value] PASSED [ 41%]
tests/tests.py::test_lazy_fixture_callable_with_func[format] PASSED      [ 45%]
tests/tests.py::test_lazy_fixture_callable_with_lf[entity_format] PASSED [ 50%]
tests/tests.py::test_lazy_fixture_callable_with_attr_lf[entity.sum] PASSED [ 54%]
tests/tests.py::test[data0] PASSED                                       [ 58%]
tests/tests.py::test_foo[service1-a] PASSED                              [ 62%]
tests/tests.py::test_foo[service1-b] PASSED                              [ 66%]
tests/tests.py::test_foo[fixture21-a] PASSED                             [ 70%]
tests/tests.py::test_foo[fixture21-b] PASSED                             [ 75%]
tests/tests.py::test_foo[fixture22-a] PASSED                             [ 79%]
tests/tests.py::test_foo[fixture22-b] PASSED                             [ 83%]
tests/tests.py::test_foo[fixture23-a] PASSED                             [ 87%]
tests/tests.py::test_foo[fixture23-b] PASSED                             [ 91%]
tests/tests.py::test_foo[fixture24-a] PASSED                             [ 95%]
tests/tests.py::test_foo[fixture24-b] PASSED                             [100%]

============================== 24 passed in 0.06s ==============================

@kloczek
Copy link
Author

kloczek commented Mar 13, 2024

OK so what could be wrong and/or how can I try diagnose this issue? 🤔

@mtelka
Copy link

mtelka commented Mar 14, 2024

OK so what could be wrong and/or how can I try diagnose this issue? 🤔

I'm sorry, but I do not know your environment so I've no idea what might be wrong on your side. Maybe the difference is that I run tests in clean environment and you probably don't.

I use the following command: /usr/bin/env -i PYTHONPATH=/data/builds/oi-userland/components/python/pytest-lazy-fixtures/build/prototype/i386/usr/lib/python3.9/vendor-packages /usr/bin/python3.9 -m pytest --verbose. There are also many -p no:XXX options to disable all other installed pytest plugins.

@mgorny
Copy link

mgorny commented Mar 14, 2024

Or export PYTEST_DISABLE_PLUGIN_AUTOLOAD=1.

@kloczek
Copy link
Author

kloczek commented Mar 14, 2024

I use the following command: /usr/bin/env -i PYTHONPATH=/data/builds/oi-userland/components/python/pytest-lazy-fixtures/build/prototype/i386/usr/lib/python3.9/vendor-packages /usr/bin/python3.9 -m pytest --verbose. There are also many -p no:XXX options to disable all other installed pytest plugins.

I'm not using any-p no:XXX options 😋

Or export PYTEST_DISABLE_PLUGIN_AUTOLOAD=1.

That output which I've provided is already with that env variable (as you already advised to use it)

Here is pytest output with -vv:
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-lazy-fixtures-1.0.6-2.fc36.x86_64/usr/lib64/python3.9/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-lazy-fixtures-1.0.6-2.fc36.x86_64/usr/lib/python3.9/site-packages
+ /usr/bin/pytest -ra -m 'not network' -vv
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.9.18, pytest-8.1.1, pluggy-1.4.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /home/tkloczko/rpmbuild/BUILD/pytest-lazy-fixtures-1.0.6
configfile: pyproject.toml
collected 19 items

tests/tests.py::test_lazy_fixture_repr PASSED                                                                                                                                         [  5%]
tests/tests.py::test_lazy_fixture_callable_repr PASSED                                                                                                                                [ 10%]
tests/tests.py::test_lazy_fixture[first0-second0-4] FAILED                                                                                                                            [ 15%]
tests/tests.py::test_lazy_fixture_data_types[actual0-expected0] FAILED                                                                                                                [ 21%]
tests/tests.py::test_lazy_fixture_data_types[actual1-expected1] FAILED                                                                                                                [ 26%]
tests/tests.py::test_lazy_fixture_data_types[actual2-expected2] FAILED                                                                                                                [ 31%]
tests/tests.py::test_lazy_fixture_data_types[actual3-expected3] FAILED                                                                                                                [ 36%]
tests/tests.py::test_lazy_fixture_data_types[actual4-expected4] FAILED                                                                                                                [ 42%]
tests/tests.py::test_lazy_fixture_with_fixtures[item0] FAILED                                                                                                                         [ 47%]
tests/tests.py::test_lazy_fixture_with_attrs[only_entity0-entity_value0] FAILED                                                                                                       [ 52%]
tests/tests.py::test_lazy_fixture_callable_with_func[message0] FAILED                                                                                                                 [ 57%]
tests/tests.py::test_lazy_fixture_callable_with_lf[formatted0] FAILED                                                                                                                 [ 63%]
tests/tests.py::test_lazy_fixture_callable_with_attr_lf[result0] FAILED                                                                                                               [ 68%]
tests/tests.py::test[data0] PASSED                                                                                                                                                    [ 73%]
tests/tests.py::test_foo[fixture20] PASSED                                                                                                                                            [ 78%]
tests/tests.py::test_foo[fixture21] PASSED                                                                                                                                            [ 84%]
tests/tests.py::test_foo[fixture22] PASSED                                                                                                                                            [ 89%]
tests/tests.py::test_foo[fixture23] PASSED                                                                                                                                            [ 94%]
tests/tests.py::test_foo[fixture24] PASSED                                                                                                                                            [100%]

========================================================================================= FAILURES ==========================================================================================
____________________________________________________________________________ test_lazy_fixture[first0-second0-4] ____________________________________________________________________________

first = one, second = two, other = 4

    @pytest.mark.parametrize(("first", "second", "other"), [(lf("one"), lf("two"), 4)])
    def test_lazy_fixture(first, second, other):
>       assert first == 1
E       assert one == 1

tests/tests.py:19: AssertionError
______________________________________________________________________ test_lazy_fixture_data_types[actual0-expected0] ______________________________________________________________________

actual = {'a': [one, two]}, expected = {'a': [1, 2]}

    @pytest.mark.parametrize(
        ("actual", "expected"),
        [
            ({"a": [lf("one"), lf("two")]}, {"a": [1, 2]}),
            ({"a": {lf("one"), lf("two")}}, {"a": {1, 2}}),
            ({lf("three"), lf("two"), lf("three")}, {3, 2}),
            ([lf("three"), lf("two")], [3, 2]),
            (
                {"first_level": {"second_level": (lf("one"), lf("one"), lf("two")), "not_lazy_fixture": 1}},
                {"first_level": {"second_level": (1, 1, 2), "not_lazy_fixture": 1}},
            ),
        ],
    )
    def test_lazy_fixture_data_types(actual, expected):
>       assert actual == expected
E       AssertionError: assert {'a': [one, two]} == {'a': [1, 2]}
E
E         Differing items:
E         {'a': [one, two]} != {'a': [1, 2]}
E
E         Full diff:
E           {
E               'a': [
E         -         1,
E         ?         ^
E         +         one,
E         ?         ^^^
E         -         2,
E         ?         ^
E         +         two,
E         ?         ^^^
E               ],
E           }

tests/tests.py:38: AssertionError
______________________________________________________________________ test_lazy_fixture_data_types[actual1-expected1] ______________________________________________________________________

actual = {'a': {one, two}}, expected = {'a': {1, 2}}

    @pytest.mark.parametrize(
        ("actual", "expected"),
        [
            ({"a": [lf("one"), lf("two")]}, {"a": [1, 2]}),
            ({"a": {lf("one"), lf("two")}}, {"a": {1, 2}}),
            ({lf("three"), lf("two"), lf("three")}, {3, 2}),
            ([lf("three"), lf("two")], [3, 2]),
            (
                {"first_level": {"second_level": (lf("one"), lf("one"), lf("two")), "not_lazy_fixture": 1}},
                {"first_level": {"second_level": (1, 1, 2), "not_lazy_fixture": 1}},
            ),
        ],
    )
    def test_lazy_fixture_data_types(actual, expected):
>       assert actual == expected
E       AssertionError: assert {'a': {one, two}} == {'a': {1, 2}}
E
E         Differing items:
E         {'a': {one, two}} != {'a': {1, 2}}
E
E         Full diff:
E           {
E               'a': {
E         -         1,
E         ?         ^
E         +         two,
E         ?         ^^^
E         -         2,
E         ?         ^
E         +         one,
E         ?         ^^^
E               },
E           }

tests/tests.py:38: AssertionError
______________________________________________________________________ test_lazy_fixture_data_types[actual2-expected2] ______________________________________________________________________

actual = {two, three}, expected = {2, 3}

    @pytest.mark.parametrize(
        ("actual", "expected"),
        [
            ({"a": [lf("one"), lf("two")]}, {"a": [1, 2]}),
            ({"a": {lf("one"), lf("two")}}, {"a": {1, 2}}),
            ({lf("three"), lf("two"), lf("three")}, {3, 2}),
            ([lf("three"), lf("two")], [3, 2]),
            (
                {"first_level": {"second_level": (lf("one"), lf("one"), lf("two")), "not_lazy_fixture": 1}},
                {"first_level": {"second_level": (1, 1, 2), "not_lazy_fixture": 1}},
            ),
        ],
    )
    def test_lazy_fixture_data_types(actual, expected):
>       assert actual == expected
E       assert {two, three} == {2, 3}
E
E         Extra items in the left set:
E         two
E         three
E         Extra items in the right set:
E         2
E         3
E
E         Full diff:
E           {
E         -     2,
E         -     3,
E         +     two,
E         +     three,
E           }

tests/tests.py:38: AssertionError
______________________________________________________________________ test_lazy_fixture_data_types[actual3-expected3] ______________________________________________________________________

actual = [three, two], expected = [3, 2]

    @pytest.mark.parametrize(
        ("actual", "expected"),
        [
            ({"a": [lf("one"), lf("two")]}, {"a": [1, 2]}),
            ({"a": {lf("one"), lf("two")}}, {"a": {1, 2}}),
            ({lf("three"), lf("two"), lf("three")}, {3, 2}),
            ([lf("three"), lf("two")], [3, 2]),
            (
                {"first_level": {"second_level": (lf("one"), lf("one"), lf("two")), "not_lazy_fixture": 1}},
                {"first_level": {"second_level": (1, 1, 2), "not_lazy_fixture": 1}},
            ),
        ],
    )
    def test_lazy_fixture_data_types(actual, expected):
>       assert actual == expected
E       assert [three, two] == [3, 2]
E
E         At index 0 diff: three != 3
E
E         Full diff:
E           [
E         -     3,
E         -     2,
E         +     three,
E         +     two,
E           ]

tests/tests.py:38: AssertionError
______________________________________________________________________ test_lazy_fixture_data_types[actual4-expected4] ______________________________________________________________________

actual = {'first_level': {'not_lazy_fixture': 1, 'second_level': (one, one, two)}}, expected = {'first_level': {'not_lazy_fixture': 1, 'second_level': (1, 1, 2)}}

    @pytest.mark.parametrize(
        ("actual", "expected"),
        [
            ({"a": [lf("one"), lf("two")]}, {"a": [1, 2]}),
            ({"a": {lf("one"), lf("two")}}, {"a": {1, 2}}),
            ({lf("three"), lf("two"), lf("three")}, {3, 2}),
            ([lf("three"), lf("two")], [3, 2]),
            (
                {"first_level": {"second_level": (lf("one"), lf("one"), lf("two")), "not_lazy_fixture": 1}},
                {"first_level": {"second_level": (1, 1, 2), "not_lazy_fixture": 1}},
            ),
        ],
    )
    def test_lazy_fixture_data_types(actual, expected):
>       assert actual == expected
E       AssertionError: assert {'first_level': {'second_level': (one, one, two), 'not_lazy_fixture': 1}} == {'first_level': {'second_level': (1, 1, 2), 'not_lazy_fixture': 1}}
E
E         Differing items:
E         {'first_level': {'not_lazy_fixture': 1, 'second_level': (one, one, two)}} != {'first_level': {'not_lazy_fixture': 1, 'second_level': (1, 1, 2)}}
E
E         Full diff:
E           {
E               'first_level': {
E                   'not_lazy_fixture': 1,
E                   'second_level': (
E         -             1,
E         ?             ^
E         +             one,
E         ?             ^^^
E         -             1,
E         ?             ^
E         +             one,
E         ?             ^^^
E         -             2,
E         ?             ^
E         +             two,
E         ?             ^^^
E                   ),
E               },
E           }

tests/tests.py:38: AssertionError
__________________________________________________________________________ test_lazy_fixture_with_fixtures[item0] ___________________________________________________________________________

item = four

    @pytest.mark.parametrize("item", [lf("four")])
    def test_lazy_fixture_with_fixtures(item):
>       assert item == 4
E       assert four == 4

tests/tests.py:43: AssertionError
_________________________________________________________________ test_lazy_fixture_with_attrs[only_entity0-entity_value0] __________________________________________________________________

only_entity = entity, entity_value = entity.value

    @pytest.mark.parametrize("only_entity,entity_value", [(lf("entity"), lf("entity.value"))])
    def test_lazy_fixture_with_attrs(only_entity, entity_value):
>       assert isinstance(only_entity, Entity)
E       assert False
E        +  where False = isinstance(entity, Entity)

tests/tests.py:48: AssertionError
______________________________________________________________________ test_lazy_fixture_callable_with_func[message0] _______________________________________________________________________

message = format

    @pytest.mark.parametrize(
        "message",
        [
            lfc(
                "There is two lazy fixture args, {} and {}! And one kwarg {field}! And also simple value {simple}".format,
                lf("one"),
                lf("two"),
                field=lf("three"),
                simple="value",
            ),
        ],
    )
    def test_lazy_fixture_callable_with_func(message):
>       assert message == "There is two lazy fixture args, 1 and 2! And one kwarg 3! And also simple value value"
E       AssertionError: assert format == 'There is two lazy fixture args, 1 and 2! And one kwarg 3! And also simple value value'

tests/tests.py:65: AssertionError
______________________________________________________________________ test_lazy_fixture_callable_with_lf[formatted0] _______________________________________________________________________

formatted = entity_format, entity = Entity(value=1)

    @pytest.mark.parametrize("formatted", [lfc("entity_format", lf("entity"))])
    def test_lazy_fixture_callable_with_lf(formatted, entity):
>       assert formatted == {"value": entity.value}
E       AssertionError: assert entity_format == {'value': 1}

tests/tests.py:70: AssertionError
_____________________________________________________________________ test_lazy_fixture_callable_with_attr_lf[result0] ______________________________________________________________________

result = entity.sum

    @pytest.mark.parametrize("result", [lfc("entity.sum", lf("two"))])
    def test_lazy_fixture_callable_with_attr_lf(result):
>       assert result == 3
E       assert entity.sum == 3

tests/tests.py:75: AssertionError
================================================================================== short test summary info ==================================================================================
FAILED tests/tests.py::test_lazy_fixture[first0-second0-4] - assert one == 1
FAILED tests/tests.py::test_lazy_fixture_data_types[actual0-expected0] - AssertionError: assert {'a': [one, two]} == {'a': [1, 2]}
FAILED tests/tests.py::test_lazy_fixture_data_types[actual1-expected1] - AssertionError: assert {'a': {one, two}} == {'a': {1, 2}}
FAILED tests/tests.py::test_lazy_fixture_data_types[actual2-expected2] - assert {two, three} == {2, 3}
FAILED tests/tests.py::test_lazy_fixture_data_types[actual3-expected3] - assert [three, two] == [3, 2]
FAILED tests/tests.py::test_lazy_fixture_data_types[actual4-expected4] - AssertionError: assert {'first_level': {'second_level': (one, one, two), 'not_lazy_fixture': 1}} == {'first_level': {'second_level': (1, 1, 2), 'not_lazy_fixture': 1}}
FAILED tests/tests.py::test_lazy_fixture_with_fixtures[item0] - assert four == 4
FAILED tests/tests.py::test_lazy_fixture_with_attrs[only_entity0-entity_value0] - assert False
FAILED tests/tests.py::test_lazy_fixture_callable_with_func[message0] - AssertionError: assert format == 'There is two lazy fixture args, 1 and 2! And one kwarg 3! And also simple value value'
FAILED tests/tests.py::test_lazy_fixture_callable_with_lf[formatted0] - AssertionError: assert entity_format == {'value': 1}
FAILED tests/tests.py::test_lazy_fixture_callable_with_attr_lf[result0] - assert entity.sum == 3
=============================================================================== 11 failed, 8 passed in 0.17s ================================================================================

@kloczek
Copy link
Author

kloczek commented Mar 14, 2024

So looks lik test suite expects numbers and in my case are returned these numbers converted to text 🤔

@mgorny
Copy link

mgorny commented Mar 14, 2024

Actually, in 1.0.6 you also have to set PYTEST_PLUGINS=pytest_lazy_fixtures.plugin.

@mtelka
Copy link

mtelka commented Mar 14, 2024

I use the following command: /usr/bin/env -i PYTHONPATH=/data/builds/oi-userland/components/python/pytest-lazy-fixtures/build/prototype/i386/usr/lib/python3.9/vendor-packages /usr/bin/python3.9 -m pytest --verbose. There are also many -p no:XXX options to disable all other installed pytest plugins.

I'm not using any-p no:XXX options 😋

The key point was /usr/bin/env -i, not -p no:XXX.

@mtelka
Copy link

mtelka commented Mar 14, 2024

Actually, in 1.0.6 you also have to set PYTEST_PLUGINS=pytest_lazy_fixtures.plugin.

No, you don't.

@mgorny
Copy link

mgorny commented Mar 14, 2024

I meant if he's using PYTEST_DISABLE_PLUGIN_AUTOLOAD.

@kloczek
Copy link
Author

kloczek commented Mar 15, 2024

I meant if he's using PYTEST_DISABLE_PLUGIN_AUTOLOAD.

Yep .. and that is what currently I have in package build procedure (for now I've added failing units to --deselect list)

%check
export PYTEST_DISABLE_PLUGIN_AUTOLOAD=1
%pytest %{!?with_failing_tests: \
        --deselect tests/tests.py::test_lazy_fixture[first0-second0-4] \
        --deselect tests/tests.py::test_lazy_fixture_callable_with_attr_lf[result0] \
        --deselect tests/tests.py::test_lazy_fixture_callable_with_func[message0] \
        --deselect tests/tests.py::test_lazy_fixture_callable_with_lf[formatted0] \
        --deselect tests/tests.py::test_lazy_fixture_data_types[actual0-expected0] \
        --deselect tests/tests.py::test_lazy_fixture_data_types[actual1-expected1] \
        --deselect tests/tests.py::test_lazy_fixture_data_types[actual2-expected2] \
        --deselect tests/tests.py::test_lazy_fixture_data_types[actual3-expected3] \
        --deselect tests/tests.py::test_lazy_fixture_data_types[actual4-expected4] \
        --deselect tests/tests.py::test_lazy_fixture_with_attrs[only_entity0-entity_value0] \
        --deselect tests/tests.py::test_lazy_fixture_with_fixtures[item0] \
}

@mgorny
Copy link

mgorny commented Mar 16, 2024

@kloczek, your problems are entirely of your own doing.

First it failed for you because the plugin was simultaneously being autoloaded by pytest and loaded explicitly via conftest.py. This was solvable via PYTEST_DISABLE_PLUGIN_AUTOLOAD=1. However, because of your bug report, explicit loading has been removed from conftest.py.

Now it's failing for you because it's not being loaded at all because you're disabling autoloading. On top of that, you're skipping all tests that actually test the plugin, which effectively makes your test run entirely meaningless.

Either remove PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 and rely on pytest autoloading, or use PYTEST_PLUGINS to load it explicitly.

@kloczek
Copy link
Author

kloczek commented Mar 16, 2024

@kloczek, your problems are entirely of your own doing.

First it failed for you because the plugin was simultaneously being autoloaded by pytest and loaded explicitly via conftest.py. This was solvable via PYTEST_DISABLE_PLUGIN_AUTOLOAD=1. However, because of your bug report, explicit loading has been removed from conftest.py.

Now it's failing for you because it's not being loaded at all because you're disabling autoloading. On top of that, you're skipping all tests that actually test the plugin, which effectively makes your test run entirely meaningless.

Either remove PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 and rely on pytest autoloading, or use PYTEST_PLUGINS to load it explicitly.

OK tested and it works now.
Thank you 👍

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-lazy-fixtures-1.0.6-2.fc36.x86_64/usr/lib64/python3.9/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-lazy-fixtures-1.0.6-2.fc36.x86_64/usr/lib/python3.9/site-packages
+ /usr/bin/pytest -ra -m 'not network'
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.9.18, pytest-8.1.1, pluggy-1.4.0
rootdir: /home/tkloczko/rpmbuild/BUILD/pytest-lazy-fixtures-1.0.6
configfile: pyproject.toml
plugins: lazy-fixtures-1.0.6, mock-3.12.0
collected 24 items

tests/tests.py ........................                                                                                                                                               [100%]

==================================================================================== 24 passed in 0.15s =====================================================================================

@kloczek kloczek changed the title 1.0.5: pytest fails 0.0.5: pytest fails Mar 16, 2024
@kloczek kloczek changed the title 0.0.5: pytest fails 0.5.0: pytest fails Mar 16, 2024
@kloczek kloczek changed the title 0.5.0: pytest fails 1.0.5: pytest fails Mar 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants