Plugin fixtures with leading underscore ignored outside project defining them

[kohr-h]

Under my version of pytest, which is 3.5.0, fixtures that are part of a setuptools-registered plugin (using the pytest11 entry point) whose names begin with an underscore, are being ignored outside the project that define them.

I find this pretty useful, since it allows me to implement fixtures that are global to my project but don't interfere with other projects' fixtures.
However, I don't find this documented anywhere. Is it intended behavior?

[pytestbot]

GitMate.io thinks possibly related issues are #177 (Test functions with leading underscores ignored), #2793 (Overwriting a fixture in a plugin), #1480 (py.test ignores exceptions in fixtures), #1082 (Duplicate fixtures in complex projects), and #519 (fixture scope is ignored when using metafunc.parametrize()).

[nicoddemus]

Hi @kohr-h, sorry for the delay.

Weird, I don't see anything about excluding fixtures prefixed with _ in the code related to make fixtures available to nodes:

pytest/_pytest/fixtures.py

Line 1093 in e012dbe

def parsefactories(self, node_or_obj, nodeid=NOTSET, unittest=False):

@RonnyPfannschmidt do you have any insights here?

[kohr-h]

Thanks for the answer @nicoddemus. Just to make sure that my description makes sense:

Let's say I have two source trees project1 and project2 that both use an entry point like this:

entry_points={'pytest11': ['projectX_plugins = projectX.util.pytest_plugins']}

where X is 1 or 2.

Now let's consider two scenarios:

1. project1 has a single global fixture my_fixture in its pytest_plugins module
2. project1 has a single global fixture _my_fixture in its pytest_plugins module.

Both projects have been installed with pip --editable. When going to the source tree of project2 and running pytest --fixtures we see the following:

1. A section project1.util.pytest_plugins is listed, and in this section my_fixture is shown.
2. A section project1.util.pytest_plugins is listed, but the section is empty.

So it seems like by default, fixtures with leading underscore are considered "foreign" or "internal" when coming from a plugin and are omitted.

As I write above, I find this pretty useful, and my question is whether I can rely on this as a feature.

[RonnyPfannschmidt]

no idea either ^^

[nicoddemus]

@kohr-h thanks for the detailed follow up, but unfortunately I have been unable to reproduce your problem. Here's what I did:

Created a dummy package which contains two fixtures and a setup.py:

# myplugin.py
import pytest

@pytest.fixture
def visible():
    return 1

@pytest.fixture
def _invisible():
    return 2

# setup.py
from setuptools import setup

def main():
    setup(
        name='myplugin',
        description='',
        long_description='',
        url='http://myplugin.org',
        license='MIT license',
        platforms=['unix', 'linux', 'osx', 'cygwin', 'win32'],
        author='someone',
        entry_points={'pytest11': ['myplugin = myplugin']},
        py_modules=['myplugin'],
        zip_safe=False,
    )

if __name__ == '__main__':
    main()

I installed this package into a virtual environment in develop mode and this simple test can see both fixtures normally:

def test_a(visible):
    assert visible == 1

def test_b(_invisible):
    assert _invisible == 2

Output:

(.env36) λ pytest . -v
============================= test session starts =============================
platform win32 -- Python 3.6.3, pytest-3.5.1.dev40+g37ee4fbc, py-1.5.2, pluggy-0.6.1.dev -- c:\pytest\.env36\scripts\python.exe
cachedir: .pytest_cache
rootdir: c:\pytest\.tmp\underscore\app, inifile: pytest.ini
plugins: xdist-1.22.2, forked-0.2, hypothesis-3.38.9, myplugin-0.0.0
collected 2 items

test_foo.py::test_a PASSED                                               [ 50%]
test_foo.py::test_b PASSED                                               [100%]

========================== 2 passed in 0.02 seconds ===========================

We can see that myplugin was loaded as expected and the tests could access the fixture.

Can you create a small reproducible example that shows your problem? It might be related to something else or some detail I'm missing. Thanks!

[kohr-h]

I was just cooking up a minimal example to see the tests failing with two repos and two separate plugins and tests that involve the respective other fixtures.
Only... they didn't.

However, if you take your setup and run pytest --fixtures you'll see that _invisible doesn't show up. So contrary to what I thought initially, the fixtures are accessible, but not all visible when listing them.

So the real issue is in the pytest --fixtures command omitting entries with leading underscore from the list (which I took at face value).

[nicoddemus]

Oh OK, that explains it. 😅

Here's the code responsible for that:

pytest/_pytest/python.py

Lines 1069 to 1070 in 0024b71

	if verbose <= 0 and argname[0] == "_":
	continue

So fixtures with leading underscores will show up if you pass -v to pytest.

The commit at that line is 20849a4 and reads:

improve --fixtures output with per-plugin grouping and hiding underscore names in non-verbose mode, re-introduce --funcargs for compatibiliy

That change seems to have been introduced in 2.3.0 based on the tags, but I can't find anything related to that in the changelog entries or the issue tracker.

@RonnyPfannschmidt does this ring a bell to you?

[RonnyPfannschmidt]

nope

[nicoddemus]

I guess then we can remove that special handling if we can't pinpoint the reason. I've opened #3398 to track this and closing this one.

Thanks again @kohr-h for bringing this up.

[kohr-h]

Thanks for the help @nicoddemus.