fixture finalizer dependency incorrect when using autouse or getfuncargvalue

[amosonn]

Calling fixture finalizers in the correct order works by dependent fixtures registering their finalizer in their dependency (see python.FixtureDef.execute+10).
However, this only happens for fixtures explicitly requested in args, not for autouse or getfuncargvalue fixtures.
This code:

import pytest

@pytest.fixture(scope="session")
def global_dict():
    return {}

@pytest.fixture(scope="session", autouse=True)
def add_1(global_dict):
    global_dict[1] = "a"
    print("add_1 setup", global_dict)
    yield global_dict
    print("add_1 teardown", global_dict)
    del global_dict[1]

@pytest.fixture(scope="session")
def change_1_to_2(global_dict):
    global_dict[2] = global_dict.pop(1)
    print("change_1_to_2 setup", global_dict)
    yield global_dict
    print("change_1_to_2 teardown", global_dict)
    global_dict[1] = global_dict.pop(2)

def test1(change_1_to_2):
    assert change_1_to_2[2] == "a"

def test2(global_dict):
    assert global_dict[2] == "a"

gives with py.test --capture=no test.py:

============================= test session starts ==============================
platform linux2 -- Python 2.7.11, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/amos/git/test_py, inifile: 
collected 2 items 

test.py add_1 setup {1: 'a'}
change_1_to_2 setup {2: 'a'}
..add_1 teardown {2: 'a'}
change_1_to_2 teardown {2: 'a'}
E

==================================== ERRORS ====================================
__________________________ ERROR at teardown of test2 __________________________

global_dict = {1: 'a'}

    @pytest.yield_fixture(scope="session", autouse=True)
    def add_1(global_dict):
        global_dict[1] = "a"
        print "add_1 setup", global_dict
        yield global_dict
        print "add_1 teardown", global_dict
>       del global_dict[1]
E       KeyError: 1

test.py:14: KeyError
====================== 2 passed, 1 error in 0.01 seconds =======================

as does this code:

import pytest

@pytest.fixture(scope="session")
def global_dict():
    return {}

@pytest.yield_fixture(scope="session")
def add_1(global_dict):
    global_dict[1] = "a"
    print "add_1 setup", global_dict
    yield global_dict
    print "add_1 teardown", global_dict
    del global_dict[1]

@pytest.yield_fixture(scope="session")
def change_1_to_2(request):
    add_1 = request.getfuncargvalue("add_1")
    add_1[2] = add_1.pop(1)
    print "change_1_to_2 setup", add_1
    yield add_1
    print "change_1_to_2 teardown", add_1
    add_1[1] = add_1.pop(2)

def test1(change_1_to_2):
    assert change_1_to_2[2] == "a"

def test2(global_dict):
    assert global_dict[2] == "a"

[nicoddemus]

Hi @amosonn,

Thanks for taking the time to write down this report.

(I edited your post to add syntax highlight to your examples to make them easier to read, hope you don't mind).

[pchanial]

I thought I could use autouse fixtures to do some setup and teardown, but since the teardown part is not garanteed to come after the teardown of non-autouse fixtures, it's not straightforward. Here is a more compact and modern test case that shows the problem (the scope could also be set to package or session):

import pytest

@pytest.fixture(scope='module', autouse=True)
def fixture_autouse():
    print('\nSETUP FIXTURE AUTOUSE')
    yield
    print('\nTEARDOWN FIXTURE AUTOUSE')

@pytest.fixture(scope='module')
def fixture_test():
    print('\nSETUP FIXTURE TEST')
    yield
    print('\nTEARDOWN FIXTURE TEST')

def test_1(fixture_test):
    pass

def test_2():
    pass

running the test gives:

============================== 2 passed in 0.00s ===============================
(venv) (base) 22) lilla:~/work/bug_pytest> pytest -s tests/
============================= test session starts ==============================
platform linux -- Python 3.8.3, pytest-6.3.0.dev454+gb8f1b7cdc, py-1.10.0, pluggy-0.13.1
rootdir: /home/chanial/work/bug_pytest
collected 2 items                                                              

tests/test_fixture.py 
SETUP FIXTURE AUTOUSE

SETUP FIXTURE TEST
..
TEARDOWN FIXTURE AUTOUSE

TEARDOWN FIXTURE TEST


============================== 2 passed in 0.01s ===============================

which shows that the autouse fixture teardown does not come after that of the non-autouse fixture. Since the autouse fixture setup comes first wrt non-autouse setups, it is expected that their teardowns come last.

[jordanlibrande]

+1 ran into this bug. This behavior is different from the documented expected behavior that finalizers will run in reverse order to the order the fixtures were initialized. From https://docs.pytest.org/en/6.2.x/fixture.html#yield-fixtures-recommended :

Once the test is finished, pytest will go back down the list of fixtures, but in the reverse order, taking each one that yielded, and running the code inside it that was after the yield statement.

Perhaps adding a caveat to the documentation would be good until this is fixed?

[jakkdl]

Started looking into this, and it's actually not autouse that is the problem. We can modify pchanial's example to not have autouse and get the same behaviour:

import pytest

@pytest.fixture(scope='module')
def fixture_1():
    print('\nSETUP FIXTURE 1')
    yield
    print('\nTEARDOWN FIXTURE 1')

@pytest.fixture(scope='module')
def fixture_2():
    print('\nSETUP FIXTURE 2')
    yield
    print('\nTEARDOWN FIXTURE 2')

def test_1(fixture_1, fixture_2):
    pass

def test_2(fixture_1):
    pass

SETUP FIXTURE 1

SETUP FIXTURE 2
..
TEARDOWN FIXTURE 1

TEARDOWN FIXTURE 2

if the test order is reversed, or test_2 is removed, it does not occur. What seems to be happening is that the teardown for fixtures are queued each time they're called, so we get the following finalizer stack:

[<bound method Node.teardown of <Module mytest2.py>>,
 functools.partial(<bound method FixtureDef.finish of <FixtureDef argname='fixture_1' scope='module' baseid='testing/python/mytest2.py'>>, request=<SubRequest 'fixture_1' for <Function test_1>>),
 functools.partial(<bound method FixtureDef.finish of <FixtureDef argname='fixture_2' scope='module' baseid='testing/python/mytest2.py'>>, request=<SubRequest 'fixture_2' for <Function test_1>>),
 functools.partial(<bound method FixtureDef.finish of <FixtureDef argname='fixture_1' scope='module' baseid='testing/python/mytest2.py'>>, request=<SubRequest 'fixture_1' for <Function test_2>>)]

it processes it from the end, tearing down fixture_1, then fixture_2, and then fixture_1 again - but FixtureDef.finish ignores the second teardown of fixture_1 since it's already been finalized.

My current best-guess on how to fix it is to add some logic to runner.py:SetupState.addfinalizer so we only add a finalizer for a given fixture once.
edit: probably better to add it to fixtures.py:FixtureRequest._compute_fixture_value