Package fixtures teardown not being invoked after package scope

[jvaesteves]

Hello, I'm currently trying to separate some package fixtures from some local conftest.py files. I know that this is not expected behaviour, but it actually works for my team setup. It look something something like this:

Python version: 3.11.3
Pytest version: 7.3.1

Project structure:

tests/__init__.py
tests/test_1/conftest.py
tests/test_1/__init__.py
tests/test_1/foo/__init__.py
tests/test_1/foo/test_foo.py
tests/test_1/fixtures/__init__.py
tests/test_1/fixtures/fix.py
tests/test_2/conftest.py
tests/test_2/fixtures/__init__.py
tests/test_2/fixtures/fix.py
tests/test_2/bar/__init__.py
tests/test_2/bar/test_bar.py

tests/test_1/fixtures/fix.py and tests/test_2/fixtures/fix.py

@pytest.fixture(scope='package')
def foo():
    # consider 'bar' for test_2/fixtures
    print('foo start')
    yield
    print('foo end')

tests/test_1/conftest.py and tests/test_2/conftest.py

from tests.test_1.fixtures.fix import foo
# consider 'bar' for test_1/conftest.py

tests/test_1/foo/test_foo.py and tests/test_2/bar/test_bar.py

def test_foo(foo):
    # consider 'bar' for tests/test_2/bar/test_bar.py
    print(foo)

When calling the fixtures in our tests with this modularisation, the fixture remains only available on the conftest.py level, and the startup of the fixture is also invoked by its first occurrence, but its teardown is deferred by beyond the end of the related package. Something like this:

test_1/foo/test_foo.py foo start
None
.
test_2/bar/test_bar.py bar start
None
.bar end
foo end

Is there any reason why this is happening and how can I enforce the package scope?

[jvaesteves]

To add more details: I'm using fixtures to mock some datasets as Spark tables. What I do is to write the data in a temporary directory, saving it as a table in the Spark metastore in the startup and dropping it during the teardown.

But due to this behaviour of package fixtures, I get an error when invoking a different fixture pointing to the same table on another test as the table is not being dropped before its recreation.

I also just tried to create a local conftest.py with all needed fixtures (instead of importing them) for different packages to double check if the error actually happens because of the modularisation, but I still get a "TableAlreadyExists" exception indicating that the fixture teardown is not being invoked before the table is recreated.

[RonnyPfannschmidt]

This is most likely related to the way package scope fixtures where initially created

It's not well defined what package they belong to

It's also unclear if a easy way to fix it exists

[bluetech]

We have made a lot of changes to Package support in #11646. Please try pip install pytest==8.0.0rc1 and let us know if things still don't work as you expect.