new pytest runner mixes test classes

[lmgarret]

Type: Bug

Behaviour

Expected vs. Actual

I have enabled the experiment pythonTestAdapter to use the new test runner. If two test files have identically named test classes, their test methods are mixed

Steps to reproduce:

1. Create these two files:

# tests/test_a.py

class TestSomething:
    def test_a(self):
        assert True

# tests/test_b.py

class TestSomething:
    def test_b(self):
        assert True

2. Refresh the test suites
3. You should get the following test structure:

test_a.py
    > TestSomething
        > test_a
        > test_b
test_b.py
    > TestSomething
        > test_a
        > test_b

instead of the expected

test_a.py
    > TestSomething
        > test_a
test_b.py
    > TestSomething
        > test_b

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.9.2
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Venv
• Value of the python.languageServer setting: Default

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

XXX

User Settings

Multiroot scenario, following user settings may not apply:

languageServer: "Pylance"

formatting
• provider: "yapf"

experiments
• optInto: ["pythonTestAdapter"]

Extension version: 2023.12.0
VS Code version: Code 1.80.0 (660393deaaa6d1996740ff4880f1bad43768c814, 2023-07-04T13:39:48.236Z)
OS version: Linux x64 5.15.0-75-generic
Modes:
Remote OS version: Linux x64 5.15.0-75-generic

A/B Experiments

vsliv368:30146709
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
vserr242cf:30382550
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593cf:30376535
pythonvs932:30410667
py29gd2263:30776702
vscaat:30438848
vsclangdc:30486549
c4g48928:30535728
dsvsc012:30540252
pynewext54:30695312
azure-dev_surveyone:30548225
vsccc:30610678
2e4cg342:30602488
pyind779:30671433
89544117:30613380
pythonsymbol12:30671437
2i9eh265:30646982
showlangstatbar:30737416
vsctsb:30748421
pythonms35:30701012
03d35959:30757346
pythonfmttext:30731395
pythoncmvfstrcf:30756944
fixshowwlkth:30771522
hideindicator:30766889
pythongtdpath:30769146
bgfeh915:30780428
pythonnosmt12:30779714
pythonidxpt:30784022
pythonnoceb:30776495
e537b577:30772215

[eleanorjboyd]

Hello! Thank you for trying out the new rewrite and finding this bug! We appreciate users like you who help make our new features bug-free as we roll them out! I am able to repro this issue and will fix it now. Thanks!

[lmgarret]

I just tested it, and it seems fixed to me, thanks for the quick action! Can't wait to use this new runner as my daily driver :)