Tests raising an exception are not marked as run

[Melebius]

Type: Bug

Behaviour

Expected vs. Actual

Expected

A test that raises an exception is marked as such and the exception traceback is provided.

Actual

A test that raises an exception is marked as not run. If it passed previously, it remains marked as passed.

Steps to reproduce:

1. Write and run:

• a test raising an exception
• a passing test

2. Change the passing test to raise an exception.
3. Run them.

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.10.6
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Global
• Value of the python.languageServer setting: Default

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2023-07-13 12:51:02.713 [info] Experiment 'pythonPromptNewFormatterExt' is active
2023-07-13 12:51:02.713 [info] Experiment 'pythonPromptNewToolsExt' is active
2023-07-13 12:51:02.713 [info] Experiment 'pythonTestAdapter' is active
2023-07-13 12:51:03.093 [info] Test server listening.
2023-07-13 12:51:03.097 [info] Python interpreter path: /bin/python
2023-07-13 12:51:06.128 [info] Discover tests for workspace name: experimental - uri: /home/mmateju/sw/experimental
2023-07-13 12:51:06.129 [info] Running discovery for unittest using the new test adapter.
2023-07-13 12:51:07.196 [info] Running unittests with arguments: /home/mmateju/.vscode-insiders/extensions/ms-python.python-2023.13.11931008/pythonFiles/unittestadapter/discovery.py,--port,40815,--uuid,8255c0be-7c9b-4773-9ce1-c7c9d41ce9d8,--udiscovery,-v,-s,.,-p,test_*.py

2023-07-13 12:51:07.196 [info] > /bin/python ~/.vscode-insiders/extensions/ms-python.python-2023.13.11931008/pythonFiles/unittestadapter/discovery.py --port 40815 --uuid 8255c0be-7c9b-4773-9ce1-c7c9d41ce9d8 --udiscovery -v -s . -p test_*.py
2023-07-13 12:51:07.196 [info] cwd: .
2023-07-13 12:51:07.830 [info] Test server connected to a client.
2023-07-13 12:51:07.850 [info] Using result resolver for discovery
2023-07-13 12:51:12.323 [info] Starting Pylance language server.
2023-07-13 13:17:24.361 [info] Discover tests for workspace name: experimental - uri: /home/mmateju/sw/experimental/test_fail.py
2023-07-13 13:17:24.361 [info] Running discovery for unittest using the new test adapter.
2023-07-13 13:17:24.376 [info] Running unittests with arguments: /home/mmateju/.vscode-insiders/extensions/ms-python.python-2023.13.11931008/pythonFiles/unittestadapter/discovery.py,--port,40815,--uuid,a77ebc5d-79ea-468c-b66d-66dcaa86f3e9,--udiscovery,-v,-s,.,-p,test_*.py

2023-07-13 13:17:24.376 [info] > /bin/python ~/.vscode-insiders/extensions/ms-python.python-2023.13.11931008/pythonFiles/unittestadapter/discovery.py --port 40815 --uuid a77ebc5d-79ea-468c-b66d-66dcaa86f3e9 --udiscovery -v -s . -p test_*.py
2023-07-13 13:17:24.376 [info] cwd: .
2023-07-13 13:17:24.426 [info] Test server connected to a client.
2023-07-13 13:17:24.428 [info] Using result resolver for discovery
2023-07-13 13:18:04.619 [info] Discover tests for workspace name: experimental - uri: /home/mmateju/sw/experimental/test_fail.py
2023-07-13 13:18:04.619 [info] Running discovery for unittest using the new test adapter.
2023-07-13 13:18:04.632 [info] Running unittests with arguments: /home/mmateju/.vscode-insiders/extensions/ms-python.python-2023.13.11931008/pythonFiles/unittestadapter/discovery.py,--port,40815,--uuid,fb32160a-fc5b-4f84-932d-903efe41bfa9,--udiscovery,-v,-s,.,-p,test_*.py

2023-07-13 13:18:04.632 [info] > /bin/python ~/.vscode-insiders/extensions/ms-python.python-2023.13.11931008/pythonFiles/unittestadapter/discovery.py --port 40815 --uuid fb32160a-fc5b-4f84-932d-903efe41bfa9 --udiscovery -v -s . -p test_*.py
2023-07-13 13:18:04.632 [info] cwd: .
2023-07-13 13:18:04.672 [info] Test server connected to a client.
2023-07-13 13:18:04.674 [info] Using result resolver for discovery
2023-07-13 13:18:15.727 [info] Discover tests for workspace name: experimental - uri: /home/mmateju/sw/experimental/test_fail.py
2023-07-13 13:18:15.728 [info] Running discovery for unittest using the new test adapter.
2023-07-13 13:18:15.739 [info] Running unittests with arguments: /home/mmateju/.vscode-insiders/extensions/ms-python.python-2023.13.11931008/pythonFiles/unittestadapter/discovery.py,--port,40815,--uuid,8b642c2d-be3d-4e0f-96f4-1101d74e498f,--udiscovery,-v,-s,.,-p,test_*.py

2023-07-13 13:18:15.739 [info] > /bin/python ~/.vscode-insiders/extensions/ms-python.python-2023.13.11931008/pythonFiles/unittestadapter/discovery.py --port 40815 --uuid 8b642c2d-be3d-4e0f-96f4-1101d74e498f --udiscovery -v -s . -p test_*.py
2023-07-13 13:18:15.739 [info] cwd: .
2023-07-13 13:18:15.784 [info] Test server connected to a client.
2023-07-13 13:18:15.786 [info] Using result resolver for discovery
2023-07-13 13:18:35.529 [info] Running UNITTEST execution for the following test ids: test_fail.FailTest.test_exception,test_fail.FailTest.test_fail,test_fail.FailTest.test_pass,test_fail.FailTest.test_pass_or_exception
2023-07-13 13:18:35.529 [info] Server listening on port 42759
2023-07-13 13:18:35.555 [info] Discovering unittest tests with arguments: /home/mmateju/.vscode-insiders/extensions/ms-python.python-2023.13.11931008/pythonFiles/unittestadapter/execution.py,--port,40815,--uuid,e3ef58ab-3a13-4f3d-829f-d1d886e91302,--udiscovery,-v,-s,.,-p,test_*.py

2023-07-13 13:18:35.555 [info] > /bin/python ~/.vscode-insiders/extensions/ms-python.python-2023.13.11931008/pythonFiles/unittestadapter/execution.py --port 40815 --uuid e3ef58ab-3a13-4f3d-829f-d1d886e91302 --udiscovery -v -s . -p test_*.py
2023-07-13 13:18:35.555 [info] cwd: .
2023-07-13 13:18:35.606 [info] Test server connected to a client.
2023-07-13 13:18:35.606 [info] Test server connected to a client.
2023-07-13 13:18:35.606 [info] Test server connected to a client.
2023-07-13 13:18:35.606 [info] Test server connected to a client.
2023-07-13 13:18:35.609 [info] Client disconnected
2023-07-13 13:19:22.489 [info] Experiment 'pythonPromptNewFormatterExt' is active
2023-07-13 13:19:22.489 [info] Experiment 'pythonPromptNewToolsExt' is active
2023-07-13 13:19:22.489 [info] Experiment 'pythonTestAdapter' is active
2023-07-13 13:19:22.537 [info] Test server listening.
2023-07-13 13:19:22.541 [info] Python interpreter path: /bin/python
2023-07-13 13:19:24.157 [info] Starting Pylance language server.
2023-07-13 13:19:29.476 [info] Discover tests for workspace name: experimental - uri: /home/mmateju/sw/experimental
2023-07-13 13:19:29.476 [info] Running discovery for unittest using the new test adapter.
2023-07-13 13:19:29.501 [info] Running unittests with arguments: /home/mmateju/.vscode-insiders/extensions/ms-python.python-2023.13.11941005/pythonFiles/unittestadapter/discovery.py,--port,46721,--uuid,6f297b1c-398b-4902-9447-bbc0c63a9cf2,--udiscovery,-v,-s,.,-p,test_*.py

2023-07-13 13:19:29.501 [info] > /bin/python ~/.vscode-insiders/extensions/ms-python.python-2023.13.11941005/pythonFiles/unittestadapter/discovery.py --port 46721 --uuid 6f297b1c-398b-4902-9447-bbc0c63a9cf2 --udiscovery -v -s . -p test_*.py
2023-07-13 13:19:29.501 [info] cwd: .
2023-07-13 13:19:29.554 [info] Test server connected to a client.
2023-07-13 13:19:29.556 [info] Using result resolver for discovery
2023-07-13 14:01:20.744 [info] Discover tests for workspace name: experimental - uri: /home/mmateju/sw/experimental/test_fail.py
2023-07-13 14:01:20.744 [info] Running discovery for unittest using the new test adapter.
2023-07-13 14:01:20.773 [info] Running unittests with arguments: /home/mmateju/.vscode-insiders/extensions/ms-python.python-2023.13.11941005/pythonFiles/unittestadapter/discovery.py,--port,46721,--uuid,e1b2c280-a7c7-4245-82a3-3cab8e779f14,--udiscovery,-v,-s,.,-p,test_*.py

2023-07-13 14:01:20.773 [info] > /bin/python ~/.vscode-insiders/extensions/ms-python.python-2023.13.11941005/pythonFiles/unittestadapter/discovery.py --port 46721 --uuid e1b2c280-a7c7-4245-82a3-3cab8e779f14 --udiscovery -v -s . -p test_*.py
2023-07-13 14:01:20.773 [info] cwd: .
2023-07-13 14:01:20.826 [info] Test server connected to a client.
2023-07-13 14:01:20.829 [info] Using result resolver for discovery
2023-07-13 14:01:39.785 [info] Running UNITTEST execution for the following test ids: test_fail.FailTest.test_exception,test_fail.FailTest.test_fail,test_fail.FailTest.test_pass,test_fail.FailTest.test_pass_or_exception
2023-07-13 14:01:39.785 [info] Server listening on port 40195
2023-07-13 14:01:39.821 [info] Discovering unittest tests with arguments: /home/mmateju/.vscode-insiders/extensions/ms-python.python-2023.13.11941005/pythonFiles/unittestadapter/execution.py,--port,46721,--uuid,130d2fa7-ae0a-49d1-b8f4-1dc597671225,--udiscovery,-v,-s,.,-p,test_*.py

2023-07-13 14:01:39.821 [info] > /bin/python ~/.vscode-insiders/extensions/ms-python.python-2023.13.11941005/pythonFiles/unittestadapter/execution.py --port 46721 --uuid 130d2fa7-ae0a-49d1-b8f4-1dc597671225 --udiscovery -v -s . -p test_*.py
2023-07-13 14:01:39.821 [info] cwd: .
2023-07-13 14:01:39.891 [info] Test server connected to a client.
2023-07-13 14:01:39.893 [info] Test server connected to a client.
2023-07-13 14:01:39.893 [info] Test server connected to a client.
2023-07-13 14:01:39.893 [info] Test server connected to a client.
2023-07-13 14:01:39.900 [info] Client disconnected
2023-07-13 14:01:49.515 [info] Running UNITTEST execution for the following test ids: test_fail.FailTest.test_exception
2023-07-13 14:01:49.515 [info] Server listening on port 36117
2023-07-13 14:01:49.586 [info] Discovering unittest tests with arguments: /home/mmateju/.vscode-insiders/extensions/ms-python.python-2023.13.11941005/pythonFiles/unittestadapter/execution.py,--port,46721,--uuid,763bf721-36fc-4b87-a60e-dff3a9789ae0,--udiscovery,-v,-s,.,-p,test_*.py

2023-07-13 14:01:49.586 [info] > /bin/python ~/.vscode-insiders/extensions/ms-python.python-2023.13.11941005/pythonFiles/unittestadapter/execution.py --port 46721 --uuid 763bf721-36fc-4b87-a60e-dff3a9789ae0 --udiscovery -v -s . -p test_*.py
2023-07-13 14:01:49.586 [info] cwd: .
2023-07-13 14:01:49.669 [info] Test server connected to a client.
2023-07-13 14:01:49.677 [info] Client disconnected
2023-07-13 14:01:51.684 [info] Running UNITTEST execution for the following test ids: test_fail.FailTest.test_exception
2023-07-13 14:01:51.685 [info] Server listening on port 38647
2023-07-13 14:01:51.759 [info] Discovering unittest tests with arguments: /home/mmateju/.vscode-insiders/extensions/ms-python.python-2023.13.11941005/pythonFiles/unittestadapter/execution.py,--port,46721,--uuid,1a12823f-5dcb-4df8-b809-0312085ab96f,--udiscovery,-v,-s,.,-p,test_*.py

2023-07-13 14:01:51.759 [info] > /bin/python ~/.vscode-insiders/extensions/ms-python.python-2023.13.11941005/pythonFiles/unittestadapter/execution.py --port 46721 --uuid 1a12823f-5dcb-4df8-b809-0312085ab96f --udiscovery -v -s . -p test_*.py
2023-07-13 14:01:51.759 [info] cwd: .
2023-07-13 14:01:51.819 [info] Test server connected to a client.
2023-07-13 14:01:51.824 [info] Client disconnected
2023-07-13 14:01:58.604 [info] Discover tests for workspace name: experimental - uri: /home/mmateju/sw/experimental/test_fail.py
2023-07-13 14:01:58.604 [info] Running discovery for unittest using the new test adapter.
2023-07-13 14:01:58.656 [info] Running unittests with arguments: /home/mmateju/.vscode-insiders/extensions/ms-python.python-2023.13.11941005/pythonFiles/unittestadapter/discovery.py,--port,46721,--uuid,0e25b31b-1088-49a9-a321-0ff166b7d3f7,--udiscovery,-v,-s,.,-p,test_*.py

2023-07-13 14:01:58.656 [info] > /bin/python ~/.vscode-insiders/extensions/ms-python.python-2023.13.11941005/pythonFiles/unittestadapter/discovery.py --port 46721 --uuid 0e25b31b-1088-49a9-a321-0ff166b7d3f7 --udiscovery -v -s . -p test_*.py
2023-07-13 14:01:58.656 [info] cwd: .
2023-07-13 14:01:58.739 [info] Test server connected to a client.
2023-07-13 14:01:58.741 [info] Using result resolver for discovery
2023-07-13 14:02:04.542 [info] Running UNITTEST execution for the following test ids: test_fail.FailTest.test_pass_or_exception
2023-07-13 14:02:04.542 [info] Server listening on port 42085
2023-07-13 14:02:04.570 [info] Discovering unittest tests with arguments: /home/mmateju/.vscode-insiders/extensions/ms-python.python-2023.13.11941005/pythonFiles/unittestadapter/execution.py,--port,46721,--uuid,3beeb8ce-599e-4736-b350-6e45bb4f47f8,--udiscovery,-v,-s,.,-p,test_*.py

2023-07-13 14:02:04.570 [info] > /bin/python ~/.vscode-insiders/extensions/ms-python.python-2023.13.11941005/pythonFiles/unittestadapter/execution.py --port 46721 --uuid 3beeb8ce-599e-4736-b350-6e45bb4f47f8 --udiscovery -v -s . -p test_*.py
2023-07-13 14:02:04.570 [info] cwd: .
2023-07-13 14:02:04.659 [info] Test server connected to a client.
2023-07-13 14:02:04.667 [info] Client disconnected
2023-07-13 14:02:07.140 [info] Running UNITTEST execution for the following test ids: test_fail.FailTest.test_pass_or_exception
2023-07-13 14:02:07.140 [info] Server listening on port 38009
2023-07-13 14:02:07.209 [info] Discovering unittest tests with arguments: /home/mmateju/.vscode-insiders/extensions/ms-python.python-2023.13.11941005/pythonFiles/unittestadapter/execution.py,--port,46721,--uuid,a8329780-8d95-4ba7-be49-554a1a7c1944,--udiscovery,-v,-s,.,-p,test_*.py

2023-07-13 14:02:07.209 [info] > /bin/python ~/.vscode-insiders/extensions/ms-python.python-2023.13.11941005/pythonFiles/unittestadapter/execution.py --port 46721 --uuid a8329780-8d95-4ba7-be49-554a1a7c1944 --udiscovery -v -s . -p test_*.py
2023-07-13 14:02:07.209 [info] cwd: .
2023-07-13 14:02:07.299 [info] Test server connected to a client.
2023-07-13 14:02:07.310 [info] Client disconnected
2023-07-13 14:04:13.123 [info] Discover tests for workspace name: experimental - uri: /home/mmateju/sw/experimental/test_fail.py
2023-07-13 14:04:13.123 [info] Running discovery for unittest using the new test adapter.
2023-07-13 14:04:13.137 [info] Running unittests with arguments: /home/mmateju/.vscode-insiders/extensions/ms-python.python-2023.13.11941005/pythonFiles/unittestadapter/discovery.py,--port,46721,--uuid,37585a9e-4a60-4896-856b-5289b4b09bbf,--udiscovery,-v,-s,.,-p,test_*.py

2023-07-13 14:04:13.137 [info] > /bin/python ~/.vscode-insiders/extensions/ms-python.python-2023.13.11941005/pythonFiles/unittestadapter/discovery.py --port 46721 --uuid 37585a9e-4a60-4896-856b-5289b4b09bbf --udiscovery -v -s . -p test_*.py
2023-07-13 14:04:13.137 [info] cwd: .
2023-07-13 14:04:13.217 [info] Test server connected to a client.
2023-07-13 14:04:13.220 [info] Using result resolver for discovery

User Settings

languageServer: "Pylance"

testing
• unittestArgs: "<placeholder>"
• unittestEnabled: true

Extension version: 2023.13.11941005
VS Code version: Code - Insiders 1.81.0-insider (d568855, 2023-07-12T11:46:48.659Z)
OS version: Linux x64 5.19.0-46-generic snap
Modes:

System Info

Item	Value
CPUs	11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz (8 x 3079)
GPU Status	2d_canvas: unavailable_software canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: disabled_software multiple_raster_threads: enabled_on opengl: disabled_off rasterization: disabled_software raw_draw: disabled_off_ok video_decode: disabled_software video_encode: disabled_software vulkan: disabled_off webgl: unavailable_software webgl2: unavailable_software webgpu: disabled_off
Load (avg)	0, 0, 1
Memory (System)	31.07GB (19.85GB free)
Process Argv	--no-sandbox --force-user-env --unity-launch --crash-reporter-id c795f6f0-a82e-48cd-a020-40eff6fabf0c
Screen Reader	no
VM	0%
DESKTOP_SESSION	ubuntu
XDG_CURRENT_DESKTOP	Unity
XDG_SESSION_DESKTOP	ubuntu
XDG_SESSION_TYPE	x11

A/B Experiments

vsliv695:30137379
vsins829:30139715
vsliv368:30146709
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
pythontb:30258533
vsc_aa:30263845
pythonptprofiler:30281269
vshan820:30294714
vscod805:30301674
bridge0708:30335490
bridge0723:30353136
vsaa593:30376534
pythonvs932:30404738
py29gd2263:30784851
vscaac:30438845
vsclangdf:30492506
c4g48928:30535728
dsvsc012cf:30540253
pynewext54:30618038
pylantcb52:30590116
pyind779:30611226
pythonsymbol12:30651887
showlangstatbar:30737417
pythonms35:30671666
24365598:30687740
pythonfmttext:30716741
pythoncmvfstr:30726892
fixshowwlkth:30771523
showindicator:30766888
pythongtdpath:30726887
i26e3531:30780429
gsofa:30778558
pythonnosmt12:30779711
pythonidxpt:30768918
pythondjangots:30768917
pythonnoceb:30776497
copilotsettingt:30767686
e537b577:30772214
h0f32768:30779712
asynctok:30783811
dsvsc013:30777762
dsvsc014:30777825
cmakesidepanelv2:30779593
diffeditorv2:30786206

[eleanorjboyd]

Hello! If you put this in your settings does it work as expected? "python.experiments.optInto": ["pythonTestAdapter"],. I want to determine if it is a regression from the rewrite because I can see in your logs that Experiment 'pythonTestAdapter' is active so this is happening in the rewrite.

[eleanorjboyd]

Hi! Following up again, I see this as well and will put in a fix. Thanks

[eleanorjboyd]

Here is the code I tested it with, this should return an error and marked as such

import pytest


@pytest.fixture
def raise_fixture():
    raise Exception("Dummy exception")


class TestSomething:
    def test_a(self, raise_fixture):
        raise Exception("Dummy exception")
        assert True