Pytest debugging assumes pytest-cov is installed

[mjpieters]

Type: Bug

Behaviour

Expected vs. Actual

I can't debug pytest tests unless I also install pytest-cov.

Instead, testlauncher.py reports:

ERROR: usage: testlauncher.py [options] [file_or_dir] [file_or_dir] [...]
testlauncher.py: error: unrecognized arguments: --no-cov
  inifile: /<workspace>/setup.cfg
  rootdir: /<workspace>

If you first run pip install pytest-cov the debug session starts correctly.

Alternatively, I can edit testlauncher.py and insert a if "--no-cov" in args: args.remove("--no-cov") line in the parse_argv() function, before the return statement, as a crude work-around.

Steps to reproduce:

1. Open a Python project with pytest tests, but no pytest-cov included
2. Run "Python: Configure Tests" and select pytest
3. Select a test in the Testing tab, click on the debug icon
4. The console opens to run the testlauncher.py script, then fails

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.11.3
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Poetry
• Value of the python.languageServer setting: Default

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2023-04-28 12:48:26.278 [info] > . ~/Library/Caches/pypoetry/virtualenvs/[[venv]]/bin/activate && echo '[[UUID]]' && python ~/.vscode/extensions/ms-python.python-2023.7.11111009/pythonFiles/printEnvVariables.py
2023-04-28 12:48:26.278 [info] shell: bash
2023-04-28 12:48:26.306 [info] > ~/Library/Caches/pypoetry/virtualenvs/[[venv]]/bin/python ~/.vscode/extensions/ms-python.python-2023.7.11111009/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear
2023-04-28 12:48:26.306 [info] cwd: .
2023-04-28 12:48:30.546 [info] DAP Server launched with command: /Users/[[username]]/Library/Caches/pypoetry/virtualenvs/[[venv]]/bin/python /Users/[[username]]/.vscode/extensions/ms-python.python-2023.7.11111009/pythonFiles/lib/python/debugpy/adapter
2023-04-28 12:48:30.716 [info] Send text to terminal: source /Users/[[username]]/Library/Caches/pypoetry/virtualenvs/[[venv]]/bin/activate
2023-04-28 12:48:31.759 [info] Send text to terminal:  /usr/bin/env /Users/[[username]]/Library/Caches/pypoetry/virtualenvs/[[venv]]/bin/python /Users/[[username]]/.vscode/extensions/ms-python.python-2023.7.11111009/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher 65263 -- /Users/[[username]]/.vscode/extensions/ms-python.python-2023.7.11111009/pythonFiles/testlauncher.py [[workspace]] pytest --rootdir [[workspace]] --override-ini junit_family=xunit1 --junit-xml=/[[tempdir]]/tmp-[[random]].xml --capture no --no-cov ./tests/test_decorator.py::test_method 
2023-04-28 12:50:41.270 [info] > ~/Library/Caches/pypoetry/virtualenvs/[[venv]]/bin/python ~/.vscode/extensions/ms-python.python-2023.7.11111009/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear
2023-04-28 12:50:41.271 [info] cwd: .
2023-04-28 12:50:43.997 [info] > . ~/Library/Caches/pypoetry/virtualenvs/[[venv]]/bin/activate && echo '[[UUID]]' && python ~/.vscode/extensions/ms-python.python-2023.7.11111009/pythonFiles/printEnvVariables.py
2023-04-28 12:50:43.997 [info] shell: bash
2023-04-28 12:50:44.020 [info] > ~/Library/Caches/pypoetry/virtualenvs/[[venv]]/bin/python -c "import pytest"
2023-04-28 12:54:43.413 [info] DAP Server launched with command: /Users/[[username]]/Library/Caches/pypoetry/virtualenvs/[[venv]]/bin/python /Users/[[username]]/.vscode/extensions/ms-python.python-2023.7.11111009/pythonFiles/lib/python/debugpy/adapter
2023-04-28 12:54:43.497 [info] Send text to terminal:  cd [[workspace]] ; /usr/bin/env [[workspace]] pytest --rootdir [[workspace]] --override-ini junit_family=xunit1 --junit-xml=/var/folders/zr/sp474f_d38xfvml_n2y_8tfr0000gn/T/tmp-881198HN8O7Twwo6S.xml --capture no --no-cov ./tests/test_decorator.py::test_method 

Sensitive information (virtual-env name, local username and random values) have been redacted by replacing them with [[valuetype]] blocks.

User Settings

languageServer: "Pylance"

formatting
• provider: "black"

testing
• pytestEnabled: true

Extension version: 2023.7.11111009
VS Code version: Code 1.77.3 (704ed70d4fd1c6bd6342c436f1ede30d1cff4710, 2023-04-12T09:41:48.638Z)
OS version: Darwin arm64 21.6.0
Modes:
Sandboxed: No

System Info

Item	Value
CPUs	Apple M1 Pro (10 x 24)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled metal: disabled_off multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_renderer: enabled_on video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: disabled_off
Load (avg)	3, 4, 6
Memory (System)	32.00GB (0.26GB free)
Process Argv	--crash-reporter-id 9a523bf8-41cd-4244-ba19-546775630d3c
Screen Reader	no
VM	0%

Connection to 'ssh-remote+parsel' could not be established

A/B Experiments

vsliv368cf:30146710
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
vserr242cf:30382550
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vsdfh931:30280409
vshan820:30294714
vstes263:30335439
pythondataviewer:30285071
vscod805cf:30301675
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
cmake_vspar411:30581797
vsaa593:30376534
pythonvs932:30410667
cppdebug:30492333
vsclangdc:30486549
c4g48928:30535728
dsvsc012:30540252
pynewext54:30695312
azure-dev_surveyone:30548225
nodejswelcome1cf:30587006
2e4cg342:30602488
pyind779:30671433
89544117:30613380
pythonsymbol12:30671437
azdwalk:30721579
pythonms35:30701012
pythonfmttextcf:30716743
pythonclientmvcf:30725893

[mjpieters]

The root cause is #21048.

The extension should only add the --no-cov switch if pytest-cov is actually installed. Or, alternatively, the testlauncher script could remove the argument after loading pytest and enumerating what plugins are installed.

[mjpieters]

If testlauncher.py was to be made responsible, then the following code would verify if pytest_cov is installed:

from _pytest.config import PytestPluginManager
manager = PytestPluginManager()
manager.load_setuptools_entrypoints('pytest11')
if not manager.has_plugin('pytest_cov'):
    args.remove("--no-cov")

This feels somewhat invasive and inefficient.