Test Icons Not Updating when using pytest

[aaronsmith1234]

Issue Type: Bug

Hi,
While using the built-in Python test tab in VSCode, occasionally the test status icons (green checkmark or red X) get out of sync with the actual test results. As an example, I have tests that were once failing, and are now passing, but the icons still are red X's; when running tests, the icons dont update to be the spinning arrows, and after the tests are done, they are what they were before. At the bottom of the screen, the test status is correct, and inside the test file the test status is correct, its just the test explorer tab.

Note that closing VSCode entirely and re-opening it fixes the problem. I can't reliably reproduce, it just happens sometimes on some projects

Extension version: 2021.1.502429796
VS Code version: Code 1.52.1 (ea3859d4ba2f3e577a159bc91e3074c5d85c0523, 2020-12-16T16:34:46.910Z)
OS version: Windows_NT x64 10.0.18363

System Info

Item	Value
CPUs	Intel(R) Core(TM) i5-8365U CPU @ 1.60GHz (8 x 1896)
GPU Status	2d_canvas: enabled flash_3d: enabled flash_stage3d: enabled flash_stage3d_baseline: enabled gpu_compositing: enabled multiple_raster_threads: enabled_on oop_rasterization: disabled_off opengl: enabled_on protected_video_decode: enabled rasterization: enabled skia_renderer: disabled_off_ok video_decode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled
Load (avg)	undefined
Memory (System)	15.75GB (6.86GB free)
Process Argv	--crash-reporter-id 72142afe-7e98-4075-91ad-09c713b0d234
Screen Reader	no
VM	0%

A/B Experiments

vsliv368:30146709
vsreu685:30147344
openlogontheside:30221877
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vswsl492:30211401
wsl2promptcf:30224613
pythonvsdeb440:30248342
unusedpromptcf:30224611
folderexplorer:30224614
openfilemenu:30224647
pythonvsded773:30248341
pythonvspyt600cf:30247032

[kimadeline]

Hi @aaronsmith1234,

Thanks for the bug report! We investigate issues in order based on priority and severity, which includes the impact it has on your ability to use the extension to do productive work, and the number of people affected. If other users come forward and leave a comment demonstrating they are seeing/reproducing the problem then we will raise this issue's priority.

If you do end up having a sample project that we could use to reliably repro this issue, along with extra info like which test framework you're using, or your general workspace setup, that would be very useful 🙂

Thank you for your understanding and patience!

[aaronsmith1234]

For sure! General workspace settings is minimal; really only saying what test framework I'm using (Pytest)

[rubenvdg]

Same here, the pytest icons are stale:

while all tests are actually passing:

[c-byte88]

Also experiencing the issue, icons don't update from ? when running using pytest.

Test running

Showing as passed in test output:

Environment Details

Running attached to a container via wsl2 (ubuntu-20.04)

Extension version: v2021.4.765268190
VS Code version: Code 1.52.2
OS version: Windows_NT x64 10.0.19042

Using Remote settings:

{
	"python.formatting.provider": "yapf",
	"python.formatting.yapfArgs": [
		" — style",
		"${workspaceFolder}/setup.cfg"
	],
	"python.linting.pylintArgs": [
		"--rcfile=${workspaceFolder}/setup.cfg"
	],
	"python.linting.mypyArgs": [
		"--config-file=${workspaceFolder}/setup.cfg"
	],
	"python.linting.enabled": true,
	"python.linting.pylintEnabled": true,
	"remote.autoForwardPorts": false,
	"remote.restoreForwardedPorts": false,
	"terminal.integrated.shell.linux": "/bin/bash",
	"python.languageServer": "Jedi",
	"python.jediEnabled": true,
	"python.autoComplete.addBrackets": true,
	"python.linting.mypyEnabled": true,
	"python.linting.flake8Enabled": true,
	"python.testing.pytestEnabled": true,
	"python.testing.pytestArgs": [
		"--rootdir=./src/python/"
	],
	"python.linting.banditEnabled": true,
	"python.formatting.blackPath": "/opt/conda/envs/xxxxxxxxx/bin/black",
	"python.linting.pylintPath": "/opt/conda/envs/xxxxxxxxx/bin/pylint",
	"python.pythonPath": "/opt/conda/envs/xxxxxxxxx/bin/python",
	"python.venvPath": "/opt/conda/envs/xxxxxxxxx/lib/python3.7/site-packages/pyspark/jars",
	"python.formatting.autopep8Path": "/opt/conda/envs/xxxxxxxxx/bin/autopep8",
	"python.formatting.yapfPath": "/opt/conda/envs/xxxxxxxxx/bin/yapf",
	"python.linting.banditPath": "/opt/conda/envs/xxxxxxxxx/bin/bandit",
	"python.linting.flake8Path": "/opt/conda/envs/xxxxxxxxx/bin/flake8",
	"python.linting.mypyPath": "/opt/conda/envs/xxxxxxxxx/bin/mypy",
	"python.linting.pycodestylePath": "/opt/conda/envs/xxxxxxxxx/bin/pycodestyle",
	"python.envFile": "${workspaceFolder}/.env",
	"files.watcherExclude": {
		"**/build/**": true,
		"**/dist/**": true,
		"**/.ipynb_checkpoints/**": true,
		"**/*.egg-info/**": true,
		"**/.pytest_cache/**": true,
		"**/__pycache__/**": true,
		"**/.mypy_cache/**": true,
		"**/.venv/**": true
	},
	"files.exclude": {
		"**/.pytest_cache/**": true,
		"**/.mypy_cache/**": true,
		"**/__pycache__/**": true,
		"**/*.egg-info/**": true
	},
	"workbench.editorAssociations": [
		{
			"viewType": "jupyter.notebook.ipynb",
			"filenamePattern": "*.ipynb"
		}
	],
	"jupyter.defaultCellMarker": "# COMMAND ----------",
	"jupyter.codeRegularExpression": "^(#\\s*%%|#\\s*\\<codecell\\>|#\\s*In\\[\\d*?\\]|#\\s*In\\[ \\]|#\\s*Databricks\\snotebook\\ssource)",
	"jupyter.jupyterServerType": "remote",
	"jupyter.allowUnauthorizedRemoteConnection": true,
	"jupyter.alwaysTrustNotebooks": true
}

[kimadeline]

Does any of you have a sample repo I could use to try and reproduce the issue? Also does it happen every time, or only with specific projects or configurations?

[c-byte88]

@kimadeline I've included the docker setup as it was pertinent to my environment. Thanks

https://github.com/nigelc-byte88/vscode-pytest-bugsample

This is happening every time on this project (the full version) and every project that follows this particular template layout. (i.e. using docker dev container)