Pytest Test Discovery fails when additional output appears

[simondmorias]

Environment data

• VSOnline Container
• Version: 1.42.0 (user setup)
• Commit: ae08d5460b5a45169385ff3fd44208f431992451
• Date: 2020-02-06T10:51:34.058Z
• Electron: 6.1.6
• Chrome: 76.0.3809.146
• Node.js: 12.4.0
• V8: 7.6.303.31-electron.0
• OS: Windows_NT x64 10.0.18362
• Python: 3.7.3
• Python VSCode Extension: 2020.2.63072
• Pytest: 5.3.5
• Python Language Server: 0.5.30.0

Expected behaviour

2 pytest tests should be discovered in this project.

Actual behaviour

Test discovery fails.

Steps to reproduce

• Open VSCode Remote Session
• Create new environment (in VSOnline)
• Custom Settings
• Paste this repo URL: https://github.com/simondmorias/vscode-pytest-discoveryissue

Wait 5-6 minutes for the container to build

Once connected you can run (from the default terminal):

pytest tests

You will see some deprecation warnings - but 2 tests should pass.

Now run Test discovery:

• Either from the test extension
• Or from the footer bar at the bottom of screen

You should see the error "Test Discovery Failed" in the bottom toolbar.

If we open test_broken.py and comment out the from src import broken then test discovery works again.

Logs:

Python Test Log output:

python /home/vscode/.vscode-remote/extensions/ms-python.python-2020.2.63072/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir /home/vscode/workspace -s --cache-clear tests
Test Discovery failed: 
Error: 20/02/13 12:16:01 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).

It appears that the output by pyspark interferes with the discovery process. If we run the command manually:

python /home/vscode/.vscode-remote/extensions/ms-python.python-2020.2.63072/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir /home/vscode/workspace -s --cache-clear tests

The following output is logged:

20/02/13 12:17:22 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
[{"rootid": ".", "root": "/home/vscode/workspace", "parents": [{"id": "./tests", "kind": "folder", "name": "tests", "parentid": ".", "relpath": "./tests"}, {"id": "./tests/test_broken.py", "kind": "file", "name": "test_broken.py", "parentid": "./tests", "relpath": "./tests/test_broken.py"}, {"id": "./tests/test_works.py", "kind": "file", "name": "test_works.py", "parentid": "./tests", "relpath": "./tests/test_works.py"}], "tests": [{"id": "./tests/test_broken.py::test_simple", "name": "test_simple", "source": "./tests/test_broken.py:5", "markers": [], "parentid": "./tests/test_broken.py"}, {"id": "./tests/test_works.py::test_aPlusb", "name": "test_aPlusb", "source": "./tests/test_works.py:4", "markers": [], "parentid": "./tests/test_works.py"}]}]

Here you can see discovery worked - but the logging in front appears to be an issue with the discovery process. Disabling that output is sadly not possible.

[luabud]

Thanks for reporting this, @simondmorias! This is a problem that will be addressed by #6594, so I'm closing this issue in favour of that.