Report errors on discovery more intuitively

[eleanorjboyd]

Currently when there is an error during test collection an item is added to the test explorer tree describing an error occurred and to find the reason in the output panel. A few things:

bugs:

• if you don't reload the window after fixing the error the error node will still exist

feature requests:

• a way to click on the error and have it bring you to the output where the error is printed Improve python test discovery error reporting #19978 and [Pytest] Add an inline error indicating the location in code where test discovery failed #19401
• a more prominent display that an error occurred (dependent on community feedback) Show Notification when Pytest Discovery Fails #21874 and tests are not configured after reloading the window #22093
• a opt-in pop up when discovery fails: Show Notification when Pytest Discovery Fails #21874

[docktermj]

Completely agree on need to improve this. I've blown 8 hours on debugging this so far and still haven't figured out what's wrong.

[docktermj]

I'm on Visual Studio Code 1.83.1 and am seeing this

2023-10-31 14:31:58.447 [info] Experiment 'pythonPromptNewFormatterExt' is active
2023-10-31 14:31:58.447 [info] Experiment 'pythonPromptNewToolsExt' is active
2023-10-31 14:31:58.447 [info] Experiment 'pythonSurveyNotificationcf' is active
2023-10-31 14:31:58.447 [info] Experiment 'pythonTerminalEnvVarActivation' is active
2023-10-31 14:31:58.447 [info] Experiment 'pythonTestAdapter' is active
2023-10-31 14:31:58.447 [info] Test server listening.
2023-10-31 14:31:58.447 [info] Python interpreter path: /bin/python3.10
2023-10-31 14:31:59.793 [info] Starting Pylance language server.
2023-10-31 14:31:59.895 [info] Discover tests for workspace name: undefined - uri: /home/xxxx/.config/Code/User/settings.json
2023-10-31 14:31:59.895 [info] Running discovery for pytest using the new test adapter.
2023-10-31 14:31:59.911 [info] > /bin/python3.10 -m pytest -p vscode_pytest --collect-only tests
2023-10-31 14:31:59.911 [info] cwd: .
2023-10-31 14:32:00.033 [info] Test server connected to a client.
2023-10-31 14:32:00.035 [info] ResultResolver EOT received for discovery.
2023-10-31 14:32:00.053 [error] Subprocess exited unsuccessfully with exit code 4 and signal null.
2023-10-31 14:32:00.054 [error] Subprocess exited unsuccessfully with exit code 4 and signal null. Creating and sending error discovery payload
2023-10-31 14:32:00.054 [error] pytest test discovery error 
  
 The python test process was terminated before it could exit on its own, the process errored with: Code: 4, Signal: null
2023-10-31 14:32:00.054 [info] ResultResolver EOT received for discovery.
2023-10-31 14:32:00.054 [info] Disposing data receiver for /home/xxxx/senzing.git/g2-sdk-python-next and deleting UUID; pytest discovery.
2023-10-31 14:32:17.970 [info] Discover tests for workspace name: g2-sdk-python-next - uri: /home/xxxx/xxxx.git/g2-sdk-python-next
2023-10-31 14:32:17.970 [info] Running discovery for pytest using the new test adapter.
2023-10-31 14:32:17.999 [info] > /bin/python3.10 -m pytest -p vscode_pytest --collect-only tests
2023-10-31 14:32:17.999 [info] cwd: .
2023-10-31 14:32:18.116 [info] Test server connected to a client.
2023-10-31 14:32:18.117 [info] ResultResolver EOT received for discovery.
2023-10-31 14:32:18.134 [error] Subprocess exited unsuccessfully with exit code 4 and signal null.
2023-10-31 14:32:18.135 [error] Subprocess exited unsuccessfully with exit code 4 and signal null. Creating and sending error discovery payload
2023-10-31 14:32:18.135 [error] pytest test discovery error 
  
 The python test process was terminated before it could exit on its own, the process errored with: Code: 4, Signal: null
2023-10-31 14:32:18.135 [info] ResultResolver EOT received for discovery.
2023-10-31 14:32:18.135 [info] Disposing data receiver for /home/xxxx/xxxx.git/g2-sdk-python-next and deleting UUID; pytest discovery.

and have no clue what to do next except spend 8 hours searching the web in vain.

[eleanorjboyd]

@docktermj if you send over your "python" logs and the logs out of "python test log" in a new issue happy to take a look! (just use the command palette developer: set log levels command to put your logs to Trace).

edit: just saw your message, please change log level and resend. The error will also likely be in "python test log".

[docktermj]

I found the fix. In settings.json:

 "python.experiments.optOutFrom": [
        "pythonTestAdapter"
    ],

But I gotta' admit, wasn't easy to figure this out.

[eleanorjboyd]

Hello @docktermj, glad you figured out a work around. Where were you initially looking that you think could better surface that setting so it would have been easier to find? Also would you be able to send your logs from before? We will be switching over to only using the rewrite and removing the opt-out options and I would love to understand / fix your bug so you don't face problems in the future. Thanks!

[PterX]

I found the fix. In settings.json:

 "python.experiments.optOutFrom": [
        "pythonTestAdapter"
    ],

But I gotta' admit, wasn't easy to figure this out.

I have same issue