pytest colored output (with --color=yes) makes the test output log unreadable (by humans)

[ogrisel]

VS Code version

1.62.3

Extension version

v2021.11.1422169775

OS type

macOS

OS version

12.0

Python distribution

Other

Python version

3.9

Language server

Pylance

Expected behaviour

I expect the output log in "Python Test Log" to not display the control characters, for instance by stripping them, that is obtaining the following output:

============================= test session starts ==============================
platform darwin -- Python 3.9.7, pytest-6.2.5, py-1.10.0, pluggy-0.13.1
rootdir: /Users/ogrisel/code/scikit-learn, configfile: setup.cfg
plugins: anyio-3.3.0, xdist-2.3.0, timeout-1.4.2, forked-1.3.0
collected 1 item

sklearn/impute/tests/test_base.py .                                      [100%]

- generated xml file: /var/folders/_y/lfnx34p13w3_sr2k12bjb05w0000gn/T/tmp-17017Jlww71GJ2jF6.xml -
============================== 1 passed in 0.07s ===============================

Alternatively, colors could be rendered in the log as they are when running the tests in a shell terminal:

Actual behaviour

The "Python Test Log" displays the color control characters such as �[32m without rendering them:

�[1m============================= test session starts ==============================�[0m
platform darwin -- Python 3.9.7, pytest-6.2.5, py-1.10.0, pluggy-0.13.1
rootdir: /Users/ogrisel/code/debugpytestvscode, configfile: setup.cfg
plugins: anyio-3.3.0, xdist-2.3.0, timeout-1.4.2, forked-1.3.0
collected 1 item

test_debug_vscode.py �[32m.�[0m�[32m                                                   [100%]�[0m

- generated xml file: /var/folders/_y/lfnx34p13w3_sr2k12bjb05w0000gn/T/tmp-24919mvRdR0hh0QGh.xml -
�[32m============================== �[32m�[1m1 passed�[0m�[32m in 0.01s�[0m�[32m ===============================�[0m

Steps to reproduce

Create a new test Python test project, configure a Python interpreter with pytest installed and add a setup.cfg file containing:

[aliases]
test = pytest

[tool:pytest]
addopts = --color=yes

and a test file with a single test function, for instance a file named test_debug_vscode.py with the following content:

def test_donothing_but_pass():
    assert True

Then:

Shift-Ctrl-P > Python: Configure Tests > pytest > Use existing config file setup.cfg

Then launch the test by clicking on the green > icon in the VS Code UI and open the "Python Test Log" output view.

Logs

The Python log itself has no problem:

> ~/mambaforge/envs/dev/bin/python -m pytest --rootdir ~/code/debugpytestvscode --override-ini junit_family=xunit1 --junit-xml=/var/folders/_y/lfnx34p13w3_sr2k12bjb05w0000gn/T/tmp-24919pLfLJqAh0u2i.xml
cwd: ~/code/debugpytestvscode

Code of Conduct

• I agree to follow this project's Code of Conduct

[ogrisel]

The same (or a similar) bug was previously reported as #11497 but wasn't fixed because it was judged by @karrtikr to lack details to make it reproducible while the --color=yes options was already suspected as the culprit by @ksamuel in the discussion and actually helped pinpoint the cause of the problem on my environment.

Hopefully the minimal reproducer above should make the cause of the problem clearer.

[ogrisel]

This StackOverflow answer might help trim ANSI escape sequences from Python test log outputs:

https://stackoverflow.com/a/14693789/163740

[karthiknadig]

This should be addressed as a part of re-writing the test adapter. #17242

Currently we parse the pytest output, that causes this problem.

[ogrisel]

Currently we parse the pytest output, that causes this problem.

Parsing is not the problem: the test runner can complete successfully with green markers everywhere. It's just the display in the "Python Test Log" output view that is unreadable.

[karthiknadig]

@ogrisel I see, thanks for clarifying.

[flying-sheep]

This StackOverflow answer might help trim ANSI escape sequences from Python test log outputs:

We don’t want to trim the color, we want to display the color.

[toniengelhardt]

The Python Test Log should produce the same colored output as when running pytest in the terminal, otherwise it is useless. Ideally it would also respect the pytest-sugar plugin IMO.

[eleanorjboyd]

Color output is now possible on the new testing rewrite! The output will now go to the Test Result panel and this can use the color if enabled via including it in the pytest args.

[hoechenberger]

@eleanorjboyd Thank you very much for this update! Just to clarify, by "testing rewrite" you mean – what? Is this something we'll likely see in the next release of the Python extension? :)

[eleanorjboyd]

Hello! Sorry for the lack of clarity there, you can read about the rewrite which is under an experiment here: https://devblogs.microsoft.com/python/python-in-visual-studio-code-june-2023-release/ under Test Discovery and Execution Rewrite.

To use the rewrite yourself just add ”python.experiments.optInto": ["pythonTestAdapter"] to your user settings. You can confirm you have the rewrite enabled by setting your log level to trace, via the Developer: Set Log Level command in the command palette. Then check to see if Experiment 'pythonTestAdapter' is active is in your python logs.

Additional information on the change in output can be found here: https://github.com/microsoft/vscode-python/wiki/New-Method-for-Output-Handling-in-Python-Testing

[hoechenberger]

Thank you very much, @eleanorjboyd!

[mehdigmira]

Hello,

I tried ”python.experiments.optInto": ["pythonTestAdapter"], and I now have colored output in the test result window, which is cool. But noticed:

• When running in debug, the output is not colored
• The output displayed in the editor (the one that is displayed in a red window that you can fold/unfold next to the test declaration) is not colored, which is too bad, since that's the one I use the most
  Is this expected ?

[eleanorjboyd]

Hello @mehdigmira, I have just returned from vacation and will take a look at this shortly thank you for your detailed notes.

one quick question, @connor4312 do you know if the pop up window they are mentioning supports coloring the same way as the test result panel?