coverage not launched anymore

[revesansparole]

I want to run pytest with coverage option only when the command is not launched inside pycharm. Previously I just added a hook in conftest.py and it was working well:

def pytest_cmdline_preparse(args):
    if 'PYCHARM_HOSTED' not in os.environ:
        args.append("--cov=waloupkg")

However the latest pytest calls this hook only after pytest-cov hooks have been already called and I'm unable to launch the cov pluggin as demonstrated below. In case 1) I do not get a coverage report while in case 2) I get one (not displayed here).

1. This is what happen when I do not specify '--cov' on the command line expecting my hook to do it for me:

(toto) C:\Users\jchopard\Documents\eval\pkging\waloupkg
>pytest -s
cov: pytest_load_initial_conftests ['--maxfail=2', '-rf', '-s']
mine: pytest_cmdline_preparse
=============================================== test session starts ================================================
platform win32 -- Python 3.7.1, pytest-3.10.0, py-1.7.0, pluggy-0.8.0
rootdir: C:\Users\jchopard\Documents\eval\pkging\waloupkg, inifile: setup.cfg
plugins: cov-2.6.0
collected 1 item

2. This is what happen when I explicitly specify '--cov' on the command line

(toto) C:\Users\jchopard\Documents\eval\pkging\waloupkg
>pytest -s --cov
cov: pytest_load_initial_conftests ['--maxfail=2', '-rf', '-s', '--cov']
cov: __init__
cov: start
mine: pytest_cmdline_preparse
=============================================== test session starts ================================================
platform win32 -- Python 3.7.1, pytest-3.10.0, py-1.7.0, pluggy-0.8.0
rootdir: C:\Users\jchopard\Documents\eval\pkging\waloupkg, inifile: setup.cfg
plugins: cov-2.6.0
collected 1 item

I tried to use pytest_load_initial_conftestsinstead of pytest_cmdline_preparse but the hook was never called.

Apparently the problem is not the order but the fact that pytest_configure does not exists anymore in pytest-cov (plugin.py) since version 2.6. I tried to add it by hand but pytest is not calling it. I must be missing something.

Is there any way to obtain the researched behaviour?

[nedbat]

@revesansparole I'm curious: why don't you want the coverage measurement when you run tests inside pycharm?

[revesansparole]

There is an issue between pycharm and pytest-cov. If you have the option '--cov' set while launching the tests in pycharm you cannot use the debugger any more and all your break points are silently ignored.

If you have any leads on how to solve the problem, it would be greatly appreciated.

[ionelmc]

There is the --no-cov option which you should should add to the "debug configuration" in pycharm.

[revesansparole]

While using the --no-cov option works and is certainly the way to go, I'm looking at a solution that does not require all the coders in my company to configure their pycharm if possible.

[nedbat]

Instead of trying to un-coverage inside pycharm, it might be easier to adjust how you are running coverage outside of pycharm. How do you run the tests outside? If you have a Makefile, or tox, or even a shell script, you can use the --cov options you need there.

[revesansparole]

That's the solution we adopted for the moment. No coverage by default and our continuous integration script specifically use the --cov option.

However this solution penalize command line users that want to run the tests locally and are forced to also add the --cov option by hand instead of just typing 'pytest' and relying on common default behaviour in our company.

It's also a question of philosophy. I like the idea of defining common default for developing a package (in this particular case we decided that running coverage with the tests all the time is a good idea) and not being constrained by the IDE that will be used to edit the code.

[nedbat]

I guess we have a difference of philosophy. I would rather teach my collaborators to type "tox" or "make test" than to try to have everything done implicitly by "pytest".

[revesansparole]

My co-worker just transferred from another company where they use in-house command like 'make test' and got lost since, by definition, we don't have that exact command available. Therefore I'd rather teach people to use 'pytest' and all the setting machinery that comes with it, knowing they will be able to use that knowledge everywhere.

Of course, I also agree with your position and realize it's just a matter of where to put the cursor. I considered 'tox' for a while but our deployment workflow is still simple enough to justify staying at just pytest level.

[ionelmc]

I don't really see the problem of typing --cov - it's just 5 characters. But regardless, if you really want to have magical enable/disable behavior (my gun, your foot 😁):

def pytest_configure(config):
    if 'PYCHARM_HOSTED' in os.environ:
        config.pluginmanager.set_blocked('_cov')

And you set the --cov options in your pytest.ini or whatever (that's the right place for configuration anyway, not some function inside a likely massive conftest).