Calling pytest.main() without arguments unpredictably reads arguments from the command line

[antosikv]

In my opinion, it would be quite reasonable to assume that running

pytest.main()

would be equivalent to simply executing pytest from the command line with no arguments. The documentation even seems to suggest that this should be the case:

You can invoke pytest from Python code directly:

retcode = pytest.main()

this acts as if you would call “pytest” from the command line.

Instead, the default value of args passed into the function is None, in which case we end up reading arguments from the command line. That seems to be a fairly weird choice for the "default" behavior, since we are now trying to use arguments passed to a completely different program. At best, this can easily lead to complaints about unrecognized options and at worst, if we happen to share some options with the calling script, possible misinterpretation of those options and subsequent unexpected behavior.

It seems to me that automatically reading arguments directly from the command line would only be reasonable and expected when pytest is executed directly from the command line. When called programmatically from a script, it would probably be more predictable to just assume having no arguments unless explicitly specified otherwise.

This would obviously introduce a breaking change for those relying on main() silently reading arguments from the command line but that frankly does not seem like a good practice anyway. And considering that it would be a trivial single-line change of using main(sys.argv[1:]) instead, it definitely seems like a good idea to be explicit about it.

Anyway, if introducing a breaking change would be unacceptable, despite the arguably more predictable default behavior, it would probably be a good idea to at least amend the documentation and add an explicit warning about this behavior.

[RonnyPfannschmidt]

This is the standard pattern for callables that get used as console script entry point

[antosikv]

But main is not used as an entry point. There is a function specifically for that called console_main, which should imo read the console arguments and subsequently pass them to main instead of calling it with no arguments. Even the docs specifically describe main as used to "Perform an in-process test run", in which case this behavior definitely does not seem expected.

[bluetech]

@antosikv FWIW I agree it would have been better if main required explicitly passing sys.argv. Historically console_main didn't exist and pytest.main was the entry point. Unfortunately I think it would be too disruptive to change now, even with a deprecation.

However it should definitely be clear in the docs. Would you like to send a PR updating the docs? (You don't need to worry about building the docs locally etc, when you send a PR there is a link to the rendered docs)

[antosikv]

@bluetech Ah, makes more sense if main used to be the entrypoint. I will at least open a PR to update the docs then.