Add pyproject.toml support (#7247)

Author: nicoddemus

.gitignore

@@ -29,6 +29,7 @@ doc/*/_changelog_towncrier_draft.rst
 build/
 dist/
 *.egg-info
+htmlcov/
 issue/
 env/
 .env/

changelog/1556.feature.rst

@@ -0,0 +1,17 @@
+pytest now supports ``pyproject.toml`` files for configuration.
+
+The configuration options is similar to the one available in other formats, but must be defined
+in a ``[tool.pytest.ini_options]`` table to be picked up by pytest:
+
+.. code-block:: toml
+
+    # pyproject.toml
+    [tool.pytest.ini_options]
+    minversion = "6.0"
+    addopts = "-ra -q"
+    testpaths = [
+        "tests",
+        "integration",
+    ]
+
+More information can be found `in the docs <https://docs.pytest.org/en/stable/customize.html#configuration-file-formats>`__.

doc/en/customize.rst

@@ -14,15 +14,112 @@ configurations files by using the general help option:
 This will display command line and configuration file settings
 which were registered by installed plugins.
 
+.. _`config file formats`:
+
+Configuration file formats
+--------------------------
+
+Many :ref:`pytest settings <ini options ref>` can be set in a *configuration file*, which
+by convention resides on the root of your repository or in your
+tests folder.
+
+A quick example of the configuration files supported by pytest:
+
+pytest.ini
+~~~~~~~~~~
+
+``pytest.ini`` files take precedence over other files, even when empty.
+
+.. code-block:: ini
+
+    # pytest.ini
+    [pytest]
+    minversion = 6.0
+    addopts = -ra -q
+    testpaths =
+        tests
+        integration
+
+
+pyproject.toml
+~~~~~~~~~~~~~~
+
+.. versionadded:: 6.0
+
+``pyproject.toml`` are considered for configuration when they contain a ``tool.pytest.ini_options`` table.
+
+.. code-block:: toml
+
+    # pyproject.toml
+    [tool.pytest.ini_options]
+    minversion = "6.0"
+    addopts = "-ra -q"
+    testpaths = [
+        "tests",
+        "integration",
+    ]
+
+.. note::
+
+    One might wonder why ``[tool.pytest.ini_options]`` instead of ``[tool.pytest]`` as is the
+    case with other tools.
+
+    The reason is that the pytest team intends to fully utilize the rich TOML data format
+    for configuration in the future, reserving the ``[tool.pytest]`` table for that.
+    The ``ini_options`` table is being used, for now, as a bridge between the existing
+    ``.ini`` configuration system and the future configuration format.
+
+tox.ini
+~~~~~~~
+
+``tox.ini`` files are the configuration files of the `tox <https://tox.readthedocs.io>`__ project,
+and can also be used to hold pytest configuration if they have a ``[pytest]`` section.
+
+.. code-block:: ini
+
+    # tox.ini
+    [pytest]
+    minversion = 6.0
+    addopts = -ra -q
+    testpaths =
+        tests
+        integration
+
+
+setup.cfg
+~~~~~~~~~
+
+``setup.cfg`` files are general purpose configuration files, used originally by `distutils <https://docs.python.org/3/distutils/configfile.html>`__, and can also be used to hold pytest configuration
+if they have a ``[tool:pytest]`` section.
+
+.. code-block:: ini
+
+    # setup.cfg
+    [tool:pytest]
+    minversion = 6.0
+    addopts = -ra -q
+    testpaths =
+        tests
+        integration
+
+.. warning::
+
+    Usage of ``setup.cfg`` is not recommended unless for very simple use cases. ``.cfg``
+    files use a different parser than ``pytest.ini`` and ``tox.ini`` which might cause hard to track
+    down problems.
+    When possible, it is recommended to use the latter files, or ``pyproject.toml``, to hold your
+    pytest configuration.
+
+
 .. _rootdir:
-.. _inifiles:
+.. _configfiles:
 
-Initialization: determining rootdir and inifile
------------------------------------------------
+Initialization: determining rootdir and configfile
+--------------------------------------------------
 
 pytest determines a ``rootdir`` for each test run which depends on
 the command line arguments (specified test files, paths) and on
-the existence of *ini-files*.  The determined ``rootdir`` and *ini-file* are
+the existence of configuration files.  The determined ``rootdir`` and ``configfile`` are
 printed as part of the pytest header during startup.
 
 Here's a summary what ``pytest`` uses ``rootdir`` for:
@@ -48,48 +145,47 @@ Finding the ``rootdir``
 
 Here is the algorithm which finds the rootdir from ``args``:
 
-- determine the common ancestor directory for the specified ``args`` that are
+- Determine the common ancestor directory for the specified ``args`` that are
   recognised as paths that exist in the file system. If no such paths are
   found, the common ancestor directory is set to the current working directory.
 
-- look for ``pytest.ini``, ``tox.ini`` and ``setup.cfg`` files in the ancestor
-  directory and upwards.  If one is matched, it becomes the ini-file and its
-  directory becomes the rootdir.
+- Look for ``pytest.ini``, ``pyproject.toml``, ``tox.ini``, and ``setup.cfg`` files in the ancestor
+  directory and upwards.  If one is matched, it becomes the ``configfile`` and its
+  directory becomes the ``rootdir``.
 
-- if no ini-file was found, look for ``setup.py`` upwards from the common
+- If no configuration file was found, look for ``setup.py`` upwards from the common
   ancestor directory to determine the ``rootdir``.
 
-- if no ``setup.py`` was found, look for ``pytest.ini``, ``tox.ini`` and
+- If no ``setup.py`` was found, look for ``pytest.ini``, ``pyproject.toml``, ``tox.ini``, and
   ``setup.cfg`` in each of the specified ``args`` and upwards. If one is
-  matched, it becomes the ini-file and its directory becomes the rootdir.
+  matched, it becomes the ``configfile`` and its directory becomes the ``rootdir``.
 
-- if no ini-file was found, use the already determined common ancestor as root
+- If no ``configfile`` was found, use the already determined common ancestor as root
   directory. This allows the use of pytest in structures that are not part of
-  a package and don't have any particular ini-file configuration.
+  a package and don't have any particular configuration file.
 
 If no ``args`` are given, pytest collects test below the current working
-directory and also starts determining the rootdir from there.
+directory and also starts determining the ``rootdir`` from there.
 
-:warning: custom pytest plugin commandline arguments may include a path, as in
-    ``pytest --log-output ../../test.log args``. Then ``args`` is mandatory,
-    otherwise pytest uses the folder of test.log for rootdir determination
-    (see also `issue 1435 <https://github.com/pytest-dev/pytest/issues/1435>`_).
-    A dot ``.`` for referencing to the current working directory is also
-    possible.
+Files will only be matched for configuration if:
 
-Note that an existing ``pytest.ini`` file will always be considered a match,
-whereas ``tox.ini`` and ``setup.cfg`` will only match if they contain a
-``[pytest]`` or ``[tool:pytest]`` section, respectively. Options from multiple ini-files candidates are never
-merged - the first one wins (``pytest.ini`` always wins, even if it does not
-contain a ``[pytest]`` section).
+* ``pytest.ini``: will always match and take precedence, even if empty.
+* ``pyproject.toml``: contains a ``[tool.pytest.ini_options]`` table.
+* ``tox.ini``: contains a ``[pytest]`` section.
+* ``setup.cfg``: contains a ``[tool:pytest]`` section.
 
-The ``config`` object will subsequently carry these attributes:
+The files are considered in the order above. Options from multiple ``configfiles`` candidates
+are never merged - the first match wins.
+
+The internal :class:`Config <_pytest.config.Config>` object (accessible via hooks or through the :fixture:`pytestconfig` fixture)
+will subsequently carry these attributes:
 
 - ``config.rootdir``: the determined root directory, guaranteed to exist.
 
-- ``config.inifile``: the determined ini-file, may be ``None``.
+- ``config.inifile``: the determined ``configfile``, may be ``None`` (it is named ``inifile``
+  for historical reasons).
 
-The rootdir is used as a reference directory for constructing test
+The ``rootdir`` is used as a reference directory for constructing test
 addresses ("nodeids") and can be used also by plugins for storing
 per-testrun information.
 
@@ -100,73 +196,36 @@ Example:
     pytest path/to/testdir path/other/
 
 will determine the common ancestor as ``path`` and then
-check for ini-files as follows:
+check for configuration files as follows:
 
 .. code-block:: text
 
     # first look for pytest.ini files
     path/pytest.ini
-    path/tox.ini    # must also contain [pytest] section to match
-    path/setup.cfg  # must also contain [tool:pytest] section to match
+    path/pyproject.toml  # must contain a [tool.pytest.ini_options] table to match
+    path/tox.ini         # must contain [pytest] section to match
+    path/setup.cfg       # must contain [tool:pytest] section to match
     pytest.ini
-    ... # all the way down to the root
+    ... # all the way up to the root
 
     # now look for setup.py
     path/setup.py
     setup.py
-    ... # all the way down to the root
-
-
-.. _`how to change command line options defaults`:
-.. _`adding default options`:
-
-
+    ... # all the way up to the root
 
-How to change command line options defaults
-------------------------------------------------
 
-It can be tedious to type the same series of command line options
-every time you use ``pytest``.  For example, if you always want to see
-detailed info on skipped and xfailed tests, as well as have terser "dot"
-progress output, you can write it into a configuration file:
+.. warning::
 
-.. code-block:: ini
-
-    # content of pytest.ini or tox.ini
-    [pytest]
-    addopts = -ra -q
-
-    # content of setup.cfg
-    [tool:pytest]
-    addopts = -ra -q
-
-Alternatively, you can set a ``PYTEST_ADDOPTS`` environment variable to add command
-line options while the environment is in use:
-
-.. code-block:: bash
-
-    export PYTEST_ADDOPTS="-v"
-
-Here's how the command-line is built in the presence of ``addopts`` or the environment variable:
-
-.. code-block:: text
-
-    <pytest.ini:addopts> $PYTEST_ADDOPTS <extra command-line arguments>
-
-So if the user executes in the command-line:
-
-.. code-block:: bash
-
-    pytest -m slow
-
-The actual command line executed is:
-
-.. code-block:: bash
+    Custom pytest plugin commandline arguments may include a path, as in
+    ``pytest --log-output ../../test.log args``. Then ``args`` is mandatory,
+    otherwise pytest uses the folder of test.log for rootdir determination
+    (see also `issue 1435 <https://github.com/pytest-dev/pytest/issues/1435>`_).
+    A dot ``.`` for referencing to the current working directory is also
+    possible.
 
-    pytest -ra -q -v -m slow
 
-Note that as usual for other command-line applications, in case of conflicting options the last one wins, so the example
-above will show verbose output because ``-v`` overwrites ``-q``.
+.. _`how to change command line options defaults`:
+.. _`adding default options`:
 
 
 Builtin configuration file options

doc/en/example/pythoncollection.rst

@@ -115,15 +115,13 @@ Changing naming conventions
 
 You can configure different naming conventions by setting
 the :confval:`python_files`, :confval:`python_classes` and
-:confval:`python_functions` configuration options.
+:confval:`python_functions` in your :ref:`configuration file <config file formats>`.
 Here is an example:
 
 .. code-block:: ini
 
     # content of pytest.ini
     # Example 1: have pytest look for "check" instead of "test"
-    # can also be defined in tox.ini or setup.cfg file, although the section
-    # name in setup.cfg files should be "tool:pytest"
     [pytest]
     python_files = check_*.py
     python_classes = Check
@@ -165,8 +163,7 @@ You can check for multiple glob patterns by adding a space between the patterns:
 .. code-block:: ini
 
     # Example 2: have pytest look for files with "test" and "example"
-    # content of pytest.ini, tox.ini, or setup.cfg file (replace "pytest"
-    # with "tool:pytest" for setup.cfg)
+    # content of pytest.ini
     [pytest]
     python_files = test_*.py example_*.py
 

doc/en/example/simple.rst

@@ -3,6 +3,50 @@
 Basic patterns and examples
 ==========================================================
 
+How to change command line options defaults
+-------------------------------------------
+
+It can be tedious to type the same series of command line options
+every time you use ``pytest``.  For example, if you always want to see
+detailed info on skipped and xfailed tests, as well as have terser "dot"
+progress output, you can write it into a configuration file:
+
+.. code-block:: ini
+
+    # content of pytest.ini
+    [pytest]
+    addopts = -ra -q
+
+
+Alternatively, you can set a ``PYTEST_ADDOPTS`` environment variable to add command
+line options while the environment is in use:
+
+.. code-block:: bash
+
+    export PYTEST_ADDOPTS="-v"
+
+Here's how the command-line is built in the presence of ``addopts`` or the environment variable:
+
+.. code-block:: text
+
+    <pytest.ini:addopts> $PYTEST_ADDOPTS <extra command-line arguments>
+
+So if the user executes in the command-line:
+
+.. code-block:: bash
+
+    pytest -m slow
+
+The actual command line executed is:
+
+.. code-block:: bash
+
+    pytest -ra -q -v -m slow
+
+Note that as usual for other command-line applications, in case of conflicting options the last one wins, so the example
+above will show verbose output because ``-v`` overwrites ``-q``.
+
+
 .. _request example:
 
 Pass different values to a test function, depending on command line options