Move to markdown.

tobiasraabe · tobiasraabe · commit aaa1af7dd694 · 2022-04-15T10:21:16.000+02:00
diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md
@@ -1,11 +1,9 @@
----
+______________________________________________________________________
 
-name: Bug Report
-about: Create a bug report to help us improve pytask-parallel
-title: "BUG:"
-labels: "bug"
+name: Bug Report about: Create a bug report to help us improve pytask-parallel title:
+"BUG:" labels: "bug"
 
----
+______________________________________________________________________
 
 - [ ] I have checked that this issue has not already been reported.
 
@@ -14,11 +12,11 @@ labels: "bug"
 - [ ] (optional) I have confirmed this bug exists on the `main` branch of
   pytask-parallel.
 
----
+______________________________________________________________________
 
-**Note**: Please read [this
-guide](https://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports) detailing
-how to provide the necessary information for us to reproduce your bug.
+**Note**: Please read
+[this guide](https://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports)
+detailing how to provide the necessary information for us to reproduce your bug.
 
 #### Code Sample, a copy-pastable example
 
diff --git a/.github/ISSUE_TEMPLATE/documentation.md b/.github/ISSUE_TEMPLATE/documentation.md
@@ -1,11 +1,9 @@
----
+______________________________________________________________________
 
-name: Documentation Improvement
-about: Report wrong or missing documentation
-title: "DOC:"
-labels: "documentation"
+name: Documentation Improvement about: Report wrong or missing documentation title:
+"DOC:" labels: "documentation"
 
----
+______________________________________________________________________
 
 #### Location of the documentation
 
diff --git a/.github/ISSUE_TEMPLATE/enhancement.md b/.github/ISSUE_TEMPLATE/enhancement.md
@@ -1,16 +1,14 @@
----
+______________________________________________________________________
 
-name: Enhancement
-about: Suggest an idea for pytask-parallel
-title: "ENH:"
-labels: "enhancement"
+name: Enhancement about: Suggest an idea for pytask-parallel title: "ENH:" labels:
+"enhancement"
 
----
+______________________________________________________________________
 
 #### Is your feature request related to a problem?
 
 Provide a description of what the problem is, e.g. "I wish I could use pytask-parallel
-to do [...]".
+to do \[...\]".
 
 #### Describe the solution you'd like
 
diff --git a/.github/ISSUE_TEMPLATE/question.md b/.github/ISSUE_TEMPLATE/question.md
@@ -1,17 +1,15 @@
----
+______________________________________________________________________
 
-name: Submit Question
-about: Ask a general question about pytask-parallel
-title: "QST:"
+name: Submit Question about: Ask a general question about pytask-parallel title: "QST:"
 labels: "question"
 
----
+______________________________________________________________________
 
 #### Question about pytask-parallel
 
-**Note**: If you'd still like to submit a question, please read [this guide](
-https://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports) detailing how to
-provide the necessary information for us to reproduce your question.
+**Note**: If you'd still like to submit a question, please read
+[this guide](https://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports)
+detailing how to provide the necessary information for us to reproduce your question.
 
 ```python
 # Your code here, if applicable
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
@@ -6,4 +6,4 @@ Provide a description and/or bullet points to describe the changes in this PR.
 
 - [ ] Reference issues which can be closed due to this PR with "Closes #x".
 - [ ] Review whether the documentation needs to be updated.
-- [ ] Document PR in docs/changes.rst.
+- [ ] Document PR in CHANGES.md.
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -16,9 +16,6 @@ repos:
     -   id: python-no-eval
     -   id: python-no-log-warn
     -   id: python-use-type-annotations
-    -   id: rst-backticks
-    -   id: rst-directive-colons
-    -   id: rst-inline-touching-normal
     -   id: text-unicode-replacement-char
 -   repo: https://github.com/asottile/pyupgrade
     rev: v2.32.0
@@ -64,20 +61,24 @@ repos:
             pydocstyle,
             Pygments,
         ]
--   repo: https://github.com/PyCQA/doc8
-    rev: 0.11.1
-    hooks:
-    -   id: doc8
 -   repo: https://github.com/econchick/interrogate
     rev: 1.5.0
     hooks:
     -   id: interrogate
         args: [-v, --fail-under=40, src, tests]
+-   repo: https://github.com/executablebooks/mdformat
+    rev: 0.7.14
+    hooks:
+    -   id: mdformat
+        additional_dependencies: [
+            mdformat-gfm,
+            mdformat-black,
+        ]
+        args: [--wrap, "88"]
 - repo: https://github.com/codespell-project/codespell
   rev: v2.1.0
   hooks:
     -   id: codespell
-        args: [-L unparseable]
 -   repo: https://github.com/mgedmin/check-manifest
     rev: "0.48"
     hooks:
diff --git a/CHANGES.md b/CHANGES.md
@@ -0,0 +1,62 @@
+# Changes
+
+This is a record of all past pytask-parallel releases and what went into them in reverse
+chronological order. Releases follow [semantic versioning](https://semver.org/) and all
+releases are available on [PyPI](https://pypi.org/project/pytask-parallel) and
+[Anaconda.org](https://anaconda.org/conda-forge/pytask-parallel).
+
+## 0.1.2 - 2022-xx-xx
+
+- {pull}`36` adds a test for <https://github.com/pytask-dev/pytask/issues/216>.
+
+## 0.1.1 - 2022-02-08
+
+- {pull}`30` removes unnecessary content from `tox.ini`.
+- {pull}`33` skips concurrent CI builds.
+- {pull}`34` deprecates Python 3.6 and adds support for Python 3.10.
+
+## 0.1.0 - 2021-07-20
+
+- {pull}`19` adds `conda-forge` to the `README.rst`.
+- {pull}`22` add note that the debugger cannot be used together with pytask-parallel.
+- {pull}`24` replaces versioneer with setuptools-scm.
+- {pull}`25` aborts build and prints reports on `KeyboardInterrupt`.
+- {pull}`27` enables rich tracebacks from subprocesses.
+
+## 0.0.8 - 2021-03-05
+
+- {pull}`17` fixes the unidentifiable version.
+
+## 0.0.7 - 2021-03-04
+
+- {pull}`14` fixes some post-release issues.
+- {pull}`16` add dependencies to `setup.py` and changes the default backend to `loky`.
+
+## 0.0.6 - 2021-02-27
+
+- {pull}`12` replaces all occurrences of `n_processes` with `n_workers`.
+- {pull}`13` adds a license, versioneer, and allows publishing on PyPI.
+
+## 0.0.5 - 2020-12-28
+
+- {pull}`5` fixes the CI and other smaller issues.
+- {pull}`8` aligns pytask-parallel with task priorities in pytask v0.0.11.
+- {pull}`9` enables --max-failures. Closes {issue}`7`.
+- {pull}`10` releases v0.0.5.
+
+## 0.0.4 - 2020-10-30
+
+- {pull}`4` implement an executor with `loky`.
+
+## 0.0.3 - 2020-09-12
+
+- {pull}`3` align the program with pytask v0.0.6.
+
+## 0.0.2 - 2020-08-12
+
+- {pull}`1` prepares the plugin for pytask v0.0.5.
+- {pull}`2` better parsing and callbacks.
+
+## 0.0.1 - 2020-07-17
+
+- Initial commit which combined the whole effort to release v0.0.1.
diff --git a/CHANGES.rst b/CHANGES.rst
diff --git a/MANIFEST.in b/MANIFEST.in
@@ -1,11 +1,9 @@
 prune tests
 
-exclude *.rst
+exclude *.md
 exclude *.yaml
 exclude *.yml
 exclude tox.ini
 
-include README.rst
+include README.md
 include LICENSE
-include versioneer.py
-include src/pytask_parallel/_version.py
diff --git a/README.md b/README.md
@@ -0,0 +1,97 @@
+# pytask-parallel
+
+[![PyPI](https://img.shields.io/pypi/v/pytask-parallel?color=blue)](https://pypi.org/project/pytask-parallel)
+[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/pytask-parallel)](https://pypi.org/project/pytask-parallel)
+[![image](https://img.shields.io/conda/vn/conda-forge/pytask-parallel.svg)](https://anaconda.org/conda-forge/pytask-parallel)
+[![image](https://img.shields.io/conda/pn/conda-forge/pytask-parallel.svg)](https://anaconda.org/conda-forge/pytask-parallel)
+[![PyPI - License](https://img.shields.io/pypi/l/pytask-parallel)](https://pypi.org/project/pytask-parallel)
+[![image](https://img.shields.io/github/workflow/status/pytask-dev/pytask-parallel/Continuous%20Integration%20Workflow/main)](https://github.com/pytask-dev/pytask-parallel/actions?query=branch%3Amain)
+[![image](https://codecov.io/gh/pytask-dev/pytask-parallel/branch/main/graph/badge.svg)](https://codecov.io/gh/pytask-dev/pytask-parallel)
+[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/pytask-dev/pytask-parallel/main.svg)](https://results.pre-commit.ci/latest/github/pytask-dev/pytask-parallel/main)
+[![image](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
+
+______________________________________________________________________
+
+Parallelize the execution of tasks with `pytask-parallel` which is a plugin for
+[pytask](https://github.com/pytask-dev/pytask).
+
+## Installation
+
+pytask-parallel is available on [PyPI](https://pypi.org/project/pytask-parallel) and
+[Anaconda.org](https://anaconda.org/conda-forge/pytask-parallel). Install it with
+
+```console
+$ pip install pytask-parallel
+
+# or
+
+$ conda install -c conda-forge pytask-parallel
+```
+
+By default, the plugin uses `loky`'s robust implementation of the `ProcessPoolExecutor`.
+
+It is also possible to select the `ProcessPoolExecutor` or `ThreadPoolExecutor` from the
+[concurrent.futures](https://docs.python.org/3/library/concurrent.futures.html) module
+as backends to execute tasks asynchronously.
+
+## Usage
+
+To parallelize your tasks across many workers, pass an integer greater than 1 or
+`'auto'` to the command-line interface.
+
+```console
+$ pytask -n 2
+$ pytask --n-workers 2
+
+# Starts os.cpu_count() - 1 workers.
+$ pytask -n auto
+```
+
+Using processes to parallelize the execution of tasks is useful for CPU bound tasks such
+as numerical computations. ([Here](https://stackoverflow.com/a/868577/7523785) is an
+explanation on what CPU or IO bound means.)
+
+For IO bound tasks, tasks where the limiting factor are network responses, access to
+files, you can parallelize via threads.
+
+```console
+$ pytask --parallel-backend threads
+```
+
+You can also set the options in one of the configuration files (`pytask.ini`, `tox.ini`,
+or `setup.cfg`).
+
+```ini
+# This is the default configuration. Note that, parallelization is turned off.
+
+[pytask]
+n_workers = 1
+parallel_backend = loky  # or processes or threads
+```
+
+## Warning
+
+It is not possible to combine parallelization with debugging. That is why `--pdb` or
+`--trace` deactivate parallelization.
+
+If you parallelize the execution of your tasks using two or more workers, do not use
+`breakpoint()` or `import pdb; pdb.set_trace()` since both will cause exceptions.
+
+## Changes
+
+Consult the [release notes](CHANGES.md) to find out about what is new.
+
+## Development
+
+- `pytask-parallel` does not call the `pytask_execute_task_protocol` hook
+  specification/entry-point because `pytask_execute_task_setup` and
+  `pytask_execute_task` need to be separated from `pytask_execute_task_teardown`. Thus,
+  plugins which change this hook specification may not interact well with the
+  parallelization.
+
+- There are two PRs for CPython which try to re-enable setting custom reducers which
+  should have been working, but does not. Here are the references.
+
+  > - <https://bugs.python.org/issue28053>
+  > - <https://github.com/python/cpython/pull/9959>
+  > - <https://github.com/python/cpython/pull/15058>
diff --git a/README.rst b/README.rst
diff --git a/setup.cfg b/setup.cfg
