Skip to content

Fix directory structure in the best practices guide on parametrizations. #208

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 26, 2022
Merged

Fix directory structure in the best practices guide on parametrizations. #208

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
847ae0f
Fix directory structure in the best practices guide on parametrizations.
tobiasraabe Jan 26, 2022
115893c
fix.
tobiasraabe Jan 26, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/source/changes.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ all releases are available on `PyPI <https://pypi.org/project/pytask>`_ and
`Anaconda.org <https://anaconda.org/conda-forge/pytask>`_.


0.1.6 - 2022-xx-xx
0.1.6 - 2022-01-27
------------------

- :gh:`191` adds a guide on how to profile pytask to the developer's guide.
Expand All @@ -23,6 +23,7 @@ all releases are available on `PyPI <https://pypi.org/project/pytask>`_ and
- :gh:`201` adds tests for ``_pytask.mark_utils``.
- :gh:`204` removes internal traceback frames from exceptions raised somewhere in
pytask.
- :gh:`208` fixes the best practices guide for parametrizations.


0.1.5 - 2022-01-10
Expand Down
65 changes: 39 additions & 26 deletions docs/source/how_to_guides/bp_parametrizations.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,37 +34,50 @@ First, let us take a look at the folder and file structure of such a project.

.. code-block::

src
│ config.py
my_project
├───pytask.ini or tox.ini or setup.cfg
│
├───data
│ data_0.csv
│ data_1.csv
│ data_2.csv
│ data_3.csv
├───src
│ └───my_project
│ ├────config.py
│ │
│ ├───data
│ │ ├────data_0.csv
│ │ ├────data_1.csv
│ │ ├────data_2.csv
│ │ └────data_3.csv
│ │
│ ├───data_preparation
│ │ ├────data_preparation_config.py
│ │ └────task_prepare_data.py
│ │
│ └───estimation
│ ├────estimation_config.py
│ └────task_estimate_models.py
│
├───data_preparation
│ data_preparation_config.py
│ task_prepare_data.py
│
└───estimation
estimation_config.py
task_estimate_models.py
├───setup.py
│
├───.pytask.sqlite3
│
└───bld



The folder structure, the general ``config.py`` which holds ``SRC`` and ``BLD`` and the
tasks follow the same structure which is advocated for throughout the tutorials.

What is new are the local configuration files in each of the subfolders of ``src`` which
contain objects which are shared across tasks. For example,
What is new are the local configuration files in each of the subfolders of
``my_project`` which contain objects which are shared across tasks. For example,
``data_preparation_config.py`` holds the paths to the processed data and the names of
the data sets.

.. code-block:: python

# Content of data_preparation_config.py

from src.config import BLD
from src.config import SRC
from my_project.config import BLD
from my_project.config import SRC


DATA = ["data_0", "data_1", "data_2", "data_3"]
Expand All @@ -87,9 +100,9 @@ parametrization.

import pytask

from src.data_preparation.data_preparation_config import DATA
from src.data_preparation.data_preparation_config import path_to_input_data
from src.data_preparation.data_preparation_config import path_to_processed_data
from my_project.data_preparation.data_preparation_config import DATA
from my_project.data_preparation.data_preparation_config import path_to_input_data
from my_project.data_preparation.data_preparation_config import path_to_processed_data


def _create_parametrization(data):
Expand Down Expand Up @@ -121,7 +134,7 @@ an explicit id.
.. code-block::

# With id
.../src/data_preparation/task_prepare_data.py::task_prepare_data[data_0]
.../my_project/data_preparation/task_prepare_data.py::task_prepare_data[data_0]

Next, we move to the estimation to see how we can build another parametrization upon the
previous one.
Expand All @@ -130,8 +143,8 @@ previous one.

# Content of estimation_config.py

from src.config import BLD
from src.data_preparation.data_preparation_config import DATA
from my_project.config import BLD
from my_project.data_preparation.data_preparation_config import DATA


_MODELS = ["linear_probability", "logistic_model", "decision_tree"]
Expand Down Expand Up @@ -164,9 +177,9 @@ And, here is the task file.

import pytask

from src.data_preparation.data_preparation_config import path_to_processed_data
from src.data_preparation.estimation_config import ESTIMATIONS
from src.data_preparation.estimation_config import path_to_estimation_result
from my_project.data_preparation.data_preparation_config import path_to_processed_data
from my_project.data_preparation.estimation_config import ESTIMATIONS
from my_project.data_preparation.estimation_config import path_to_estimation_result


def _create_parametrization(estimations):
Expand Down