From 020fa6596a70572c3f7c03100d99c5a7864adc1c Mon Sep 17 00:00:00 2001 From: Jerry Guo Date: Tue, 12 Mar 2024 09:50:42 +0100 Subject: [PATCH 01/11] Quick patch on remaining issues on the docs. ToDo: The hierarchy of batch datatypes is rather arbitrary and non-consistent, should we consider change that in the future? Signed-off-by: Jerry Guo --- docs/user_manual/dataset-terminology.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/user_manual/dataset-terminology.md b/docs/user_manual/dataset-terminology.md index 3df005dbcf..0527dc7b12 100644 --- a/docs/user_manual/dataset-terminology.md +++ b/docs/user_manual/dataset-terminology.md @@ -12,9 +12,9 @@ Some terms regarding the data structures are explained here, including the defin - **Dataset:** Either a single or a batch dataset. - **SingleDataset:** A data type storing input data (i.e. all elements of all components) for a single scenario. - - **BatchDataset:** A data type storing update and or output data for one or more scenarios. A batch dataset can be either a sparse or a dense one. - - **SparseBatchDataset:** Dictionaries with a one-dimensional numpy int32 array and a one-dimensional structured numpy arrays. - - **DenseBatchDataset:** A dictionary where the keys are the component types and the values are two-dimensional structured numpy arrays. + - **BatchDataset:** A data type storing update and or output data for one or more scenarios. A batch dataset can contain sparse or dense data, depending on the component. + - **BatchArray:** An array of dictionaries where the keys are the component types and the values are two-dimensional structured numpy arrays. + - **SparseBatchArray:** A dictionary with a one-dimensional numpy int64 array and a one-dimensional structured numpy arrays. ### Type of Dataset From 3f0965ebf24134626c9859c9982123458d768337 Mon Sep 17 00:00:00 2001 From: Jerry Guo Date: Tue, 12 Mar 2024 10:26:59 +0100 Subject: [PATCH 02/11] Added several data types: `DataArray`, `DenseBatchArray` and `SingleArray`. For ease, now `SingleArray` and `SingleDataset` are synonym, otherwise we have to change much more that necessary. Signed-off-by: Jerry Guo --- docs/user_manual/dataset-terminology.md | 7 +++++-- src/power_grid_model/data_types.py | 23 +++++++++++++++++++++-- 2 files changed, 26 insertions(+), 4 deletions(-) diff --git a/docs/user_manual/dataset-terminology.md b/docs/user_manual/dataset-terminology.md index 0527dc7b12..4649aaf657 100644 --- a/docs/user_manual/dataset-terminology.md +++ b/docs/user_manual/dataset-terminology.md @@ -13,8 +13,11 @@ Some terms regarding the data structures are explained here, including the defin - **Dataset:** Either a single or a batch dataset. - **SingleDataset:** A data type storing input data (i.e. all elements of all components) for a single scenario. - **BatchDataset:** A data type storing update and or output data for one or more scenarios. A batch dataset can contain sparse or dense data, depending on the component. - - **BatchArray:** An array of dictionaries where the keys are the component types and the values are two-dimensional structured numpy arrays. - - **SparseBatchArray:** A dictionary with a one-dimensional numpy int64 array and a one-dimensional structured numpy arrays. +- **DataArray** A data array can be a single or a batch array. It is a numpy structured array. + - **SingleArray** A dictionary where the keys are the component types and the values are one-dimensional structured numpy arrays. + - **BatchArray:** An array of dictionaries where the keys are the component types and the values are two-dimensional structured numpy arrays. + - **DenseBatchArray:** A two-dimensional structured numpy array containing a list of components of the same type for each scenario. + - **SparseBatchArray:** A dictionary with a one-dimensional numpy int64 array and a one-dimensional structured numpy arrays. ### Type of Dataset diff --git a/src/power_grid_model/data_types.py b/src/power_grid_model/data_types.py index 1236e074fe..4ee70892dd 100644 --- a/src/power_grid_model/data_types.py +++ b/src/power_grid_model/data_types.py @@ -22,7 +22,13 @@ - Example: {"indptr": <1d-array>, "data": <1d-array>} """ -BatchArray = Union[np.ndarray, SparseBatchArray] +DenseBatchArray = Union[np.ndarray] +""" +A dense batch array is a two-dimensional structured numpy array containing a list of components of +the same type for each scenario. +""" + +BatchArray = Union[DenseBatchArray, SparseBatchArray] """ A batch is a either a dense or a sparse batch array. @@ -33,7 +39,20 @@ sparse: {"indptr": <1d-array>, "data": <1d-array>} """ -SingleDataset = Dict[str, np.ndarray] +SingleArray = Dict[str, np.ndarray] +""" +A single array is a dictionary where the keys are the component types and the values are one-dimensional +structured numpy arrays. + +- Example: {"node": <1d-array>, "line": <1d-array>} +""" + +DataArray = Union[SingleArray, BatchArray] +""" +A data array can be a single or a batch array. It is a numpy structured array. +""" + +SingleDataset = Union[SingleArray] """ A single dataset is a dictionary where the keys are the component types and the values are one-dimensional structured numpy arrays. From 8dbb6df4d0ee35ea0c78e8663604fbdf1fff1bc7 Mon Sep 17 00:00:00 2001 From: Martijn Govers Date: Tue, 12 Mar 2024 15:04:12 +0100 Subject: [PATCH 03/11] update API references Signed-off-by: Martijn Govers --- docs/api_reference/python-api-reference.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/api_reference/python-api-reference.md b/docs/api_reference/python-api-reference.md index 56b4dd6395..842a49fa0a 100644 --- a/docs/api_reference/python-api-reference.md +++ b/docs/api_reference/python-api-reference.md @@ -31,6 +31,12 @@ SPDX-License-Identifier: MPL-2.0 .. autofunction:: power_grid_model.validation.errors_to_string ``` +### errors + +```{eval-rst} +.. autoclass:: power_grid_model.validation.errors.ValidationError +``` + ## data types ```{eval-rst} @@ -51,12 +57,6 @@ SPDX-License-Identifier: MPL-2.0 .. autoclass:: power_grid_model.data_types.PythonDataset ``` -### errors - -```{eval-rst} -.. autoclass:: power_grid_model.validation.errors.ValidationError -``` - ## utils ```{eval-rst} @@ -72,6 +72,6 @@ SPDX-License-Identifier: MPL-2.0 .. autofunction:: power_grid_model.utils.msgpack_serialize_to_file .. autofunction:: power_grid_model.utils.import_json_data .. autofunction:: power_grid_model.utils.export_json_data -.. autofunction:: power_grid_model._utils.get_and_verify_batch_sizes -.. autofunction:: power_grid_model._utils.get_batch_size +.. autofunction:: power_grid_model.utils.get_and_verify_batch_sizes +.. autofunction:: power_grid_model.utils.get_batch_size ``` From 238c529fa257630655ec3a99d81006b46e1de3cf Mon Sep 17 00:00:00 2001 From: Martijn Govers Date: Tue, 12 Mar 2024 15:57:07 +0100 Subject: [PATCH 04/11] more elaborate documentation on sparse data Signed-off-by: Martijn Govers --- docs/api_reference/python-api-reference.md | 4 +- src/power_grid_model/data_types.py | 99 +++++++++++++--------- 2 files changed, 60 insertions(+), 43 deletions(-) diff --git a/docs/api_reference/python-api-reference.md b/docs/api_reference/python-api-reference.md index 842a49fa0a..760029cbc5 100644 --- a/docs/api_reference/python-api-reference.md +++ b/docs/api_reference/python-api-reference.md @@ -72,6 +72,6 @@ SPDX-License-Identifier: MPL-2.0 .. autofunction:: power_grid_model.utils.msgpack_serialize_to_file .. autofunction:: power_grid_model.utils.import_json_data .. autofunction:: power_grid_model.utils.export_json_data -.. autofunction:: power_grid_model.utils.get_and_verify_batch_sizes -.. autofunction:: power_grid_model.utils.get_batch_size +.. autofunction:: power_grid_model._utils.get_and_verify_batch_sizes +.. autofunction:: power_grid_model._utils.get_batch_size ``` diff --git a/src/power_grid_model/data_types.py b/src/power_grid_model/data_types.py index 4ee70892dd..16ac44b618 100644 --- a/src/power_grid_model/data_types.py +++ b/src/power_grid_model/data_types.py @@ -13,13 +13,13 @@ # When we're dropping python 3.8, we should introduce proper NumPy type hinting -SparseBatchArray = Dict[str, np.ndarray] +SingleArray = Union[np.ndarray] """ -A sparse batch array is a dictionary containing the keys `indptr` and `data`. +A single array is a one-dimensional structured containing a list of components of the same type. -- indptr: a one-dimensional numpy int64 array -- data: a one-dimensional structured numpy array. The exact dtype depends on the type of component. -- Example: {"indptr": <1d-array>, "data": <1d-array>} +- Examples: + - structure: <1d-array> + - concrete example: array([(0, 10500.0), (0, 10500.0)], dtype=power_grid_meta_data["input"]["node"].dtype) """ DenseBatchArray = Union[np.ndarray] @@ -28,6 +28,24 @@ the same type for each scenario. """ +SparseBatchArray = Dict[str, np.ndarray] +""" +A sparse batch array is a dictionary containing the keys `indptr` and `data`. + +- data: a one-dimensional structured numpy array. The exact dtype depends on the type of component. +- indptr: a one-dimensional numpy int64 array containing n+1 elements where n is the amount of scenarios. + - The elements are the indices in the data that point to the first element of that scenario. + - The last element is one after the data index of the last element of the last scenario. + - Usually, the last element will therefore be the size of the data. + +- Examples: + - structure: {"indptr": <1d-array>, "data": SingleArray} + - concrete example: {"indptr": [0, 2, 2, 3], "data": [(0, 1, 1), (1, 1, 1), (0, 0, 0)]} + - the scenario 0 sets the statuses of components ids 0 and 1 to 1 (and keeps defaults for other components) + - scenario 1 keeps the default values for all components + - scenario 2 sets the statuses of component with id 0 to 0 (and keeps defaults for other components) +""" + BatchArray = Union[DenseBatchArray, SparseBatchArray] """ A batch is a either a dense or a sparse batch array. @@ -36,15 +54,7 @@ dense: <2d-array> -sparse: {"indptr": <1d-array>, "data": <1d-array>} -""" - -SingleArray = Dict[str, np.ndarray] -""" -A single array is a dictionary where the keys are the component types and the values are one-dimensional -structured numpy arrays. - -- Example: {"node": <1d-array>, "line": <1d-array>} +sparse: {"indptr": SingleArray, "data": SingleArray} """ DataArray = Union[SingleArray, BatchArray] @@ -57,7 +67,7 @@ A single dataset is a dictionary where the keys are the component types and the values are one-dimensional structured numpy arrays. -- Example: {"node": <1d-array>, "line": <1d-array>} +- Example: {"node": SingleArray, "line": SingleArray} """ BatchDataset = Dict[str, BatchArray] @@ -66,7 +76,7 @@ structured numpy arrays (dense batch array) or dictionaries with an indptr and a one-dimensional structured numpy array (sparse batch array). -- Example: {"node": <2d-array>, "line": {"indptr": <1d-array>, "data": <1d-array>}} +- Example: {"node": <2d-array>, "line": {"indptr": SingleArray, "data": SingleArray}} """ Dataset = Union[SingleDataset, BatchDataset] @@ -74,10 +84,8 @@ A general data set can be a single or a batch dataset. - Examples: - -single: {"node": <1d-array>, "line": <1d-array>} - -batch: {"node": <2d-array>, "line": {"indptr": <1d-array>, "data": <1d-array>}} + - single: {"node": SingleArray, "line": SingleArray} + - batch: {"node": <2d-array>, "line": {"indptr": SingleArray, "data": SingleArray}} """ BatchList = List[SingleDataset] @@ -85,7 +93,7 @@ A batch list is an alternative representation of a batch. It is a list of single datasets, where each single dataset is actually a batch. The batch list is intended as an intermediate data type, during conversions. -- Example: [{"node": <1d-array>, "line": <1d-array>}, {"node": <1d-array>, "line": <1d-array>}] +- Example: [{"node": SingleArray, "line": SingleArray}, {"node": SingleArray, "line": SingleArray}] """ NominalValue = int @@ -115,12 +123,9 @@ a real value, or a tuple of three real values. - Examples: - -real: 10500.0 - -nominal: 123 - -asym: (10400.0, 10500.0, 10600.0) + - real: 10500.0 + - nominal: 123 + - asym: (10400.0, 10500.0, 10600.0) """ Component = Dict[str, Union[AttributeValue, str]] @@ -147,9 +152,10 @@ - Example: - {"node": [{"id": 1, "u_rated": 10500.0}, {"id": 2, "u_rated": 10500.0}], - - "line": [{"id": 3, "from_node": 1, "to_node": 2, ...}],} + { + "node": [{"id": 1, "u_rated": 10500.0}, {"id": 2, "u_rated": 10500.0}], + "line": [{"id": 3, "from_node": 1, "to_node": 2, ...}], + } """ BatchPythonDataset = List[SinglePythonDataset] @@ -160,9 +166,14 @@ - Example: - [{"line": [{"id": 3, "from_status": 0, "to_status": 0, ...}],}, - - {"line": [{"id": 3, "from_status": 1, "to_status": 1, ...}],}] + [ + { + "line": [{"id": 3, "from_status": 0, "to_status": 0, ...}], + }, + { + "line": [{"id": 3, "from_status": 1, "to_status": 1, ...}], + } + ] """ PythonDataset = Union[SinglePythonDataset, BatchPythonDataset] @@ -171,15 +182,21 @@ - Examples: - single: - - {"node": [{"id": 1, "u_rated": 10500.0}, {"id": 2, "u_rated": 10500.0}], - - "line": [{"id": 3, "from_node": 1, "to_node": 2, ...}],} + - single: - batch: + { + "node": [{"id": 1, "u_rated": 10500.0}, {"id": 2, "u_rated": 10500.0}], + "line": [{"id": 3, "from_node": 1, "to_node": 2, ...}], + } - [{"line": [{"id": 3, "from_status": 0, "to_status": 0, ...}],}, + - batch: - {"line": [{"id": 3, "from_status": 1, "to_status": 1, ...}],}] + [ + { + "line": [{"id": 3, "from_status": 0, "to_status": 0, ...}], + }, + { + "line": [{"id": 3, "from_status": 1, "to_status": 1, ...}], + } + ] """ From 166ac77b628c7dd14324ffda784b809d82ff677d Mon Sep 17 00:00:00 2001 From: Martijn Govers Date: Tue, 12 Mar 2024 16:10:08 +0100 Subject: [PATCH 05/11] improve layout Signed-off-by: Martijn Govers --- docs/api_reference/python-api-reference.md | 19 +++----- src/power_grid_model/data_types.py | 56 ++++++++-------------- 2 files changed, 27 insertions(+), 48 deletions(-) diff --git a/docs/api_reference/python-api-reference.md b/docs/api_reference/python-api-reference.md index 760029cbc5..cce803e5dc 100644 --- a/docs/api_reference/python-api-reference.md +++ b/docs/api_reference/python-api-reference.md @@ -40,21 +40,14 @@ SPDX-License-Identifier: MPL-2.0 ## data types ```{eval-rst} -.. autoclass:: power_grid_model.data_types.SparseBatchArray -.. autoclass:: power_grid_model.data_types.BatchArray +.. autoclass:: power_grid_model.data_types.Dataset .. autoclass:: power_grid_model.data_types.SingleDataset .. autoclass:: power_grid_model.data_types.BatchDataset -.. autoclass:: power_grid_model.data_types.Dataset -.. autoclass:: power_grid_model.data_types.BatchList -.. autoclass:: power_grid_model.data_types.NominalValue -.. autoclass:: power_grid_model.data_types.RealValue -.. autoclass:: power_grid_model.data_types.AsymValue -.. autoclass:: power_grid_model.data_types.AttributeValue -.. autoclass:: power_grid_model.data_types.Component -.. autoclass:: power_grid_model.data_types.ComponentList -.. autoclass:: power_grid_model.data_types.SinglePythonDataset -.. autoclass:: power_grid_model.data_types.BatchPythonDataset -.. autoclass:: power_grid_model.data_types.PythonDataset +.. autoclass:: power_grid_model.data_types.DataArray +.. autoclass:: power_grid_model.data_types.SingleArray +.. autoclass:: power_grid_model.data_types.BatchArray +.. autoclass:: power_grid_model.data_types.SparseBatchArray +.. autoclass:: power_grid_model.data_types.DenseBatchArray ``` ## utils diff --git a/src/power_grid_model/data_types.py b/src/power_grid_model/data_types.py index 16ac44b618..16bfa1868a 100644 --- a/src/power_grid_model/data_types.py +++ b/src/power_grid_model/data_types.py @@ -18,8 +18,8 @@ A single array is a one-dimensional structured containing a list of components of the same type. - Examples: - - structure: <1d-array> - - concrete example: array([(0, 10500.0), (0, 10500.0)], dtype=power_grid_meta_data["input"]["node"].dtype) + - structure: <1d-array> + - concrete: array([(0, 10500.0), (0, 10500.0)], dtype=power_grid_meta_data["input"]["node"].dtype) """ DenseBatchArray = Union[np.ndarray] @@ -34,16 +34,16 @@ - data: a one-dimensional structured numpy array. The exact dtype depends on the type of component. - indptr: a one-dimensional numpy int64 array containing n+1 elements where n is the amount of scenarios. - - The elements are the indices in the data that point to the first element of that scenario. - - The last element is one after the data index of the last element of the last scenario. - - Usually, the last element will therefore be the size of the data. + - The elements are the indices in the data that point to the first element of that scenario. + - The last element is one after the data index of the last element of the last scenario. + - Usually, the last element will therefore be the size of the data. - Examples: - - structure: {"indptr": <1d-array>, "data": SingleArray} - - concrete example: {"indptr": [0, 2, 2, 3], "data": [(0, 1, 1), (1, 1, 1), (0, 0, 0)]} - - the scenario 0 sets the statuses of components ids 0 and 1 to 1 (and keeps defaults for other components) - - scenario 1 keeps the default values for all components - - scenario 2 sets the statuses of component with id 0 to 0 (and keeps defaults for other components) + - structure: {"indptr": <1d-array>, "data": SingleArray} + - concrete example: {"indptr": [0, 2, 2, 3], "data": [(0, 1, 1), (1, 1, 1), (0, 0, 0)]} + - the scenario 0 sets the statuses of components ids 0 and 1 to 1 (and keeps defaults for other components) + - scenario 1 keeps the default values for all components + - scenario 2 sets the statuses of component with id 0 to 0 (and keeps defaults for other components) """ BatchArray = Union[DenseBatchArray, SparseBatchArray] @@ -51,10 +51,8 @@ A batch is a either a dense or a sparse batch array. - Examples: - -dense: <2d-array> - -sparse: {"indptr": SingleArray, "data": SingleArray} + - dense: <2d-array> + - sparse: {"indptr": SingleArray, "data": SingleArray} """ DataArray = Union[SingleArray, BatchArray] @@ -84,8 +82,8 @@ A general data set can be a single or a batch dataset. - Examples: - - single: {"node": SingleArray, "line": SingleArray} - - batch: {"node": <2d-array>, "line": {"indptr": SingleArray, "data": SingleArray}} + - single: {"node": SingleArray, "line": SingleArray} + - batch: {"node": <2d-array>, "line": {"indptr": SingleArray, "data": SingleArray}} """ BatchList = List[SingleDataset] @@ -123,9 +121,9 @@ a real value, or a tuple of three real values. - Examples: - - real: 10500.0 - - nominal: 123 - - asym: (10400.0, 10500.0, 10600.0) + - real: 10500.0 + - nominal: 123 + - asym: (10400.0, 10500.0, 10600.0) """ Component = Dict[str, Union[AttributeValue, str]] @@ -166,14 +164,8 @@ - Example: - [ - { - "line": [{"id": 3, "from_status": 0, "to_status": 0, ...}], - }, - { - "line": [{"id": 3, "from_status": 1, "to_status": 1, ...}], - } - ] + [{"line": [{"id": 3, "from_status": 0, "to_status": 0, ...}],}, + {"line": [{"id": 3, "from_status": 1, "to_status": 1, ...}],}] """ PythonDataset = Union[SinglePythonDataset, BatchPythonDataset] @@ -191,12 +183,6 @@ - batch: - [ - { - "line": [{"id": 3, "from_status": 0, "to_status": 0, ...}], - }, - { - "line": [{"id": 3, "from_status": 1, "to_status": 1, ...}], - } - ] + [{"line": [{"id": 3, "from_status": 0, "to_status": 0, ...}],}, + {"line": [{"id": 3, "from_status": 1, "to_status": 1, ...}],}] """ From 16632827002ef42697cb80f7de5786d735b7fb12 Mon Sep 17 00:00:00 2001 From: Martijn Govers Date: Tue, 12 Mar 2024 16:14:16 +0100 Subject: [PATCH 06/11] improve order Signed-off-by: Martijn Govers --- docs/api_reference/python-api-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/api_reference/python-api-reference.md b/docs/api_reference/python-api-reference.md index cce803e5dc..8e059a35d4 100644 --- a/docs/api_reference/python-api-reference.md +++ b/docs/api_reference/python-api-reference.md @@ -46,8 +46,8 @@ SPDX-License-Identifier: MPL-2.0 .. autoclass:: power_grid_model.data_types.DataArray .. autoclass:: power_grid_model.data_types.SingleArray .. autoclass:: power_grid_model.data_types.BatchArray -.. autoclass:: power_grid_model.data_types.SparseBatchArray .. autoclass:: power_grid_model.data_types.DenseBatchArray +.. autoclass:: power_grid_model.data_types.SparseBatchArray ``` ## utils From 29dd47c06fcfb2d649c0713cee717520d89b1a68 Mon Sep 17 00:00:00 2001 From: Martijn Govers Date: Wed, 13 Mar 2024 07:57:37 +0100 Subject: [PATCH 07/11] improve cross-referencing Signed-off-by: Martijn Govers --- src/power_grid_model/data_types.py | 34 ++++++++++++------------------ 1 file changed, 14 insertions(+), 20 deletions(-) diff --git a/src/power_grid_model/data_types.py b/src/power_grid_model/data_types.py index 16bfa1868a..12695df5b8 100644 --- a/src/power_grid_model/data_types.py +++ b/src/power_grid_model/data_types.py @@ -28,18 +28,18 @@ the same type for each scenario. """ -SparseBatchArray = Dict[str, np.ndarray] +SparseBatchArray = Dict[str, Union[np.ndarray, SingleArray]] """ A sparse batch array is a dictionary containing the keys `indptr` and `data`. -- data: a one-dimensional structured numpy array. The exact dtype depends on the type of component. +- data: a :class:`SingleArray`. The exact dtype depends on the type of component. - indptr: a one-dimensional numpy int64 array containing n+1 elements where n is the amount of scenarios. - The elements are the indices in the data that point to the first element of that scenario. - The last element is one after the data index of the last element of the last scenario. - Usually, the last element will therefore be the size of the data. - Examples: - - structure: {"indptr": <1d-array>, "data": SingleArray} + - structure: {"indptr": <1d-array>, "data": :class:`SingleArray`} - concrete example: {"indptr": [0, 2, 2, 3], "data": [(0, 1, 1), (1, 1, 1), (0, 0, 0)]} - the scenario 0 sets the statuses of components ids 0 and 1 to 1 (and keeps defaults for other components) - scenario 1 keeps the default values for all components @@ -48,42 +48,36 @@ BatchArray = Union[DenseBatchArray, SparseBatchArray] """ -A batch is a either a dense or a sparse batch array. - -- Examples: - - dense: <2d-array> - - sparse: {"indptr": SingleArray, "data": SingleArray} +A batch array is a either a :class:`DenseBatchArray` or a :class:`SparseBatchArray`. """ DataArray = Union[SingleArray, BatchArray] """ -A data array can be a single or a batch array. It is a numpy structured array. +A data array can be a :class:`SingleArray` or a :class:`BatchArray`. """ SingleDataset = Union[SingleArray] """ -A single dataset is a dictionary where the keys are the component types and the values are one-dimensional -structured numpy arrays. +A single dataset is a dictionary where the keys are the component types and the values are +:class:`SingleArray`s -- Example: {"node": SingleArray, "line": SingleArray} +- Example: {"node": :class:`SingleArray`, "line": :class:`SingleArray`} """ BatchDataset = Dict[str, BatchArray] """ -A batch dataset is a dictionary where the keys are the component types and the values are either two-dimensional -structured numpy arrays (dense batch array) or dictionaries with an indptr and a one-dimensional structured numpy -array (sparse batch array). +A batch dataset is a dictionary where the keys are the component types and the values are :class:`BatchArray`s -- Example: {"node": <2d-array>, "line": {"indptr": SingleArray, "data": SingleArray}} +- Example: {"node": :class:`DenseBatchArray`, "line": :class:`SparseBatchArray`} """ Dataset = Union[SingleDataset, BatchDataset] """ -A general data set can be a single or a batch dataset. +A general data set can be a :class:`SingleDataset` or a :class:`BatchDataset`. - Examples: - - single: {"node": SingleArray, "line": SingleArray} - - batch: {"node": <2d-array>, "line": {"indptr": SingleArray, "data": SingleArray}} + - single: {"node": :class:`SingleArray`, "line": :class:`SingleArray`} + - batch: {"node": :class:`DenseBatchArray`, "line": :class:`SparseBatchArray`} """ BatchList = List[SingleDataset] @@ -91,7 +85,7 @@ A batch list is an alternative representation of a batch. It is a list of single datasets, where each single dataset is actually a batch. The batch list is intended as an intermediate data type, during conversions. -- Example: [{"node": SingleArray, "line": SingleArray}, {"node": SingleArray, "line": SingleArray}] +- Example: [:class:`SingleDataset`, {"node": :class:`SingleDataset`}] """ NominalValue = int From 18f051ec32bc64a0911987fbfaabd83d3372a32d Mon Sep 17 00:00:00 2001 From: Martijn Govers Date: Wed, 13 Mar 2024 08:42:19 +0100 Subject: [PATCH 08/11] attempt fix format Signed-off-by: Martijn Govers --- src/power_grid_model/data_types.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/power_grid_model/data_types.py b/src/power_grid_model/data_types.py index 12695df5b8..68786f9e22 100644 --- a/src/power_grid_model/data_types.py +++ b/src/power_grid_model/data_types.py @@ -33,6 +33,7 @@ A sparse batch array is a dictionary containing the keys `indptr` and `data`. - data: a :class:`SingleArray`. The exact dtype depends on the type of component. + - indptr: a one-dimensional numpy int64 array containing n+1 elements where n is the amount of scenarios. - The elements are the indices in the data that point to the first element of that scenario. - The last element is one after the data index of the last element of the last scenario. @@ -40,6 +41,7 @@ - Examples: - structure: {"indptr": <1d-array>, "data": :class:`SingleArray`} + - concrete example: {"indptr": [0, 2, 2, 3], "data": [(0, 1, 1), (1, 1, 1), (0, 0, 0)]} - the scenario 0 sets the statuses of components ids 0 and 1 to 1 (and keeps defaults for other components) - scenario 1 keeps the default values for all components @@ -115,7 +117,7 @@ a real value, or a tuple of three real values. - Examples: - - real: 10500.0 + - real: 10500.0 - nominal: 123 - asym: (10400.0, 10500.0, 10600.0) """ From aac7fd39599758a9ab3a38d840178d5599bdca0b Mon Sep 17 00:00:00 2001 From: Martijn Govers Date: Wed, 13 Mar 2024 08:44:26 +0100 Subject: [PATCH 09/11] fix Signed-off-by: Martijn Govers --- src/power_grid_model/data_types.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/power_grid_model/data_types.py b/src/power_grid_model/data_types.py index 68786f9e22..56168cf5dd 100644 --- a/src/power_grid_model/data_types.py +++ b/src/power_grid_model/data_types.py @@ -61,14 +61,14 @@ SingleDataset = Union[SingleArray] """ A single dataset is a dictionary where the keys are the component types and the values are -:class:`SingleArray`s +:class:`SingleArray` - Example: {"node": :class:`SingleArray`, "line": :class:`SingleArray`} """ BatchDataset = Dict[str, BatchArray] """ -A batch dataset is a dictionary where the keys are the component types and the values are :class:`BatchArray`s +A batch dataset is a dictionary where the keys are the component types and the values are :class:`BatchArray` - Example: {"node": :class:`DenseBatchArray`, "line": :class:`SparseBatchArray`} """ From 5e4b1150e40327195c254fa14d936debe64cb08a Mon Sep 17 00:00:00 2001 From: Martijn Govers Date: Wed, 13 Mar 2024 09:24:42 +0100 Subject: [PATCH 10/11] fix Signed-off-by: Martijn Govers --- src/power_grid_model/data_types.py | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/src/power_grid_model/data_types.py b/src/power_grid_model/data_types.py index 56168cf5dd..42b2c812e9 100644 --- a/src/power_grid_model/data_types.py +++ b/src/power_grid_model/data_types.py @@ -18,6 +18,7 @@ A single array is a one-dimensional structured containing a list of components of the same type. - Examples: + - structure: <1d-array> - concrete: array([(0, 10500.0), (0, 10500.0)], dtype=power_grid_meta_data["input"]["node"].dtype) """ @@ -33,16 +34,17 @@ A sparse batch array is a dictionary containing the keys `indptr` and `data`. - data: a :class:`SingleArray`. The exact dtype depends on the type of component. - - indptr: a one-dimensional numpy int64 array containing n+1 elements where n is the amount of scenarios. + - The elements are the indices in the data that point to the first element of that scenario. - The last element is one after the data index of the last element of the last scenario. - Usually, the last element will therefore be the size of the data. - Examples: - - structure: {"indptr": <1d-array>, "data": :class:`SingleArray`} + - structure: {"indptr": <1d-array>, "data": :class:`SingleArray`} - concrete example: {"indptr": [0, 2, 2, 3], "data": [(0, 1, 1), (1, 1, 1), (0, 0, 0)]} + - the scenario 0 sets the statuses of components ids 0 and 1 to 1 (and keeps defaults for other components) - scenario 1 keeps the default values for all components - scenario 2 sets the statuses of component with id 0 to 0 (and keeps defaults for other components) @@ -78,6 +80,7 @@ A general data set can be a :class:`SingleDataset` or a :class:`BatchDataset`. - Examples: + - single: {"node": :class:`SingleArray`, "line": :class:`SingleArray`} - batch: {"node": :class:`DenseBatchArray`, "line": :class:`SparseBatchArray`} """ @@ -117,6 +120,7 @@ a real value, or a tuple of three real values. - Examples: + - real: 10500.0 - nominal: 123 - asym: (10400.0, 10500.0, 10600.0) From 0df9998f2de6c9ddf3de81ed15d3df12d420647c Mon Sep 17 00:00:00 2001 From: Martijn Govers Date: Wed, 13 Mar 2024 11:01:04 +0100 Subject: [PATCH 11/11] fix single dataset Signed-off-by: Martijn Govers --- src/power_grid_model/data_types.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/power_grid_model/data_types.py b/src/power_grid_model/data_types.py index 42b2c812e9..875cfb0e3f 100644 --- a/src/power_grid_model/data_types.py +++ b/src/power_grid_model/data_types.py @@ -60,7 +60,7 @@ A data array can be a :class:`SingleArray` or a :class:`BatchArray`. """ -SingleDataset = Union[SingleArray] +SingleDataset = Dict[str, SingleArray] """ A single dataset is a dictionary where the keys are the component types and the values are :class:`SingleArray`