Merge pull request #279 from rogersamso/building_split_model_merge

Split Vensim views in separate Python files

Author: enekomartinmartinez

MANIFEST.in

@@ -1,3 +1,4 @@
 include requirements.txt
 include README.md
+include LICENSE
 include pysd/py_backend/xmile/smile.grammar

README.md

@@ -2,6 +2,7 @@ PySD
 ====
 [![Build Status](https://travis-ci.com/JamesPHoughton/pysd.svg?branch=master)](https://travis-ci.com/JamesPHoughton/pysd)
 [![Coverage Status](https://coveralls.io/repos/github/JamesPHoughton/pysd/badge.svg?branch=master)](https://coveralls.io/github/JamesPHoughton/pysd?branch=master)
+[![Anaconda-Server Badge](https://anaconda.org/conda-forge/pysd/badges/version.svg)](https://anaconda.org/conda-forge/pysd)
 [![PyPI version](https://badge.fury.io/py/pysd.svg)](https://badge.fury.io/py/pysd)
 [![PyPI status](https://img.shields.io/pypi/status/pysd.svg)](https://pypi.python.org/pypi/pysd/)
 [![Py version](https://img.shields.io/pypi/pyversions/pysd.svg)](https://pypi.python.org/pypi/pysd/)

docs/advanced_usage.rst

@@ -41,6 +41,32 @@ We can substitute this function directly for the heat_loss_to_room model compone
 
 If you want to replace a subscripted variable, you need to ensure that the output from the new function is the same as the previous one. You can check the current coordinates and dimensions of a component by using :py:data:`.get_coords(variable_name)` as it is explained in :doc:`basic usage <../basic_usage>`.
 
+
+Splitting Vensim views in different files
+-----------------------------------------
+In order to replicate the Vensim views in translated models, the user can set the `split_modules` argument to True in the :py:func:`read_vensim` function::
+
+   read_vensim("many_views_model.mdl", split_modules=True)
+
+
+The option to split the model in views is particularly interesting for large models with tens of views. Translating those models into a single file may make the resulting Python model difficult to read and maintain.
+
+In a Vensim model with three separate views (e.g. `view_1`, `view_2` and `view_3`), setting `split_modules` to True would create the following tree inside the directory where the `.mdl` model is located:
+
+| main-folder
+| ├── modules_many_views_model
+| │   ├── _modules.json
+| │   ├── view_1.py
+| │   ├── view_2.py
+| │   └── view_3.py
+| ├── _namespace_many_views_model.json
+| ├── _subscripts_dict_many_views_model.json
+| ├── many_views_model.py
+|
+|
+If macros are present, they will be self-contained in files named as the macro itself. The macro inner variables will be placed inside the module that corresponds with the view in which they were defined.
+
+
 Starting simulations from an end-state of another simulation
 ------------------------------------------------------------
 The current state of a model can be saved in a pickle file using the :py:data:`.export()`method::
@@ -68,5 +94,5 @@ the new simulation will have initial time equal to 50 with the saved values from
    The changes done with *params* arguments are not ported to the new model (*model2*) object that you initialize with *final_state.pic*. If you want to keep them, you need to call run with the same *params* values as in the original model (*model1*).
 
 .. warning::
-  Exported data is saved and loaded using `pickle <https://docs.python.org/3/library/pickle.html>`_, this data can be not compatible with future versions of
+  Exported data is saved and loaded using `pickle <https://docs.python.org/3/library/pickle.html>`_, this data can be incompatible with future versions of
   *PySD* or *xarray*. In order to prevent data losses save always the source code.

docs/command_line_usage.rst

@@ -41,6 +41,9 @@ The progress bar can be activated using *-p/--progress* command:
 
     python -m pysd --progress Teacup.mdl
 
+Translation options
+-------------------
+
 Only translate model file
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 To only translate the model file, it does not run the model, *-t/--trasnlate* command is provided:
@@ -49,6 +52,14 @@ To only translate the model file, it does not run the model, *-t/--trasnlate* co
 
     python -m pysd --translate Teacup.mdl
 
+Splitting Vensim views in different files
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+In order to split the Vensim model views in different files as explained in :doc:`advanced usage <../advanced_usage>`:
+
+.. code-block:: text
+
+    python -m pysd --split-modules many_views_model.mdl
+
 Outputting various run information
 ----------------------------------
 The output number of variables can be modified bu passing them as arguments separated by commas, using *-r/return_columns* argument:

docs/installation.rst

@@ -10,6 +10,13 @@ Python environment, use the pip command:
 
    pip install pysd
 
+Installing with conda
+---------------------
+To install PySD with conda, using the conda-forge channel, into a conda environment, use the following command:
+
+.. code-block:: bash
+
+   conda install -c conda-forge pysd
 
 Installing from source
 ----------------------

pysd/_version.py

@@ -1 +1 @@
-__version__ = "1.7.0"
+__version__ = "1.8.0"

pysd/cli/main.py

@@ -25,7 +25,8 @@ def main(args):
     """
     options = parser.parse_args(args)
 
-    model = load(options.model_file, options.missing_values)
+    model = load(options.model_file, options.missing_values,
+                 options.split_modules)
 
     if not options.run:
         print("\nFinished!")
@@ -43,7 +44,7 @@ def main(args):
     sys.exit()
 
 
-def load(model_file, missing_values):
+def load(model_file, missing_values, split_modules):
     """
     Translate and load model file.
 
@@ -60,7 +61,8 @@ def load(model_file, missing_values):
     if model_file.lower().endswith('.mdl'):
         print("\nTranslating model file...\n")
         return pysd.read_vensim(model_file, initialize=False,
-                                missing_values=missing_values)
+                                missing_values=missing_values,
+                                split_modules=split_modules)
     elif model_file.lower().endswith('.xmile'):
         print("\nTranslating model file...\n")
         return pysd.read_xmile(model_file, initialize=False,

pysd/cli/parser.py

@@ -152,12 +152,6 @@ def __call__(self, parser, namespace, values, option_string=None):
     '-v', '--version',
     action='version', version=f'PySD {__version__}')
 
-parser.add_argument(
-    '-t', '--translate', dest='run',
-    action='store_false', default=True,
-    help='only translate the model_file, '
-         'it does not run it after translation')
-
 parser.add_argument(
     '-o', '--output-file', dest='output_file',
     type=check_output, metavar='FILE',
@@ -187,6 +181,7 @@ def __call__(self, parser, namespace, values, option_string=None):
     help='import stateful objects states from a pickle file,'
          'if given initial conditions from var:value will be ignored')
 
+
 ###################
 # Model arguments #
 ###################
@@ -223,6 +218,26 @@ def __call__(self, parser, namespace, values, option_string=None):
          '--saveper will be ignored')
 
 
+###################
+# Model arguments #
+###################
+
+trans_arguments = parser.add_argument_group(
+    'translation arguments',
+    'Configure the translation of the original model.')
+
+trans_arguments.add_argument(
+    '--translate', dest='run',
+    action='store_false', default=True,
+    help='only translate the model_file, '
+         'it does not run it after translation')
+
+trans_arguments.add_argument(
+    '--split-modules', dest='split_modules',
+    action='store_true', default=False,
+    help='parse the sketch to detect model elements in each model view,'
+         ' and then translate each view in a separate python file')
+
 #######################
 # Warnings and errors #
 #######################
@@ -232,7 +247,7 @@ def __call__(self, parser, namespace, values, option_string=None):
     'Modify warning and errors management.')
 
 warn_err_arguments.add_argument(
-    '-m', '--missing-values', dest='missing_values', default="warning",
+    '--missing-values', dest='missing_values', default="warning",
     action='store', type=str, choices=['warning', 'raise', 'ignore', 'keep'],
     help='exception with missing values, \'warning\' (default) shows a '
          'warning message and interpolates the values, \'raise\' raises '