RuntimeError: Drawing dask graphs

[thedurphy]

Context

• I'm using the Docker installation method and running the notebook inside the container.
• MacOS Catalina 10.15.1

I am on the 01_dash.delayed notebook and running into the following error when running the z.visualize() command.

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
/opt/conda/lib/python3.7/site-packages/dask/utils.py in import_required(mod_name, error_msg)
     97     try:
---> 98         return import_module(mod_name)
     99     except ImportError:

/opt/conda/lib/python3.7/importlib/__init__.py in import_module(name, package)
    126             level += 1
--> 127     return _bootstrap._gcd_import(name[level:], package, level)
    128 

/opt/conda/lib/python3.7/importlib/_bootstrap.py in _gcd_import(name, package, level)

/opt/conda/lib/python3.7/importlib/_bootstrap.py in _find_and_load(name, import_)

/opt/conda/lib/python3.7/importlib/_bootstrap.py in _find_and_load_unlocked(name, import_)

ModuleNotFoundError: No module named 'graphviz'

During handling of the above exception, another exception occurred:

RuntimeError                              Traceback (most recent call last)
<ipython-input-23-be67cab1446e> in <module>
      1 # Look at the task graph for `z`
----> 2 z.visualize()

/opt/conda/lib/python3.7/site-packages/dask/base.py in visualize(self, filename, format, optimize_graph, **kwargs)
    104             format=format,
    105             optimize_graph=optimize_graph,
--> 106             **kwargs
    107         )
    108 

/opt/conda/lib/python3.7/site-packages/dask/base.py in visualize(*args, **kwargs)
    494     https://docs.dask.org/en/latest/optimize.html
    495     """
--> 496     from dask.dot import dot_graph
    497 
    498     filename = kwargs.pop("filename", "mydask")

/opt/conda/lib/python3.7/site-packages/dask/dot.py in <module>
     12 graphviz = import_required(
     13     "graphviz",
---> 14     "Drawing dask graphs requires the "
     15     "`graphviz` python library and the "
     16     "`graphviz` system library to be "

/opt/conda/lib/python3.7/site-packages/dask/utils.py in import_required(mod_name, error_msg)
     98         return import_module(mod_name)
     99     except ImportError:
--> 100         raise RuntimeError(error_msg)
    101 
    102 

RuntimeError: Drawing dask graphs requires the `graphviz` python library and the `graphviz` system library to be installed.

It seems as though this error is caused by graphviz not being installed correctly.

The solution that worked for me was to run inside the container the following commands...

conda install -c conda-forge graphviz
conda install -c conda-forge python-graphviz

[TomAugspurger]

Is the graphviz system library available outside of the python process? Just dot -V should show whether it is or not.

[thedurphy]

It is now :) since I've installed it manually. I modified the Dockerfile and spun up a new container. The new container functions as intended and not getting any graphviz module errors. I don't have permission to push my changes so I'll put the docker file contents below.

FROM jupyter/scipy-notebook:1386e2046833

USER root
# python3 setup
RUN apt-get update

USER jovyan

RUN git clone https://github.com/dask/dask-tutorial.git ./dask-tutorial
RUN cd dask-tutorial && conda env update -f binder/environment.yml && . binder/postBuild && cd ..
RUN conda install -c conda-forge graphviz python-graphviz
RUN rm dask-tutorial/github_deploy_key_dask_dask_tutorial.enc

Additionally, I would recommend updating to the latest version of conda although it is not necessary for this tutorial.

[TomAugspurger]

Hmm, I just built from scratch and things look fine. ```python $ docker build -t dask-tutorial . Sending build context to Docker daemon .... Step 1/7 : FROM jupyter/scipy-notebook:1386e2046833 ---> 9595623cae9f Step 2/7 : USER root ---> Using cache ---> 646cad32b445 Step 3/7 : RUN apt-get update && apt-get install -y graphviz ---> Using cache ---> 5eaa0e4b9c40 Step 4/7 : USER jovyan ---> Using cache ---> 6ed3504e61cc Step 5/7 : RUN git clone https://github.com/dask/dask-tutorial.git ./dask-tutorial ---> Using cache ---> 60c962e54ccc Step 6/7 : RUN cd dask-tutorial && conda env update -f binder/environment.yml && . binder/postBuild && cd .. ---> Using cache ---> 64bf896a6e45 Step 7/7 : RUN rm dask-tutorial/github_deploy_key_dask_dask_tutorial.enc ---> Using cache ---> c813a576bd75 Successfully built c813a576bd75 Successfully tagged dask-tutorial:latest $ docker run -it --rm dask-tutorial:latest /bin/bash jovyan@81eb2d61deb7:~$ dot -V dot - graphviz version 2.40.1 (20161225.0304) jovyan@81eb2d61deb7:~$ source activate dask-tutorial (dask-tutorial) jovyan@81eb2d61deb7:~$ ipython Python 3.7.3 | packaged by conda-forge | (default, Jul 1 2019, 21:52:21) Type 'copyright', 'credits' or 'license' for more information IPython 7.8.0 -- An enhanced Interactive Python. Type '?' for help. In [1]: import dask.array as da In [2]: da.ones((10, 10)).visualize() Out[2]: <IPython.core.display.Image object> ``` Are you sure you had an up to date copy of the git repo when you built your docker image?

…

On Thu, Nov 21, 2019 at 9:14 AM thedurphy ***@***.***> wrote: It is now :) since I've installed it manually. I modified the Dockerfile and spun up a new container. The new container functions as intended and not getting any graphviz module errors. I don't have permission to push my changes so I'll put the docker file contents below. FROM jupyter/scipy-notebook:1386e2046833 USER root # python3 setup RUN apt-get update USER jovyan RUN git clone https://github.com/dask/dask-tutorial.git ./dask-tutorial RUN cd dask-tutorial && conda env update -f binder/environment.yml && . binder/postBuild && cd .. RUN conda install -c conda-forge graphviz python-graphviz RUN rm dask-tutorial/github_deploy_key_dask_dask_tutorial.enc Additionally, I would recommend updating to the latest version of conda although it is not necessary for this tutorial. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#139?email_source=notifications&email_token=AAKAOIVG24OBOAI2ICUT7BLQU2QTTA5CNFSM4JQDOXDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE2R3FY#issuecomment-557129111>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKAOIU33TOACS2L3YDCFW3QU2QTTANCNFSM4JQDOXDA> .

[jrbourbeau]

I think the problem here is that the dask-tutorial conda environment isn't activated when the notebook is launched. Going through the docker installation steps and running !conda env list shows the base environment is activated

[borundev]

I just submitted a PR to fix this issue @jrbourbeau

[shinsa82]

I experienced the same issue as above. It seems that the PR @borundev submitted for the problem has not been merged yet. Does anyone know is there any reason for that?
This tutorial content is quite useful and I want to recommend this to my colleagues, so I hope the PR would be merged.

[jrbourbeau]

@shinsa82 #194 was just merged, if you want to give things a try now

[shinsa82]

Thanks! It worked:

[jrbourbeau]

Great, thanks for trying it out!