diff --git a/.github/issue_template.md b/.github/issue_template.md index 6ffc18e8ae..e7f84d35d9 100644 --- a/.github/issue_template.md +++ b/.github/issue_template.md @@ -1,3 +1,4 @@ + Hi! Thanks for using the Jupyter Docker Stacks. Please review the following guidance about how to ask questions, contribute changes, or report bugs in the Docker images maintained here. @@ -20,7 +21,7 @@ Example: `docker run -it --rm -p 8889:8888 jupyter/all-spark-notebook:latest` Example: -1. Visit http://localhost:8888 +1. Visit 2. Start an R notebook 3. ... diff --git a/.markdownlint.yaml b/.markdownlint.yaml new file mode 100644 index 0000000000..60d5e82250 --- /dev/null +++ b/.markdownlint.yaml @@ -0,0 +1,7 @@ +# Default state for all rules +default: true + +# MD013/line-length - Line length +MD013: + # Number of characters + line_length: 1000 diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 8d15d20f43..f7d4445de6 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -31,3 +31,8 @@ repos: rev: v1.5.6 hooks: - id: autopep8 + - repo: https://github.com/igorshubovych/markdownlint-cli + rev: v0.27.1 + hooks: + - id: markdownlint + args: ['--fix'] diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index a172249272..c2c72f8c9b 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -1 +1,3 @@ +# Project `jupyter/docker-stacks` Code of Conduct + Please see the [Project Jupyter Code of Conduct](https://github.com/jupyter/governance/blob/master/conduct/code_of_conduct.md). diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 60bb0152a0..e3d886348c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,3 +1,5 @@ + + Thanks for contributing! Please see the __Contributor Guide__ section in [the documentation](https://jupyter-docker-stacks.readthedocs.io) for information about how to contribute diff --git a/LICENSE.md b/LICENSE.md index bd6397d458..0f41c902c7 100644 --- a/LICENSE.md +++ b/LICENSE.md @@ -39,7 +39,7 @@ The Jupyter Development Team is the set of all contributors to the Jupyter proje This includes all of the Jupyter subprojects. The core team that coordinates development on GitHub can be found here: -https://github.com/jupyter/. +. ## Our Copyright Policy diff --git a/README.md b/README.md index 9a224b3b23..9e09500927 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,10 @@ +# Jupyter Docker Stacks + [![Discourse badge](https://img.shields.io/discourse/https/discourse.jupyter.org/users.svg?color=%23f37626)](https://discourse.jupyter.org/c/questions "Jupyter Discourse Q&A") [![Read the Docs badge](https://img.shields.io/readthedocs/jupyter-docker-stacks.svg)](https://jupyter-docker-stacks.readthedocs.io/en/latest/ "Documentation build status") [![DockerHub badge](https://images.microbadger.com/badges/version/jupyter/base-notebook.svg)](https://microbadger.com/images/jupyter/base-notebook "Recent tag/version of jupyter/base-notebook") [![Binder badget](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/jupyter/docker-stacks/master?filepath=README.ipynb "Launch a jupyter/base-notebook container on mybinder.org") -# Jupyter Docker Stacks - Jupyter Docker Stacks are a set of ready-to-run [Docker images](https://hub.docker.com/u/jupyter) containing Jupyter applications and interactive computing tools. diff --git a/all-spark-notebook/README.md b/all-spark-notebook/README.md index f17bffef4c..7e78b7d741 100644 --- a/all-spark-notebook/README.md +++ b/all-spark-notebook/README.md @@ -1,10 +1,10 @@ +# Jupyter Notebook Python, Scala, R, Spark Stack + [![docker pulls](https://img.shields.io/docker/pulls/jupyter/all-spark-notebook.svg)](https://hub.docker.com/r/jupyter/all-spark-notebook/) [![docker stars](https://img.shields.io/docker/stars/jupyter/all-spark-notebook.svg)](https://hub.docker.com/r/jupyter/all-spark-notebook/) [![image metadata](https://images.microbadger.com/badges/image/jupyter/all-spark-notebook.svg)](https://microbadger.com/images/jupyter/all-spark-notebook "jupyter/all-spark-notebook image metadata") -# Jupyter Notebook Python, Scala, R, Spark Stack - -GitHub Actions in the https://github.com/jupyter/docker-stacks project builds and pushes this image +GitHub Actions in the project builds and pushes this image to Docker Hub. Please visit the project documentation site for help using and contributing to this image and diff --git a/base-notebook/README.md b/base-notebook/README.md index 7741e990f4..2b9700e08a 100644 --- a/base-notebook/README.md +++ b/base-notebook/README.md @@ -1,10 +1,10 @@ +# Base Jupyter Notebook Stack + [![docker pulls](https://img.shields.io/docker/pulls/jupyter/base-notebook.svg)](https://hub.docker.com/r/jupyter/base-notebook/) [![docker stars](https://img.shields.io/docker/stars/jupyter/base-notebook.svg)](https://hub.docker.com/r/jupyter/base-notebook/) [![image metadata](https://images.microbadger.com/badges/image/jupyter/base-notebook.svg)](https://microbadger.com/images/jupyter/base-notebook "jupyter/base-notebook image metadata") -# Base Jupyter Notebook Stack - -GitHub Actions in the https://github.com/jupyter/docker-stacks project builds and pushes this image +GitHub Actions in the project builds and pushes this image to Docker Hub. Please visit the project documentation site for help using and contributing to this image and diff --git a/datascience-notebook/README.md b/datascience-notebook/README.md index 2f8fef4e14..a588329665 100644 --- a/datascience-notebook/README.md +++ b/datascience-notebook/README.md @@ -1,10 +1,10 @@ +# Jupyter Notebook Data Science Stack + [![docker pulls](https://img.shields.io/docker/pulls/jupyter/datascience-notebook.svg)](https://hub.docker.com/r/jupyter/datascience-notebook/) [![docker stars](https://img.shields.io/docker/stars/jupyter/datascience-notebook.svg)](https://hub.docker.com/r/jupyter/datascience-notebook/) [![image metadata](https://images.microbadger.com/badges/image/jupyter/datascience-notebook.svg)](https://microbadger.com/images/jupyter/datascience-notebook "jupyter/datascience-notebook image metadata") -# Jupyter Notebook Data Science Stack - -GitHub Actions in the https://github.com/jupyter/docker-stacks project builds and pushes this image +GitHub Actions in the project builds and pushes this image to Docker Hub. Please visit the project documentation site for help using and contributing to this image and diff --git a/docs/contributing/features.md b/docs/contributing/features.md index 4cdc23d13d..61a183b3d0 100644 --- a/docs/contributing/features.md +++ b/docs/contributing/features.md @@ -40,9 +40,11 @@ If there's agreement that the feature belongs in one or more of the core stacks: 2. Please build the image locally before submitting a pull request. Building the image locally shortens the debugging cycle by taking some load off GitHub Actions, which graciously provide free build services for open source projects like this one. If you use `make`, call: + ```bash make build/somestack-notebook ``` + 3. [Submit a pull request](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request) (PR) with your changes. 4. Watch for GitHub to report a build success or failure for your PR on GitHub. diff --git a/docs/contributing/lint.md b/docs/contributing/lint.md index 774b4d7352..42bd637ed2 100644 --- a/docs/contributing/lint.md +++ b/docs/contributing/lint.md @@ -6,7 +6,7 @@ To integrate and enforce this process in the project lifecycle we are using **gi ## Pre-commit hook -### Installation +### Pre-commit hook installation pre-commit is a Python package that needs to be installed. This can be achieved by using the generic task used to install all Python development dependencies. @@ -21,7 +21,7 @@ $ pip install pre-commit Then the git hooks scripts configured for the project in `.pre-commit-config.yaml` need to be installed in the local git repository. ```sh -$ make pre-commit-install +make pre-commit-install ``` ### Run @@ -32,7 +32,7 @@ However it is also possible to trigger it against all files. - Note: Hadolint pre-commit uses docker to run, so docker should be running while running this command. ```sh -$ make pre-commit-all +make pre-commit-all ``` ## Image Lint diff --git a/docs/contributing/packages.md b/docs/contributing/packages.md index f6e717db28..edfad60f9f 100644 --- a/docs/contributing/packages.md +++ b/docs/contributing/packages.md @@ -16,9 +16,11 @@ Please follow the process below to update a package version: 3. Please build the image locally before submitting a pull request. Building the image locally shortens the debugging cycle by taking some load off GitHub Actions, which graciously provide free build services for open source projects like this one. If you use `make`, call: + ```bash make build/somestack-notebook ``` + 4. [Submit a pull request](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request) (PR) with your changes. 5. Watch for GitHub to report a build success or failure for your PR on GitHub. diff --git a/docs/contributing/stacks.md b/docs/contributing/stacks.md index a997ba6a45..e29c1f02ef 100644 --- a/docs/contributing/stacks.md +++ b/docs/contributing/stacks.md @@ -20,7 +20,7 @@ your own path using alternative services and build tools. First, install [cookiecutter](https://github.com/audreyr/cookiecutter) using pip or conda: ```bash -pip install cookiecutter # or conda install cookiecutter +pip install cookiecutter # or conda install cookiecutter ``` Run the cookiecutter command pointing to the @@ -34,7 +34,7 @@ cookiecutter https://github.com/jupyter/cookiecutter-docker-stacks.git Enter a name for your new stack image. This will serve as both the git repository name and the part of the Docker image name after the slash. -``` +```lang-none stack_name [my-jupyter-stack]: ``` @@ -42,26 +42,26 @@ Enter the user or organization name under which this stack will reside on Docker must have access to manage this Docker Cloud org to push images here and set up automated builds. -``` +```lang-none stack_org [my-project]: ``` Select an image from the jupyter/docker-stacks project that will serve as the base for your new image. -``` +```lang-none stack_base_image [jupyter/base-notebook]: ``` Enter a longer description of the stack for your README. -``` +```lang-none stack_description [my-jupyter-stack is a community maintained Jupyter Docker Stack image]: ``` Initialize your project as a Git repository and push it to GitHub. -``` +```bash cd git init diff --git a/docs/contributing/tests.md b/docs/contributing/tests.md index 36bdfdf7e3..ea0a4e6719 100644 --- a/docs/contributing/tests.md +++ b/docs/contributing/tests.md @@ -23,10 +23,12 @@ Please follow the process below to add new tests: 2. If your test should run against a single image, add your test code to one of the modules in `some-notebook/test/` or create a new module. 3. Build one or more images you intend to test and run the tests locally. If you use `make`, call: + ```bash make build/somestack-notebook make test/somestack-notebook ``` + 4. [Submit a pull request](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request) (PR) with your changes. 5. Watch for GitHub to report a build success or failure for your PR on GitHub. diff --git a/docs/contributing/translations.md b/docs/contributing/translations.md index b9cc1150b5..e0bc23b2d4 100644 --- a/docs/contributing/translations.md +++ b/docs/contributing/translations.md @@ -3,5 +3,5 @@ We are delighted when members of the Jupyter community want to help translate these documentation pages to other languages. If you're interested, please visit links below below to join our team on [Transifex](https://transifex.com) and to start creating, reviewing, and updating translations of the Jupyter Docker Stacks documentation. 1. Follow the steps documented on the [Getting Started as a Translator](https://docs.transifex.com/getting-started-1/translators) page. -2. Look for *jupyter-docker-stacks* when prompted to choose a translation team. Alternatively, visit https://www.transifex.com/project-jupyter/jupyter-docker-stacks-1 after creating your account and request to join the project. +2. Look for *jupyter-docker-stacks* when prompted to choose a translation team. Alternatively, visit after creating your account and request to join the project. 3. See [Translating with the Web Editor](https://docs.transifex.com/translation/translating-with-the-web-editor) in the Transifex documentation. diff --git a/docs/maintaining/tasks.md b/docs/maintaining/tasks.md index f3bbf4380b..8e97278ec8 100644 --- a/docs/maintaining/tasks.md +++ b/docs/maintaining/tasks.md @@ -62,9 +62,9 @@ When there's a new stack definition, do the following before merging the PR with ## Adding a New Maintainer Account -1. Visit https://cloud.docker.com/app/jupyter/team/stacks/users +1. Visit 2. Add the maintainer's Docker Cloud username. -3. Visit https://github.com/orgs/jupyter/teams/docker-image-maintainers/members +3. Visit 4. Add the maintainer's GitHub username. ## Pushing a Build Manually @@ -80,16 +80,16 @@ If automated builds on Docker Cloud have got you down, do the following to push First enable translation on Transifex: -1. Visit https://www.transifex.com/project-jupyter/jupyter-docker-stacks-1/languages/ +1. Visit . 2. Click _Edit Languages_ in the top right. 3. Select the language from the dropdown. 4. Click _Apply_. Then setup a subproject on ReadTheDocs for the language: -1. Visit https://readthedocs.org/dashboard/import/manual/ +1. Visit . 2. Enter _jupyter-docker-stacks-language_abbreviation_ for the project name. -3. Enter https://github.com/jupyter/docker-stacks for the URL. +3. Enter for the URL. 4. Check _Edit advanced project options_. 5. Click _Next_. 6. Select the _Language_ from the dropdown on the next screen. @@ -97,6 +97,6 @@ Then setup a subproject on ReadTheDocs for the language: Finally link the new language subproject to the top level doc project: -1. Visit https://readthedocs.org/dashboard/jupyter-docker-stacks/translations/ +1. Visit . 2. Select the subproject you created from the _Project_ dropdown. 3. Click _Add_. diff --git a/docs/using/recipes.md b/docs/using/recipes.md index f64c51f8b5..affc6de926 100644 --- a/docs/using/recipes.md +++ b/docs/using/recipes.md @@ -166,11 +166,13 @@ ENTRYPOINT ["jupyter", "lab", "--ip=0.0.0.0", "--allow-root"] ``` And build the image as: + ```bash docker build -t jupyter/scipy-dasklabextension:latest . ``` Once built, run using the command: + ```bash docker run -it --rm -p 8888:8888 -p 8787:8787 jupyter/scipy-dasklabextension:latest ``` @@ -273,6 +275,7 @@ ARG BASE_CONTAINER=ubuntu:focal-20200423@sha256:238e696992ba9913d24cfc3727034985 ``` For Ubuntu 18.04 (bionic) and earlier, you may also require to workaround for a mandb bug, which was fixed in mandb >= 2.8.6.1: + ```dockerfile # https://git.savannah.gnu.org/cgit/man-db.git/commit/?id=8197d7824f814c5d4b992b4c8730b5b0f7ec589a # http://launchpadlibrarian.net/435841763/man-db_2.8.5-2_2.8.6-1.diff.gz diff --git a/docs/using/running.md b/docs/using/running.md index 86a09dfdb7..78d3baddb6 100644 --- a/docs/using/running.md +++ b/docs/using/running.md @@ -13,8 +13,8 @@ You can launch a local Docker container from the Jupyter Docker Stacks using the **Example 1** This command pulls the `jupyter/scipy-notebook` image tagged `2c80cf3537ca` from Docker Hub if it is not already present on the local host. It then starts a container running a Jupyter Notebook server and exposes the server on host port 8888. The server logs appear in the terminal and include a URL to the notebook server. -``` -docker run -p 8888:8888 jupyter/scipy-notebook:2c80cf3537ca +```bash +$ docker run -p 8888:8888 jupyter/scipy-notebook:2c80cf3537ca Executing the command: jupyter notebook [I 15:33:00.567 NotebookApp] Writing notebook server cookie secret to /home/jovyan/.local/share/jupyter/runtime/notebook_cookie_secret @@ -35,27 +35,27 @@ Executing the command: jupyter notebook Pressing `Ctrl-C` shuts down the notebook server but leaves the container intact on disk for later restart or permanent deletion using commands like the following: -``` +```bash # list containers -docker ps -a +$ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d67fe77f1a84 jupyter/base-notebook "tini -- start-noteb…" 44 seconds ago Exited (0) 39 seconds ago cocky_mirzakhani # start the stopped container -docker start -a d67fe77f1a84 +$ docker start -a d67fe77f1a84 Executing the command: jupyter notebook [W 16:45:02.020 NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using encryption. This is not recommended. ... # remove the stopped container -docker rm d67fe77f1a84 +$ docker rm d67fe77f1a84 d67fe77f1a84 ``` **Example 2** This command pulls the `jupyter/r-notebook` image tagged `e5c5a7d3e52d` from Docker Hub if it is not already present on the local host. It then starts a container running a Jupyter Notebook server and exposes the server on host port 10000. The server logs appear in the terminal and include a URL to the notebook server, but with the internal container port (8888) instead of the the correct host port (10000). -``` -docker run --rm -p 10000:8888 -v "$PWD":/home/jovyan/work jupyter/r-notebook:e5c5a7d3e52d +```bash +$ docker run --rm -p 10000:8888 -v "$PWD":/home/jovyan/work jupyter/r-notebook:e5c5a7d3e52d Executing the command: jupyter notebook [I 19:31:09.573 NotebookApp] Writing notebook server cookie secret to /home/jovyan/.local/share/jupyter/runtime/notebook_cookie_secret @@ -78,29 +78,29 @@ Pressing `Ctrl-C` shuts down the notebook server and immediately destroys the Do **Example 3** This command pulls the `jupyter/all-spark-notebook` image currently tagged `latest` from Docker Hub if an image tagged `latest` is not already present on the local host. It then starts a container named `notebook` running a JupyterLab server and exposes the server on a randomly selected port. -``` +```bash docker run -d -P --name notebook jupyter/all-spark-notebook ``` The assigned port and notebook server token are visible using other Docker commands. -``` +```bash # get the random host port assigned to the container port 8888 -docker port notebook 8888 +$ docker port notebook 8888 0.0.0.0:32769 # get the notebook token from the logs -docker logs --tail 3 notebook +$ docker logs --tail 3 notebook Copy/paste this URL into your browser when you connect for the first time, to login with a token: http://localhost:8888/?token=15914ca95f495075c0aa7d0e060f1a78b6d94f70ea373b00 ``` -Together, the URL to visit on the host machine to access the server in this case is http://localhost:32769?token=15914ca95f495075c0aa7d0e060f1a78b6d94f70ea373b00. +Together, the URL to visit on the host machine to access the server in this case is . The container runs in the background until stopped and/or removed by additional Docker commands. -``` +```bash # stop the container docker stop notebook notebook diff --git a/docs/using/specifics.md b/docs/using/specifics.md index 7536f692df..6aaf873e28 100644 --- a/docs/using/specifics.md +++ b/docs/using/specifics.md @@ -52,7 +52,7 @@ The `jupyter/pyspark-notebook` and `jupyter/all-spark-notebook` images support t Spark **local mode** is useful for experimentation on small data when you do not have a Spark cluster available. -##### In Python +##### Local Mode in Python In a Python notebook. @@ -69,7 +69,7 @@ rdd.sum() # 5050 ``` -##### In R +##### Local Mode in R In a R notebook with [SparkR][sparkr]. @@ -107,7 +107,7 @@ sdf_len(sc, 100, repartition = 1) %>% # 5050 ``` -##### In Scala +##### Local Mode in Scala Spylon kernel instantiates a `SparkContext` for you in variable `sc` after you configure Spark options in a `%%init_spark` magic cell. @@ -136,11 +136,11 @@ Connection to Spark Cluster on **[Standalone Mode](https://spark.apache.org/docs your Spark workers. (This is a [Spark networking requirement](http://spark.apache.org/docs/latest/cluster-overview.html#components).) * NOTE: When using `--net=host`, you must also use the flags `--pid=host -e - TINI_SUBREAPER=true`. See https://github.com/jupyter/docker-stacks/issues/64 for details. + TINI_SUBREAPER=true`. See for details. **Note**: In the following examples we are using the Spark master URL `spark://master:7077` that shall be replaced by the URL of the Spark master. -##### In Python +##### Standalone Mode in Python The **same Python version** need to be used on the notebook (where the driver is located) and on the Spark workers. The python version used at driver and worker side can be adjusted by setting the environment variables `PYSPARK_PYTHON` and / or `PYSPARK_DRIVER_PYTHON`, see [Spark Configuration][spark-conf] for more information. @@ -158,7 +158,7 @@ rdd.sum() # 5050 ``` -##### In R +##### Standalone Mode in R In a R notebook with [SparkR][sparkr]. @@ -195,7 +195,7 @@ sdf_len(sc, 100, repartition = 1) %>% # 5050 ``` -##### In Scala +##### Standalone Mode in Scala Spylon kernel instantiates a `SparkContext` for you in variable `sc` after you configure Spark options in a `%%init_spark` magic cell. diff --git a/examples/docker-compose/README.md b/examples/docker-compose/README.md index d4a7d2e14c..ff5372f5c9 100644 --- a/examples/docker-compose/README.md +++ b/examples/docker-compose/README.md @@ -1,3 +1,5 @@ +# Docker Compose example + This example demonstrate how to deploy docker-stack notebook containers to any Docker Machine-controlled host using Docker Compose. ## Prerequisites @@ -32,7 +34,6 @@ To stop and remove the container: notebook/down.sh ``` - ## FAQ ### How do I specify which docker-stack notebook image to deploy? @@ -73,7 +74,6 @@ NAME=your-notebook notebook/down.sh The `up.sh` creates a Docker volume named after the notebook container with a `-work` suffix, e.g., `my-notebook-work`. - ### Can multiple notebook containers share the same notebook volume? Yes. Set the `WORK_VOLUME` environment variable to the same value for each notebook. @@ -98,7 +98,6 @@ notebook/up.sh --secure --password a_secret Sure. If you want to secure access to publicly addressable notebook containers, you can generate a free certificate using the [Let's Encrypt](https://letsencrypt.org) service. - This example includes the `bin/letsencrypt.sh` script, which runs the `letsencrypt` client to create a full-chain certificate and private key, and stores them in a Docker volume. _Note:_ The script hard codes several `letsencrypt` options, one of which automatically agrees to the Let's Encrypt Terms of Service. The following command will create a certificate chain and store it in a Docker volume named `mydomain-secrets`. @@ -152,10 +151,9 @@ bin/softlayer.sh myhost bin/sl-dns.sh myhost ``` - ## Troubleshooting -### Unable to connect to VirtualBox VM on Mac OS X when using Cisco VPN client. +### Unable to connect to VirtualBox VM on Mac OS X when using Cisco VPN client The Cisco VPN client blocks access to IP addresses that it does not know about, and may block access to a new VM if it is created while the Cisco VPN client is running. diff --git a/examples/make-deploy/README.md b/examples/make-deploy/README.md index 5e5f6e8dce..6515c3171e 100644 --- a/examples/make-deploy/README.md +++ b/examples/make-deploy/README.md @@ -1,9 +1,11 @@ +# Make deploy example + This folder contains a Makefile and a set of supporting files demonstrating how to run a docker-stack notebook container on a docker-machine controlled host. ## Prerequisites * make 3.81+ - * Ubuntu users: Be aware of [make 3.81 defect 483086](https://bugs.launchpad.net/ubuntu/+source/make-dfsg/+bug/483086) which exists in 14.04 LTS but is fixed in 15.04+ + * Ubuntu users: Be aware of [make 3.81 defect 483086](https://bugs.launchpad.net/ubuntu/+source/make-dfsg/+bug/483086) which exists in 14.04 LTS but is fixed in 15.04+ * docker-machine 0.5.0+ * docker 1.9.0+ diff --git a/examples/openshift/README.md b/examples/openshift/README.md index 619047a831..9c511f1924 100644 --- a/examples/openshift/README.md +++ b/examples/openshift/README.md @@ -1,3 +1,6 @@ +OpenShift example +================= + This example provides templates for deploying the Jupyter Project docker-stacks images to OpenShift. Prerequsites @@ -7,7 +10,7 @@ Any OpenShift 3 environment. The templates were tested with OpenShift 3.7. It is Do be aware that the Jupyter Project docker-stacks images are very large. The OpenShift environment you are using must provide sufficient quota on the per user space for images and the file system for running containers. If the quota is too small, the pulling of the images to a node in the OpenShift cluster when deploying them, will fail due to lack of space. Even if the image is able to be run, if the quota is only just larger than the space required for the image, you will not be able to install many packages into the container before running out of space. -OpenShift Online, the public hosted version of OpenShift from Red Hat has a quota of only 3GB for the image and container file system. As a result, only the ``minimal-notebook`` can be started and there is little space remaining to install additional packages. Although OpenShift Online is suitable for demonstrating these templates work, what you can do in that environment will be limited due to the size of the images. +OpenShift Online, the public hosted version of OpenShift from Red Hat has a quota of only 3GB for the image and container file system. As a result, only the `minimal-notebook` can be started and there is little space remaining to install additional packages. Although OpenShift Online is suitable for demonstrating these templates work, what you can do in that environment will be limited due to the size of the images. If you want to experiment with using Jupyter Notebooks in an OpenShift environment, you should instead use [Minishift](https://www.openshift.org/minishift/). Minishift provides you the ability to run OpenShift in a virtual machine on your own local computer. @@ -20,13 +23,9 @@ To load the templates, login to OpenShift from the command line and run: oc create -f https://raw.githubusercontent.com/jupyter-on-openshift/docker-stacks/master/examples/openshift/templates.json ``` -This should create the following templates: - -``` -jupyter-notebook -``` +This should create the `jupyter-notebook` template -The template can be used from the command line using the ``oc new-app`` command, or from the OpenShift web console by selecting _Add to Project_. This ``README`` is only going to explain deploying from the command line. +The template can be used from the command line using the `oc new-app` command, or from the OpenShift web console by selecting _Add to Project_. This `README` is only going to explain deploying from the command line. Deploying a Notebook -------------------- @@ -39,7 +38,7 @@ oc new-app --template jupyter-notebook The output will be similar to: -``` +```lang-none --> Deploying template "jupyter/jupyter-notebook" to project jupyter Jupyter Notebook @@ -61,13 +60,13 @@ The output will be similar to: Run 'oc status' to view your app. ``` -When no template parameters are provided, the name of the deployed notebook will be ``notebook``. The image used will be: +When no template parameters are provided, the name of the deployed notebook will be `notebook`. The image used will be: -``` +```lang-none jupyter/minimal-notebook:latest ``` -A password you can use when accessing the notebook will be auto generated and is displayed in the output from running ``oc new-app``. +A password you can use when accessing the notebook will be auto generated and is displayed in the output from running `oc new-app`. To see the hostname for accessing the notebook run: @@ -77,14 +76,14 @@ oc get routes The output will be similar to: -``` +```lang-none NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD notebook notebook-jupyter.abcd.pro-us-east-1.openshiftapps.com notebook 8888-tcp edge/Redirect None ``` A secure route will be used to expose the notebook outside of the OpenShift cluster, so in this case the URL would be: -``` +```lang-none https://notebook-jupyter.abcd.pro-us-east-1.openshiftapps.com/ ``` @@ -93,7 +92,7 @@ When prompted, enter the password for the notebook. Passing Template Parameters --------------------------- -To override the name for the notebook, the image used, and the password, you can pass template parameters using the ``--param`` option. +To override the name for the notebook, the image used, and the password, you can pass template parameters using the `--param` option. ```bash oc new-app --template jupyter-notebook \ @@ -113,12 +112,12 @@ You can deploy any of the Jupyter Project docker-stacks images. * jupyter/pyspark-notebook * jupyter/all-spark-notebook -If you don't care what version of the image is used, add the ``:latest`` tag at the end of the image name, otherwise use the hash corresponding to the image version you want to use. +If you don't care what version of the image is used, add the `:latest` tag at the end of the image name, otherwise use the hash corresponding to the image version you want to use. Deleting the Notebook Instance ------------------------------ -To delete the notebook instance, run ``oc delete`` using a label selector for the application name. +To delete the notebook instance, run `oc delete` using a label selector for the application name. ```bash oc delete all,configmap --selector app=mynotebook @@ -127,7 +126,7 @@ oc delete all,configmap --selector app=mynotebook Enabling Jupyter Lab Interface ------------------------------ -To enable the Jupyter Lab interface for a deployed notebook set the ``JUPYTER_ENABLE_LAB`` environment variable. +To enable the Jupyter Lab interface for a deployed notebook set the `JUPYTER_ENABLE_LAB` environment variable. ```bash oc set env dc/mynotebook JUPYTER_ENABLE_LAB=true @@ -162,11 +161,11 @@ If you want to set any custom configuration for the notebook, you can edit the c oc edit configmap/mynotebook-cfg ``` -The ``data`` field of the config map contains Python code used as the ``jupyter_notebook_config.py`` file. +The `data` field of the config map contains Python code used as the `jupyter_notebook_config.py` file. If you are using a persistent volume, you can also create a configuration file at: -``` +```lang-none /home/jovyan/.jupyter/jupyter_notebook_config.py ``` @@ -174,7 +173,7 @@ This will be merged at the end of the configuration from the config map. Because the configuration is Python code, ensure any indenting is correct. Any errors in the configuration file will cause the notebook to fail when starting. -If the error is in the config map, edit it again to fix it and trigged a new deployment if necessary by running: +If the error is in the config map, edit it again to fix it and trigger a new deployment if necessary by running: ```bash oc rollout latest dc/mynotebook @@ -213,15 +212,7 @@ oc set env dc/mynotebook JUPYTER_NOTEBOOK_PASSWORD=mypassword This will trigger a new deployment so ensure you have downloaded any work if not using a persistent volume. -If using a persistent volume, you could instead setup a password in the file: - -``` -/home/jovyan/.jupyter/jupyter_notebook_config.py -``` - -as per guidelines in: - -* https://jupyter-notebook.readthedocs.io/en/stable/public_server.html +If using a persistent volume, you could instead setup a password in the file `/home/jovyan/.jupyter/jupyter_notebook_config.py` as per guidelines in . Deploying from a Custom Image ----------------------------- diff --git a/examples/source-to-image/README.md b/examples/source-to-image/README.md index 1f19a1d862..3a3c037f73 100644 --- a/examples/source-to-image/README.md +++ b/examples/source-to-image/README.md @@ -1,10 +1,13 @@ +Custom Jupyter Notebook images +============================== + This example provides scripts for building custom Jupyter Notebook images containing notebooks, data files, and with Python packages required by the notebooks already installed. The scripts provided work with the Source-to-Image tool and you can create the images from the command line on your own computer. Templates are also provided to enable running builds in OpenShift, as well as deploying the resulting image to OpenShift to make it available. -The build scripts, when used with the Source-to-Image tool, provide similar capabilities to ``repo2docker``. When builds are run under OpenShift with the supplied templates, it provides similar capabilities to ``mybinder.org``, but where notebook instances are deployed in your existing OpenShift project and JupyterHub is not required. +The build scripts, when used with the Source-to-Image tool, provide similar capabilities to `repo2docker`. When builds are run under OpenShift with the supplied templates, it provides similar capabilities to `mybinder.org`, but where notebook instances are deployed in your existing OpenShift project and JupyterHub is not required. For separate examples of using JupyterHub with OpenShift, see the project: -* https://github.com/jupyter-on-openshift/jupyterhub-quickstart +* Source-to-Image Project ----------------------- @@ -13,7 +16,7 @@ Source-to-Image (S2I) is an open source project which provides a tool for creati Details on the S2I tool, and executable binaries for Linux, macOS and Windows, can be found on GitHub at: -* https://github.com/openshift/source-to-image +* The tool is standalone, and can be used on any system which provides a docker daemon for running containers. To provide an end-to-end capability to build and deploy applications in containers, support for S2I is also integrated into container platforms such as OpenShift. @@ -31,18 +34,19 @@ s2i build \ notebook-examples ``` -This example command will pull down the Git repository ``https://github.com/jupyter/notebook`` and build the image ``notebook-examples`` using the files contained in the ``docs/source/examples/Notebook`` directory of that Git repository. The base image which the files will be combined with is ``jupyter/minimal-notebook:latest``, but you can specify any of the Jupyter Project ``docker-stacks`` images as the base image. +This example command will pull down the Git repository and build the image `notebook-examples` using the files contained in the `docs/source/examples/Notebook` directory of that Git repository. The base image which the files will be combined with is `jupyter/minimal-notebook:latest`, but you can specify any of the Jupyter Project `docker-stacks` images as the base image. -The resulting image from running the command can be seen by running ``docker images``. +The resulting image from running the command can be seen by running `docker images` command: -``` +```bash +$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE notebook-examples latest f5899ed1241d 2 minutes ago 2.59GB ``` You can now run the image. -``` +```bash $ docker run --rm -p 8888:8888 notebook-examples Executing the command: jupyter notebook [I 01:14:50.532 NotebookApp] Writing notebook server cookie secret to /home/jovyan/.local/share/jupyter/runtime/notebook_cookie_secret @@ -66,15 +70,15 @@ Open your browser on the URL displayed, and you will find the notebooks from the The S2I Builder Scripts ----------------------- -Normally when using S2I, the base image would be S2I enabled and contain the builder scripts needed to prepare the image and define how the application in the image should be run. As the Jupyter Project ``docker-stacks`` images are not S2I enabled (although they could be), in the above example the ``--scripts-url`` option has been used to specify that the example builder scripts contained in this directory of this Git repository should be used. +Normally when using S2I, the base image would be S2I enabled and contain the builder scripts needed to prepare the image and define how the application in the image should be run. As the Jupyter Project `docker-stacks` images are not S2I enabled (although they could be), in the above example the `--scripts-url` option has been used to specify that the example builder scripts contained in this directory of this Git repository should be used. -Using the ``--scripts-url`` option, the builder scripts can be hosted on any HTTP server, or you could also use builder scripts local to your computer file using an appropriate ``file://`` format URI argument to ``--scripts-url``. +Using the `--scripts-url` option, the builder scripts can be hosted on any HTTP server, or you could also use builder scripts local to your computer file using an appropriate `file://` format URI argument to `--scripts-url`. -The builder scripts in this directory of this repository are ``assemble`` and ``run`` and are provided as examples of what can be done. You can use the scripts as is, or create your own. +The builder scripts in this directory of this repository are `assemble` and `run` and are provided as examples of what can be done. You can use the scripts as is, or create your own. -The supplied ``assemble`` script performs a few key steps. +The supplied `assemble` script performs a few key steps. -The first steps copy files into the location they need to be when the image is run, from the directory where they are initially placed by the ``s2i`` command. +The first steps copy files into the location they need to be when the image is run, from the directory where they are initially placed by the `s2i` command. ```bash cp -Rf /tmp/src/. /home/$NB_USER @@ -95,7 +99,7 @@ else fi ``` -This determines whether a ``environment.yml`` or ``requirements.txt`` file exists with the files and if so, runs the appropriate package management tool to install any Python packages listed in those files. +This determines whether a `environment.yml` or `requirements.txt` file exists with the files and if so, runs the appropriate package management tool to install any Python packages listed in those files. This means that so long as a set of notebook files provides one of these files listing what Python packages they need, those packages will be automatically installed into the image so they are available when the image is run. @@ -106,11 +110,11 @@ fix-permissions $CONDA_DIR fix-permissions /home/$NB_USER ``` -This fixes up permissions on any new files created by the build. This is necessary to ensure that when the image is run, you can still install additional files. This is important for when an image is run in ``sudo`` mode, or it is hosted in a more secure container platform such as Kubernetes/OpenShift where it will be run as a set user ID that isn't known in advance. +This fixes up permissions on any new files created by the build. This is necessary to ensure that when the image is run, you can still install additional files. This is important for when an image is run in `sudo` mode, or it is hosted in a more secure container platform such as Kubernetes/OpenShift where it will be run as a set user ID that isn't known in advance. -As long as you preserve the first and last set of steps, you can do whatever you want in the ``assemble`` script to install packages, create files etc. Do be aware though that S2I builds do not run as ``root`` and so you cannot install additional system packages. If you need to install additional system packages, use a ``Dockerfile`` and normal ``docker build`` to first create a new custom base image from the Jupyter Project ``docker-stacks`` images, with the extra system packages, and then use that image with the S2I build to combine your notebooks and have Python packages installed. +As long as you preserve the first and last set of steps, you can do whatever you want in the `assemble` script to install packages, create files etc. Do be aware though that S2I builds do not run as `root` and so you cannot install additional system packages. If you need to install additional system packages, use a `Dockerfile` and normal `docker build` to first create a new custom base image from the Jupyter Project `docker-stacks` images, with the extra system packages, and then use that image with the S2I build to combine your notebooks and have Python packages installed. -The ``run`` script in this directory is very simple and just runs the notebook application. +The `run` script in this directory is very simple and just runs the notebook application. ```bash exec start-notebook.sh "$@" @@ -132,7 +136,7 @@ jupyter-notebook-builder jupyter-notebook-quickstart ``` -The templates can be used from the OpenShift web console or command line. This ``README`` is only going to explain deploying from the command line. +The templates can be used from the OpenShift web console or command line. This `README` is only going to explain deploying from the command line. To use the OpenShift command line to build into an image, and deploy, the set of notebooks used above, run: @@ -145,21 +149,17 @@ oc new-app --template jupyter-notebook-quickstart \ --param NOTEBOOK_PASSWORD=mypassword ``` -You can provide a password using the ``NOTEBOOK_PASSWORD`` parameter. If you don't set that parameter, a password will be generated, with it being displayed by the ``oc new-app`` command. +You can provide a password using the `NOTEBOOK_PASSWORD` parameter. If you don't set that parameter, a password will be generated, with it being displayed by the `oc new-app` command. -Once the image has been built, it will be deployed. To see the hostname for accessing the notebook, run ``oc get routes``. +Once the image has been built, it will be deployed. To see the hostname for accessing the notebook, run `oc get routes`. -``` +```lang-none NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD notebook-examples notebook-examples-jupyter.abcd.pro-us-east-1.openshiftapps.com notebook-examples 8888-tcp edge/Redirect None ``` -As the deployment will use a secure connection, the URL for accessing the notebook in this case would be: - -``` -https://notebook-examples-jupyter.abcd.pro-us-east-1.openshiftapps.com -``` +As the deployment will use a secure connection, the URL for accessing the notebook in this case would be . -If you only want to build an image but not deploy it, you can use the ``jupyter-notebook-builder`` template. You can then deploy it using the ``jupyter-notebook`` template provided with the [openshift](../openshift) examples directory. +If you only want to build an image but not deploy it, you can use the `jupyter-notebook-builder` template. You can then deploy it using the `jupyter-notebook` template provided with the [openshift](../openshift) examples directory. -See the ``openshift`` examples directory for further information on customizing configuration for a Jupyter Notebook deployment and deleting a deployment. +See the `openshift` examples directory for further information on customizing configuration for a Jupyter Notebook deployment and deleting a deployment. diff --git a/minimal-notebook/README.md b/minimal-notebook/README.md index 8e76114fcd..f651cd3d4b 100644 --- a/minimal-notebook/README.md +++ b/minimal-notebook/README.md @@ -1,10 +1,10 @@ +# Minimal Jupyter Notebook Stack + [![docker pulls](https://img.shields.io/docker/pulls/jupyter/minimal-notebook.svg)](https://hub.docker.com/r/jupyter/minimal-notebook/) [![docker stars](https://img.shields.io/docker/stars/jupyter/minimal-notebook.svg)](https://hub.docker.com/r/jupyter/minimal-notebook/) [![image metadata](https://images.microbadger.com/badges/image/jupyter/minimal-notebook.svg)](https://microbadger.com/images/jupyter/minimal-notebook "jupyter/minimal-notebook image metadata") -# Minimal Jupyter Notebook Stack - -GitHub Actions in the https://github.com/jupyter/docker-stacks project builds and pushes this image +GitHub Actions in the project builds and pushes this image to Docker Hub. Please visit the project documentation site for help using and contributing to this image and diff --git a/pyspark-notebook/README.md b/pyspark-notebook/README.md index 75e56cdb9e..d0c18bb75e 100644 --- a/pyspark-notebook/README.md +++ b/pyspark-notebook/README.md @@ -1,10 +1,10 @@ +# Jupyter Notebook Python, Spark Stack + [![docker pulls](https://img.shields.io/docker/pulls/jupyter/pyspark-notebook.svg)](https://hub.docker.com/r/jupyter/pyspark-notebook/) [![docker stars](https://img.shields.io/docker/stars/jupyter/pyspark-notebook.svg)](https://hub.docker.com/r/jupyter/pyspark-notebook/) [![image metadata](https://images.microbadger.com/badges/image/jupyter/pyspark-notebook.svg)](https://microbadger.com/images/jupyter/pyspark-notebook "jupyter/pyspark-notebook image metadata") -# Jupyter Notebook Python, Spark Stack - -GitHub Actions in the https://github.com/jupyter/docker-stacks project builds and pushes this image +GitHub Actions in the project builds and pushes this image to Docker Hub. Please visit the project documentation site for help using and contributing to this image and diff --git a/r-notebook/README.md b/r-notebook/README.md index 313b045057..d412e2e1f5 100644 --- a/r-notebook/README.md +++ b/r-notebook/README.md @@ -1,10 +1,10 @@ +# Jupyter Notebook R Stack + [![docker pulls](https://img.shields.io/docker/pulls/jupyter/r-notebook.svg)](https://hub.docker.com/r/jupyter/r-notebook/) [![docker stars](https://img.shields.io/docker/stars/jupyter/r-notebook.svg)](https://hub.docker.com/r/jupyter/r-notebook/) [![image metadata](https://images.microbadger.com/badges/image/jupyter/r-notebook.svg)](https://microbadger.com/images/jupyter/r-notebook "jupyter/r-notebook image metadata") -# Jupyter Notebook R Stack - -GitHub Actions in the https://github.com/jupyter/docker-stacks project builds and pushes this image +GitHub Actions in the project builds and pushes this image to Docker Hub. Please visit the project documentation site for help using and contributing to this image and diff --git a/scipy-notebook/README.md b/scipy-notebook/README.md index 5c8bc87e07..36ef80755d 100644 --- a/scipy-notebook/README.md +++ b/scipy-notebook/README.md @@ -1,10 +1,10 @@ +# Jupyter Notebook Scientific Python Stack + [![docker pulls](https://img.shields.io/docker/pulls/jupyter/scipy-notebook.svg)](https://hub.docker.com/r/jupyter/scipy-notebook/) [![docker stars](https://img.shields.io/docker/stars/jupyter/scipy-notebook.svg)](https://hub.docker.com/r/jupyter/scipy-notebook/) [![image metadata](https://images.microbadger.com/badges/image/jupyter/scipy-notebook.svg)](https://microbadger.com/images/jupyter/scipy-notebook "jupyter/scipy-notebook image metadata") -# Jupyter Notebook Scientific Python Stack - -GitHub Actions in the https://github.com/jupyter/docker-stacks project builds and pushes this image +GitHub Actions in the project builds and pushes this image to Docker Hub. Please visit the project documentation site for help using and contributing to this image and diff --git a/tagging/README.md b/tagging/README.md index b9c44dba9c..c5eee5d00c 100644 --- a/tagging/README.md +++ b/tagging/README.md @@ -26,7 +26,6 @@ In this section we will briefly describe source code in this folder and give exa `DockerRunner` is a helper class to easily run a docker container and execute commands inside this container: - ```python from .docker_runner import DockerRunner diff --git a/tensorflow-notebook/README.md b/tensorflow-notebook/README.md index 6fd94963b9..2885dde940 100644 --- a/tensorflow-notebook/README.md +++ b/tensorflow-notebook/README.md @@ -1,10 +1,10 @@ +# Jupyter Notebook Deep Learning Stack + [![docker pulls](https://img.shields.io/docker/pulls/jupyter/tensorflow-notebook.svg)](https://hub.docker.com/r/jupyter/tensorflow-notebook/) [![docker stars](https://img.shields.io/docker/stars/jupyter/tensorflow-notebook.svg)](https://hub.docker.com/r/jupyter/tensorflow-notebook/) [![image metadata](https://images.microbadger.com/badges/image/jupyter/tensorflow-notebook.svg)](https://microbadger.com/images/jupyter/tensorflow-notebook "jupyter/tensorflow-notebook image metadata") -# Jupyter Notebook Deep Learning Stack - -GitHub Actions in the https://github.com/jupyter/docker-stacks project builds and pushes this image +GitHub Actions in the project builds and pushes this image to Docker Hub. Please visit the project documentation site for help using and contributing to this image and