Merge pull request #273 from kragniz/sphinx

Automatic merge from submit-queue.

Initial sphinx documentation

This is a start to documentation managed by Sphinx.

Related to #268.

```release-note
none
```

Author: jetstack-ci-bot

Makefile

@@ -54,6 +54,10 @@ verify: .hack_verify dep_verify go_verify helm_verify
 	@echo Running generated client checker:
 	@${HACK_DIR}/verify-client-gen.sh
 
+doc_verify:
+	make -C docs html
+	make -C docs check
+
 dep_verify:
 	${HACK_DIR}/verify-deps.sh
 

README.md

@@ -1,4 +1,4 @@
-# Navigator - managed DBaaS on Kubernetes [![Build Status Widget]][Build Status]
+# Navigator - self managed DBaaS on Kubernetes [![Build Status Widget]][Build Status]
 
 Navigator is a Kubernetes extension for managing common stateful services on
 Kubernetes. It is implemented as a custom apiserver that operates behind
@@ -10,47 +10,13 @@ other resource in Kubernetes core. This means you can manage fine-grained
 permissions via conventional RBAC rules, allowing you to offer popular but
 complex services "as a Service" within your organisation.
 
-To get started, jump to the [quick-start guide](docs/quick-start).
+For more in-depth information and to get started, jump to the [docs](https://navigator-dbaas.readthedocs.io).
 
-## Design
+Here's a quick demo of creating, scaling and deleting a Cassandra database:
 
-As well as following "the operator model", Navigator additionally introduces
-the concept of 'Pilots' - small 'nanny' processes that run inside each pod
-in your application deployment. These Pilots are responsible for managing the
-lifecycle of your underlying application process (e.g. an Elasticsearch JVM
-process) and periodically report state information about the individual node
-back to the Navigator API.
+![](docs/images/demo.gif)
 
-By separating this logic into it's own binary that is run alongside each node,
-in certain failure events the Pilot is able to intervene in order to help
-prevent data loss, or otherwise update the Navigator API with details of the
-failure so that navigator-controller can take action to restore service.
-
-Navigator has a few unique traits that differ from similar projects (such as
-elasticsearch-operator, etcd-operator etc).
-
-- **navigator-apiserver** - this takes on a similar role to `kube-apiserver`.
-It is responsible for storing and coordinating all of the state stored for
-Navigator. It requires a connection to an etcd cluster in order to do this. In
-order to make Navigator API types generally consumable to users of your cluster,
-it registers itself with kube-aggregator. It performs validation of your
-resources, as well as performing conversions between API versions which allow
-us to maintain a stable API without hindering development.
-
-- **navigator-controller** - the controller is akin to `kube-controller-manager`.
-It is responsible for actually realising your deployments within the Kubernetes
-cluster. It can be seen as the 'operator' for the various applications
-supported by `navigator-apiserver`.
-
-- **pilots** - the pilot is responsible for managing each database process.
-  Currently Navigator has two types: `pilot-elasticsearch` and
-  `pilot-cassandra`.
-
-## Architecture
-
-![alt text](docs/arch.jpg)
-
-## Supported applications
+## Supported databases
 
 Whilst we aim to support as many common applications as possible, it does take
 a certain level of operational knowledge of the applications in question in
@@ -62,38 +28,11 @@ Please search for or create an issue for the application in question you'd like
 to see a part of Navigator, and we can begin discussion on implementation &
 planning.
 
-| Name          | Version   | Status      | Notes                                                       |
-| ------------- | --------- | ----------- | ----------------------------------------------------------- |
-| Elasticsearch | 5.x       | Alpha       | [more info](docs/supported-types/elasticsearch-cluster.md)  |
-| Cassandra     | 3.x       | Alpha       | [more info](docs/supported-types/cassandra-cluster.md)      |
-| Couchbase     |           | Coming soon |                                                             |
-
-## Links
-
-* [Quick-start](docs/quick-start)
-* [Developing quick-start](docs/developing.md)
-* [Resource types](docs/supported-types/README.md)
-  * [ElasticsearchCluster](docs/supported-types/elasticsearch-cluster.md)
-
-
-## E2E Testing
-
-Navigator has an end-to-end test suite which verifies that Navigator can be
-installed [as documented in the quick start guide](docs/quick-start).  The
-tests are run on a Minikube cluster.  Run the tests using the following
-sequence of commands:
-
-```
-minikube start
-# This ensures that the Docker image will be built in the Minikube VM
-eval $(minikube docker-env)
-# Override the Docker image tag so that it is built as :latest
-# (the tag used in the documented deployment)
-# XXX: This is a hack.
-# Better if we had a helm chart in the documentation,
-# so that we could provide an alternative navigator image and tag.
-make BUILD_TAG=latest e2e-test
-```
+| Name          | Version   | Status      | Notes                                                                             |
+| ------------- | --------- | ----------- | --------------------------------------------------------------------------------- |
+| Elasticsearch | 5.x       | Alpha       | [more info](https://navigator-dbaas.readthedocs.io/en/latest/elasticsearch.html)  |
+| Cassandra     | 3.x       | Alpha       | [more info](https://navigator-dbaas.readthedocs.io/en/latest/cassandra.html)      |
+| Couchbase     |           | Coming soon |                                                                                   |
 
 ## Credits
 

docs/Makefile

@@ -0,0 +1,27 @@
+# You can set these variables from the command line.
+SPHINXOPTS    =
+SPHINXBUILD   = $(VENV_PATH)/bin/python -msphinx
+SPHINXPROJ    = Navigator
+SOURCEDIR     = .
+VENV_PATH     = .venv
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: .venv Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.venv:
+	virtualenv -p $(shell which python3) $(VENV_PATH)
+	$(VENV_PATH)/bin/pip install -r requirements.txt
+	touch .venv
+
+check: .venv
+	$(SPHINXBUILD) "$(SOURCEDIR)" "$(BUILDDIR)" -b linkcheck
+	$(SPHINXBUILD) "$(SOURCEDIR)" "$(BUILDDIR)" -b spelling

docs/cassandra.rst

@@ -0,0 +1,98 @@
+Cassandra
+=========
+
+Example cluster definition
+--------------------------
+
+Example ``CassandraCluster`` resource:
+
+.. include:: quick-start/cassandra-cluster.yaml
+   :literal:
+
+Cassandra Across Multiple Availability Zones
+--------------------------------------------
+
+With rack awareness
+~~~~~~~~~~~~~~~~~~~
+
+Navigator supports running Cassandra with
+`rack and datacenter-aware replication <https://docs.datastax.com/en/cassandra/latest/cassandra/architecture/archDataDistributeReplication.html>`_
+To deploy this, you must run a ``nodePool`` in each availability zone, and mark each as a separate Cassandra rack.
+
+The
+`nodeSelector <(https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector>`_
+field of a nodePool allows scheduling the nodePool to a set of nodes matching labels.
+This should be used with a node label such as
+`failure-domain.beta.kubernetes.io/zone <https://kubernetes.io/docs/reference/labels-annotations-taints/#failure-domainbetakubernetesiozone>`_.
+
+The ``datacenter`` and ``rack`` fields mark all Cassandra nodes in a nodepool as being located in that datacenter and rack.
+This information can then be used with the
+`NetworkTopologyStrategy <http://cassandra.apache.org/doc/latest/architecture/dynamo.html#network-topology-strategy>`_
+keyspace replica placement strategy.
+If these are not specified, Navigator will select an appropriate name for each: ``datacenter`` defaults to a static value, and ``rack`` defaults to the nodePool's name.
+
+As an example, the nodePool section of a CassandraCluster spec for deploying into GKE in europe-west1 with rack awareness enabled:
+
+.. code-block:: yaml
+
+    nodePools:
+    - name: "np-europe-west1-b"
+      replicas: 3
+      datacenter: "europe-west1"
+      rack: "europe-west1-b"
+      nodeSelector:
+        failure-domain.beta.kubernetes.io/zone: "europe-west1-b"
+      persistence:
+        enabled: true
+        size: "5Gi"
+        storageClass: "default"
+    - name: "np-europe-west1-c"
+      replicas: 3
+      datacenter: "europe-west1"
+      rack: "europe-west1-c"
+      nodeSelector:
+        failure-domain.beta.kubernetes.io/zone: "europe-west1-c"
+      persistence:
+        enabled: true
+        size: "5Gi"
+        storageClass: "default"
+    - name: "np-europe-west1-d"
+      replicas: 3
+      datacenter: "europe-west1"
+      rack: "europe-west1-d"
+      nodeSelector:
+        failure-domain.beta.kubernetes.io/zone: "europe-west1-d"
+      persistence:
+        enabled: true
+        size: "5Gi"
+        storageClass: "default"
+
+Without rack awareness
+~~~~~~~~~~~~~~~~~~~~~~
+
+Since the default rack name is equal to the nodepool name,
+simply set the rack name to the same static value in each nodepool to disable rack awareness.
+
+A simplified example:
+
+.. code-block:: yaml
+
+    nodePools:
+    - name: "np-europe-west1-b"
+      replicas: 3
+      datacenter: "europe-west1"
+      rack: "default-rack"
+      nodeSelector:
+        failure-domain.beta.kubernetes.io/zone: "europe-west1-b"
+    - name: "np-europe-west1-c"
+      replicas: 3
+      datacenter: "europe-west1"
+      rack: "default-rack"
+      nodeSelector:
+        failure-domain.beta.kubernetes.io/zone: "europe-west1-c"
+    - name: "np-europe-west1-d"
+      replicas: 3
+      datacenter: "europe-west1"
+      rack: "default-rack"
+      nodeSelector:
+        failure-domain.beta.kubernetes.io/zone: "europe-west1-d"