DOCS-15702 Static Unlike Topologies (#101)

* DOCS-15702 Documents sync of Static Unlike Topologies

* Refactors sharding options

* Updates sharding options

* Expanded behavior section for static unlike

* Updates limitations

* Rename behavior

* Updates sharding table

* Adds sharding example

* Fixes build issue

* Fixes build issue

* Fixes per Jeff

* Fixes per Jeff

* Fixes per Jeff

* Fixes per Frederic Vitzikam

* Fixes per Frederic Vitzikam

* Fixes per Frederic Vitzikam

* Fixes per Frederic Vitzikam

* Fixes build issue

* Fixes per Frederic Vitzikam

* Fixes per Frederic Vitzikam

* Fixes per Frederic Vitzikam

* Fixes per Frederic Vitzikam

* Refactors synchronization to sync

* Refactors synchronization to sync

* Fixes per Frederic

* Fix per Frederic

Author: kennethdyer

source/includes/api/requests/start-rs-shard.sh

@@ -0,0 +1,21 @@
+curl localhost:27182/api/v1/start -XPOST \
+--data '
+   {
+      "source": "cluster0",
+      "destination": "cluster1",
+      "sharding": {
+         "createSupportingIndexes": true,
+         "shardingEntries": [
+            {
+                "database": "accounts",
+                "collection": "us_east",
+                "shardCollection": {
+                   "key": [
+                      { "location": 1 },
+                      { "region": 1 },
+                   ]
+                }
+            }
+         ]
+      }
+   } '

source/reference/api/start.txt

@@ -94,12 +94,30 @@ Request Body Parameters
    * - ``reversible``
      - boolean
      - Optional
-     - If set to ``true``, enables the synchronization operation to be
-       reversed. For more information, see the :ref:`reverse
-       <c2c-api-reverse>` endpoint.
+     - If set to ``true``, enables the sync operation to be
+       reversed. 
+
+       This option is not supported for the following configurations:
+
+       * Sync from a replica set to a sharded cluster
+
+       * Sync sharded clusters that have different numbers of shards
+       
+       For more information, see the :ref:`reverse <c2c-api-reverse>` endpoint.
 
        Default value is ``false``.
 
+   * - ``sharding``
+     - document
+     - Optional
+     - Configure sync between a replica set and sharded cluster.
+       Sync from a replica set to a sharded cluster requires this
+       option.
+
+       For more information, see :ref:`c2c-api-start-sharding`. 
+
+       .. versionadded:: 1.1
+
    * - ``enableUserWriteBlocking``
      - boolean
      - Optional
@@ -111,6 +129,69 @@ Request Body Parameters
 
        Default value is ``false``.
 
+
+.. _c2c-api-start-sharding:
+
+Sharding Parameters
+~~~~~~~~~~~~~~~~~~~
+
+.. versionadded:: 1.1
+
+To sync from a replica set to a sharded cluster, set the 
+``sharding`` option to shard collections on the destination cluster.
+
+The ``sharding`` option has the following parameters:
+
+.. list-table::
+   :header-rows: 1
+   :stub-columns: 1
+   :widths: 15 15 70
+
+   * - Parameter
+     - Type
+     - Description
+
+   * - ``createSupportedIndexes``
+     - boolean
+     - Optional. Sets whether sync creates a supporting index 
+       for the shard key, if none exists.  Defaults to ``false``.
+
+   * - ``shardingEntries``
+     - array of documents
+     - Required. Sets the namespace and key of collections to shard 
+       during sync.
+
+       Collections not included in this array sync to unsharded 
+       collections on the destination cluster.  If set with an empty array, 
+       no collections are sharded.
+
+   * - | ``shardingEntries``
+       | ``.collection``
+     - string
+     - Sets the collection to shard. 
+
+   * - | ``shardingEntries``
+       | ``.database``
+     - string
+     - Sets the database of the collection to shard.
+ 
+   * - | ``shardingEntries``
+       | ``.shardCollection``
+     - document 
+     - Sets the shard key to generate on the destination cluster.
+
+   * - | ``shardingEntries``
+       | ``.shardCollection``
+       | ``.key``
+     - array 
+     - Sets the fields to use for the shard key.
+
+       For more information, see :ref:`shard-key`.
+
+``mongosync`` throws an error if the ``sharding`` option is not set when
+syncing from a replica set to a sharded cluster.  ``mongosync`` also
+throws an error if the ``sharding`` option is set with any other configuration.
+
 Response
 --------
 
@@ -151,12 +232,133 @@ Request
 Response
 ~~~~~~~~
 
+.. literalinclude:: /includes/api/responses/success.json
+   :language: json
+   :copyable: false
+
+Example 3 - Start Sync from Replica Set to Sharded Cluster
+----------------------------------------------------------
+
+Request
+~~~~~~~
+
+.. literalinclude:: /includes/api/requests/start-rs-shard.sh
+   :language: shell
+
+Response
+~~~~~~~~
+
 .. literalinclude:: /includes/api/responses/success.json
    :language: json
    :copyable: false
 
 Behavior
 --------
 
+State
+~~~~~
+
 If the ``start`` request is successful, ``mongosync`` enters the
 ``RUNNING`` state.
+
+Pre-Split Chunks
+~~~~~~~~~~~~~~~~
+
+.. versionadded:: 1.1
+
+When ``mongosync`` syncs to a sharded cluster, it pre-splits chunks for
+sharded collections on the destination cluster.  This is supported in the
+following configurations:
+
+* Sync from a replica set to a sharded cluster.
+
+* Sync between sharded clusters that differ in the number of shards.
+
+
+Shard Replica Sets 
+~~~~~~~~~~~~~~~~~~
+
+.. versionadded:: 1.1
+
+Sync from a replica set to a sharded cluster requires the 
+``sharding`` option. This option configures how ``mongosync`` shards
+collections.
+
+The ``sharding.shardEntries`` array specifies the collections to shard.
+Collections that are not listed in this array replicate as unsharded.
+
+Supporting Indexes
+~~~~~~~~~~~~~~~~~~
+
+.. versionadded:: 1.1
+
+``mongosync`` syncs indexes from the source cluster to the destination
+cluster.  But, when syncing from a replica set to a sharded cluster,
+``mongosync`` may require an additional index to support the shard key,
+which may not exist on the source cluster.
+
+``mongosync`` can create supporting indexes for sharded collections during sync.
+This is done by setting the ``sharding.createSupportingIndexes`` option.
+
+When ``sharding.createSupportingIndexes`` is ``false`` (the default): 
+
+* Each shard key you provide for the ``sharding.shardEntries`` option
+  must have an existing index on the source cluster.
+
+* One of the indexes used for the shard key must have simple collation if the 
+  collection uses any other collation.
+
+* To use a unique index in the shard key, you must specify its uniqueness when
+  you create the index on the source cluster. 
+
+* Unique indexes on the source cluster that are incompatible with the requested
+  shard key on the destination cluster, such as a unique index on the source
+  that does not contain the requested shard key as a prefix on the destination,
+  can cause ``mongosync`` to fail.
+
+When ``sharding.createSupportingIndexes`` is ``true``: 
+
+* If the supporting indexes exist on the source cluster, ``mongosync`` 
+  syncs the indexes to the destination cluster and uses them 
+  as shard keys. 
+
+* If the supporting indexes don't exist, ``mongosync`` creates them on the
+  destination cluster.
+
+The ``sharding.createSupportingIndexes`` option affects all sharded collections.
+
+Rename During Sync
+~~~~~~~~~~~~~~~~~~
+
+.. versionadded:: 1.1
+
+Collections listed in the ``sharding.shardEntries`` array 
+when synced from a replica set to a sharded cluster 
+become sharded collections on the destination cluster.
+
+Renaming a collection (such as with the :dbcommand:`renameCollection` command)
+on the source cluster after calling ``start`` but before ``mongosync`` begins 
+to copy the collection can block the collection from sharding on the destination.
+
+.. note:: 
+
+   Renaming collections to use a different database while syncing from a
+   replica set to a sharded cluster is not supported.
+    
+
+To check whether it is safe to rename collections, call the 
+:ref:`c2c-api-progress` endpoint and check the value of the 
+``collectionCopy.estimatedCopiedBytes`` field in the return document.  
+
+* A value of 0 indicates that ``mongosync`` has not started to copy the
+  collection.
+
+  Renaming a collection at this point may result in an unsharded collection 
+  on the destination cluster, as the transition to copying can happen before
+  the rename takes effect on the source.
+
+* A value greater than 0 indicates that ``mongosync`` has started the copy.
+  Renaming the collection from this point on does not block
+  its sharding on the destination cluster, even in the event  of a crash.
+
+

source/reference/limitations.txt

@@ -83,13 +83,20 @@ Unsupported Collection Types
 Sharded Clusters
 ----------------
 
-- The :ref:`shard topologies <sharding-introduction>` must be the same
-  in the source and target clusters. The following configurations are
-  not supported.
-
-  - Replica set to sharded cluster.
-  - Sharded cluster to replica set.
-  - Unequal numbers of source and destination shards.
+- ``mongosync`` does not support sync from a sharded cluster
+  to a replica set.
+- Sync from a replica set to a sharded cluster has the following
+  limitations:
+
+  - Collections included in the ``sharding.shardingEntries`` option for the
+    :ref:`c2c-api-start` command cannot be renamed to use a different
+    database while ``mongosync`` is running.
+  - When using the ``sharding.createSupprtingIndexes`` option to create indexes
+    supporting shard keys on the destination cluster during sync, 
+    you cannot create these indexes afterwards on the source cluster.  
+    
+    The index must either exist before ``mongosync`` starts or be created after
+    the migration is complete and ``mongosync`` has stopped.
 - Within a collection, the ``_id`` field must be unique across all of
   the shards in the cluster. See :ref:`sharded-clusters-unique-indexes`
   for more details.
@@ -107,7 +114,6 @@ Sharded Clusters
   <sharding-shard-key-indexes>` is one lower than normal, 63 instead of
   64.
 
-
 Reversing
 ---------