diff --git a/draft/multi-data-center-awareness.txt b/draft/multi-data-center-awareness.txt new file mode 100644 index 00000000000..a4707d4e58f --- /dev/null +++ b/draft/multi-data-center-awareness.txt @@ -0,0 +1,126 @@ +.. default-domain:: mongodb + +================================== +Data Center Awareness with MongoDB +================================== + +Data Center awareness and Multi Data Center awareness are attributes of +distributed systems which take into account needs, requirements and +features within the data center and across data centers. MongoDB has +several features which can take advantage of features within a single +data center or across multiple data centers. + +MongoDB supports data center awareness through: + +* Driver level :ref:`Read Preferences ` and :ref:`Write Concerns `. +* :ref:`Replica Set Tags ` +* :ref:`Tag Aware Sharding ` + +.. seealso:: + + A review of :doc:`/core/replication` and :doc:`/core/sharding` + will help you to understand MongoDB's data center awareness + features. + +Data Center Awareness Through Read Preferences and Write Concerns +----------------------------------------------------------------- + +Read and Write Preferences are driver features +which control the conditions under which a driver reads or writes to a +MongoDB database. +You should verify that the specific driver you are using supports these +features before utilizing them for data center awareness. + +Read preferences control whether your application reads from the primary +or a secondary or the *nearest* MongoDB instance in the current topology +of the network. + +Write concerns control how durable the replication of your data is, +from insurance that at least one replica set member +has the data through assurance that a majority or all members of a +replica set have copies of the data. +The MongoDB write concern is not comparable to a transaction, it controls +how long your client application will wait for confirmation that +an update has propagated to however many replica set members you require. + +Read Preferences and Write Concerns are documented in +:doc:`/applications/replication`. + +In the context of data center awareness, read preferences can be used to +control whether a client reads from a primary, a secondary, the nearest +replica set member, or a custom preference composed of +:ref:`replica set tags `. + +In parallel, you can set the write concern on a connection to ensure +that data has been written to one or more members of the replica set, to +a majority of members of the replica set, or to a custom write concern +composed of replica set tags and a quantity of tagged members which must +acknowledge the success of the write. + +While write concerns can be set on a per-connection basis, you can also +define a default write concern for a replica set by updating the +:data:`~local.system.replset.settings.getLastErrorDefaults` +setting of the replica set. + +See :ref:`replica-set-configuration-tag-sets` for sample configurations +of replica sets with tags. + +Within a data center you can use a combination of read preferences and +write concerns to manage where information is written to and where it is +read from. + +You could direct all read activity from a reporting tool to secondary +members of a replica set, lessening the workload on your primary server, +by using a read preference of :readmode:`secondaryPreferred`. + +If your replica set members are distributed across multiple data centers +you can utilize :readmode:`nearest` to direct reads to the closest +replica set member, which may be a primary or secondary member of the +replica set. + +Write activities always start by writing to the primary member of the +replica set. The :term:`write concern` affects how long your client will +wait for confirmation that the data has been written to additional +replica set members. + +As of the :ref:`Fall 2012 driver update `, +officially supported MongoDB drivers will wait for confirmation that +updates have been committed on the primary. +You can specify to wait for additional members to acknowledge a write, +by specifying a number or ``w:majority`` on your connection, or by +changing the :data:`~local.system.replset.settings.getLastErrorDefaults` +setting of the replica set. + +Replica set tags can be combined with write operations to wait for +confirmation that an update has been committed to members of the replica +set matching the specified tags. + +For example, you could tag your replica set members with tags identifying +the rack they occupy in a datacenter, or the type of storage media +(spinning disk, solid state drive, etc.). + +.. TODO:: several examples TK. + +.. _tag-aware-sharding:: + +Multi Data Center Awareness Through Tag Aware Sharding +------------------------------------------------------ + +Shard tagging controls data location, and is complementary to but separate +from replica set tagging. Where replica set tags affect read and write +operations within a given replica set, shard tags determine which +replica sets contain tagged ranges of data. + +Given a shard key with good cardinality, you can assign tags to ranges +of the shard key. You then assign specific ranges to specific shards. + +For example, given a collection with documents containing some sort of +country and national sub-division (state, province) and a shard key composed +of those fields plus some unique identifier (perhaps the BSON ObjectId of +the document), you could assign tag ranges to the shard keys based on +country and sub-division and then assign those chunks to shards whose +primaries are geographically closer to those geographic locations. + +.. TODO:: examples + + diff --git a/source/applications/replication.txt b/source/applications/replication.txt index d0accc79941..1de11251893 100644 --- a/source/applications/replication.txt +++ b/source/applications/replication.txt @@ -25,7 +25,7 @@ Write concern issues the :dbcommand:`getLastError` command after write operations to return an object with error information or confirmation that there are no errors. -The after the :doc:`driver write concern change ` all officially +After the :doc:`driver write concern change ` all officially supported MongoDB drivers enable write concern by default. When enabled by default, write concern confirms write operations only on the primary. @@ -88,6 +88,8 @@ The :data:`~local.system.replset.settings.getLastErrorDefaults` setting affects `. Always ensure that your operations have specified the required write concern for your application. +.. seealso:: :ref:`write-operations-write-concern`, :ref:`connections-write-concern` + .. index:: read preference .. index:: slaveOk @@ -206,10 +208,10 @@ behavior `. See also the This is the default. If the primary is unavailable, read operations produce an error or throw an exception. - :readmode:`primary` read preference modes are not compatible with - read preferences mode that use :ref:`tag sets + The :readmode:`primary` read preference mode is not compatible with + read preference modes that use :ref:`tag sets `. If you specify a tag set - with :readmode:`primary`, the driver produces an error. + with :readmode:`primary`, the driver will produce an error. .. readmode:: primaryPreferred @@ -387,7 +389,7 @@ All interfaces use the same :ref:`member selection logic member to which to direct read operations, basing the choice on read preference mode and tag sets. -For more information on how read preferences :ref:`modes +For more information on how read preference :ref:`modes ` interact with tag sets, see the documentation for each read preference mode. @@ -422,7 +424,7 @@ As a result, MongoDB drivers and :program:`mongos`: connection is *pinned* to this :program:`mongod`. - Attempt to reconnect to a new member, obeying existing :ref:`read - preference modes `, if connection + preference modes `, if the connection to :program:`mongod` is lost. Reconnections are transparent to the application itself. If @@ -498,8 +500,8 @@ Member Selection ```````````````` Clients, by way of their drivers, and :program:`mongos` instances for -sharded clusters periodically update their view of the set's state: which -members are up or down, which is primary, and the latency to each +sharded clusters periodically update their view of the replica set's state: +which members are up or down, which member is primary, and the latency to each :program:`mongod` instance. For any operation that targets a member *other* than the @@ -605,10 +607,10 @@ preferences; clients send all commands to shards' primaries. See Uses for non-Primary Read Preferences ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -You must exercise care when specifying read preference: modes other +You must exercise care when specifying read preferences: modes other than :readmode:`primary` can *and will* return stale data. These secondary queries will not -include most recent write operations to the replica set's +include the most recent write operations to the replica set's :term:`primary`. Nevertheless, there are several common use cases for using non-:readmode:`primary` read preference modes: diff --git a/source/core/write-operations.txt b/source/core/write-operations.txt index 73926261565..4e6e300527a 100644 --- a/source/core/write-operations.txt +++ b/source/core/write-operations.txt @@ -89,7 +89,7 @@ Consider the following levels of conceptual write concern: Before the releases outlined in :ref:`driver-write-concern-change`, this was the default write concern. -.. _write-concern-acknowledged: +.. _write-concern-acknowledged: - receipt *acknowledged*: The :program:`mongod` will confirm the receipt of the write operation, allowing the client to catch @@ -97,7 +97,7 @@ Consider the following levels of conceptual write concern: outlined in :ref:`driver-write-concern-change`, this is the default write concern. [#default-write-concern]_ -.. _write-concern-replica-journaled: +.. _write-concern-replica-journaled: - *journaled*: The :program:`mongod` will confirm the write operation only after it has written the operation to the @@ -111,7 +111,7 @@ Consider the following levels of conceptual write concern: is not fully durable. Require *journaled* as part of the write concern to provide this durability guarantee. -.. _write-concern-replica-acknowledged: +.. _write-concern-replica-acknowledged: :term:`Replica sets ` present an additional layer of consideration for write concern. Basic write concern level affect the @@ -132,8 +132,8 @@ of a replica set. See the :ref:`Write Concern for Replica Sets .. [#default-write-concern] The default write concern is to call :dbcommand:`getLastError` with no arguments. For replica sets, you can define the default write concern settings in the - :data:`getLastErrorDefaults `. If - :data:`getLastErrorDefaults ` does + :data:`~local.system.replset.settings.getLastErrorDefaults` + If :data:`~local.system.replset.settings.getLastErrorDefaults` does not define a default write concern setting, :dbcommand:`getLastError` defaults to basic receipt acknowledgment. @@ -155,8 +155,9 @@ document's ``err`` field contains either: The definition of a "successful write" depends on the arguments specified to :dbcommand:`getLastError`, or in replica sets, the -configuration of :data:`getLastErrorDefaults -`. When deciding the level of write +configuration of +:data:`~local.system.replset.settings.getLastErrorDefaults`. +When deciding the level of write concern for your application, become familiar with the :ref:`write-concern-considerations`. @@ -365,7 +366,7 @@ they do, adds a :data:`~collStats.paddingFactor` so that the documents have room to grow on subsequent writes. The :data:`~collStats.paddingFactor` indicates the padding for new inserts and moves. -.. versionadded:: 2.2 +.. versionadded:: 2.2 You can use the :dbcommand:`collMod` command with the :collflag:`usePowerOf2Sizes` flag so that MongoDB allocates document space in sizes that are powers of 2. This helps @@ -373,7 +374,7 @@ padding for new inserts and moves. result of deletions or document relocations. As with all padding, using document space allocations with power of 2 sizes minimizes, but does not eliminate, document movements. - + To check the current :data:`~collStats.paddingFactor` on a collection, you can run the :dbcommand:`db.collection.stats()` command in the :program:`mongo` shell, as in the following example: @@ -389,7 +390,7 @@ example: .. code-block:: none - padding size = (paddingFactor - 1) * . + padding size = (paddingFactor - 1) * . For example, a :data:`~collStats.paddingFactor` of ``1.0`` specifies no padding whereas a paddingFactor of ``1.5`` specifies a padding size of ``0.5`` or 50 @@ -408,10 +409,10 @@ until you perform a :dbcommand:`compact` or a .. note:: - The following operations remove padding: - + The following operations remove padding: + - :dbcommand:`compact`, - - :dbcommand:`repairDatabase`, and + - :dbcommand:`repairDatabase`, and - initial replica sync operations. However, with the :dbcommand:`compact` command, you can run the @@ -437,8 +438,8 @@ until you perform a :dbcommand:`compact` or a .. seealso:: - :ref:`faq-developers-manual-padding` - - - `Fast Updates with MongoDB with in-place Updates `_ (blog post) + + - `Fast Updates with MongoDB with in-place Updates `_ (blog post) Architecture ------------ diff --git a/source/reference/connection-string.txt b/source/reference/connection-string.txt index 51ecbbcdd93..80ebc7210eb 100644 --- a/source/reference/connection-string.txt +++ b/source/reference/connection-string.txt @@ -208,13 +208,15 @@ connecting to the MongoDB deployment. connection to become available. For default values, see the :doc:`/applications/drivers` documentation. +.. _connections-write-concern: + Write Concern Options ~~~~~~~~~~~~~~~~~~~~~ :ref:`Write concern ` describes the kind of assurances -:that the program:`mongod` and the driver provide to the application -:regarding the success and durability of the write operation. For a -:full explanation of write concern and write operations in general see +that the program:`mongod` and the driver provide to the application +regarding the success and durability of the write operation. For a +full explanation of write concern and write operations in general see the: :doc:`/core/write-operations`: .. data:: uri.w @@ -243,7 +245,7 @@ the: :doc:`/core/write-operations`: Provides basic acknowledgment of write operations. By specifying ``1``, you require that a standalone - :program:`mongod` instance, or a the primary for :term:`replica + :program:`mongod` instance, or the primary for :term:`replica sets `, acknowledge all write operations. For drivers released after the :doc:`default write concern change `, this is the default diff --git a/source/reference/method/sh.addShardTag.txt b/source/reference/method/sh.addShardTag.txt index 9eef2feca6a..07ef0aa9cdd 100644 --- a/source/reference/method/sh.addShardTag.txt +++ b/source/reference/method/sh.addShardTag.txt @@ -8,23 +8,35 @@ sh.addShardTag() .. versionadded:: 2.2 - :param shard: Specifies the name of the shard that you want to give - a specific tag. + :param string shard: Specifies the name of the shard that you want to give + a specific tag. - :param tag: Specifies the name of the tag that you want to add to - the shard. + :param string tag: Specifies the name of the tag that you want to add to + the shard. :method:`sh.addShardTag()` associates a shard with a tag or - identifier. MongoDB can use these identifiers, to "home" or attach - (i.e. with :method:`sh.addTagRange()`) specific data to a specific - shard. + identifier. + MongoDB uses these identifiers to direct :term:`chunks ` + that fall within a tagged range to specific shards. + + Chunks are associated with a tag range using the + :method:`sh.addTagRange()` method. Always issue :method:`sh.addShardTag()` when connected to a - :program:`mongos` instance. The following example adds three tags, - ``LGA``, ``EWR``, and ``JFK``, to three shards: + :program:`mongos` instance. + + .. example:: + + The following example adds three tags, ``NYC``, ``LAX``, and + ``NRT``, to three shards: + + .. code-block:: javascript + + sh.addShardTag("shard0000", "NYC") + sh.addShardTag("shard0001", "LAX") + sh.addShardTag("shard0002", "NRT") - .. code-block:: javascript + .. seealso:: - sh.addShardTag("shard0000", "LGA") - sh.addShardTag("shard0001", "EWR") - sh.addShardTag("shard0002", "JFK") + :method:`sh.addTagRange()`, + :method:`sh.removeShardTag()` diff --git a/source/reference/method/sh.addTagRange.txt b/source/reference/method/sh.addTagRange.txt index 013083c5490..284e5f847bc 100644 --- a/source/reference/method/sh.addTagRange.txt +++ b/source/reference/method/sh.addTagRange.txt @@ -12,24 +12,33 @@ sh.addTagRange() ``.`` of the sharded collection that you would like to tag. - :param minimum: Specifies the minimum value of the :term:`shard - key` range to include in the tag. Specify the - minimum value in the form of + :param minimum: Specifies the minimum value of the + :term:`shard key` range to include in the tag. + Specify the minimum value in the form of ``:``. + This value must be of the same BSON type or types + as the shard key. :param maximum: Specifies the maximum value of the shard key range - to include in the tag. Specify the minimum value + to include in the tag. Specify the maximum value in the form of ``:``. + This value must be of the same BSON type or types + as the shard key. - :param tag: Specifies the name of the tag to attach the range - specified by the ``minimum`` and ``maximum`` arguments - to. + :param string tag: Specifies the name of the tag to attach the range + specified by the ``minimum`` and ``maximum`` + arguments to. :method:`sh.addTagRange()` attaches a range of values of the shard key to a shard tag created using the :method:`sh.addShardTag()` - helper. Use this operation to ensure that the documents that exist + method. Use this operation to ensure that the documents that exist within the specified range exist on shards that have a matching tag. Always issue :method:`sh.addTagRange()` when connected to a :program:`mongos` instance. + + .. seealso:: + + :method:`sh.addShardTag()`, + :method:`sh.removeShardTag()` diff --git a/source/reference/method/sh.removeShardTag.txt b/source/reference/method/sh.removeShardTag.txt index 878fc0abebb..a3dd81d3ec1 100644 --- a/source/reference/method/sh.removeShardTag.txt +++ b/source/reference/method/sh.removeShardTag.txt @@ -8,12 +8,18 @@ sh.removeShardTag() .. versionadded:: 2.2 - :param shard: Specifies the name of the shard that you want - to remove a tag from. + :param string shard: Specifies the name of the shard that you want + to remove a tag from. - :param tag: Specifies the name of the tag that you want to remove - from the shard. + :param string tag: Specifies the name of the tag that you want to remove + from the shard. - Removes the association between a tag and a shard. Always issue - :method:`sh.removeShardTag()` when connected to a :program:`mongos` - instance. + Removes the association between a tag and a shard. + + Always issue :method:`sh.removeShardTag()` when connected to a + :program:`mongos` instance. + + .. seealso:: + + :method:`sh.addShardTag()`, + :method:`sh.addTagRange()` diff --git a/source/reference/replica-configuration.txt b/source/reference/replica-configuration.txt index f7946de4e63..0bfb645f935 100644 --- a/source/reference/replica-configuration.txt +++ b/source/reference/replica-configuration.txt @@ -402,7 +402,7 @@ Tag sets provide custom and configurable :term:`write concern` and :term:`read preferences ` for a :term:`replica set`. This section will outline the process for specifying tags for a replica set, for more information see the -full documentation of the behavior of :ref:`tags sets write concern +full documentation of the behavior of :ref:`tags sets for write concern ` and :ref:`tag sets for read preference `. diff --git a/source/release-notes/drivers-write-concern.txt b/source/release-notes/drivers-write-concern.txt index 26ab9053039..00facad172f 100644 --- a/source/release-notes/drivers-write-concern.txt +++ b/source/release-notes/drivers-write-concern.txt @@ -44,8 +44,9 @@ drivers: :dbcommand:`getLastError` without arguments, which is logically equivalent to the ``w: 1`` option; however, this operation allows :term:`replica set` users to override the default write concern - with the :data:`getLastErrorDefaults - ` setting in the + with the + :data:`getLastErrorDefaults <~local.system.replset.settings.getLastErrorDefaults>` + setting in the :doc:`/reference/replica-configuration`. .. _write-concern-change-releases: