From 6de3b08ac7fd36f20d230536d2b40c9bf0ab1250 Mon Sep 17 00:00:00 2001 From: kay Date: Wed, 6 May 2015 19:46:33 -0400 Subject: [PATCH 1/3] DOCS-5324 quorum reads with findAndModify This cherry-picks 046152322b4fae3a98835ed74b718ef7d88f6da2 to apply it again, undoing the reversion in cf6ab08992633803fa79ee8d8a863200bea435fa. The changes in source/reference/write-concern.txt have been ignored in favor of more recent changes. --- .../fact-findAndModify-update-comparison.rst | 2 +- .../steps-findAndModify-quorum-reads.yaml | 92 +++++++++++++++++++ source/includes/toc-crud-tutorials.yaml | 4 + source/reference/command/findAndModify.txt | 2 + .../method/db.collection.findAndModify.txt | 2 + source/reference/write-concern.txt | 3 +- .../perform-findAndModify-quorum-reads.txt | 82 +++++++++++++++++ 7 files changed, 185 insertions(+), 2 deletions(-) create mode 100644 source/includes/steps-findAndModify-quorum-reads.yaml create mode 100644 source/tutorial/perform-findAndModify-quorum-reads.txt diff --git a/source/includes/fact-findAndModify-update-comparison.rst b/source/includes/fact-findAndModify-update-comparison.rst index c0c9f376df6..cdeaa909f86 100644 --- a/source/includes/fact-findAndModify-update-comparison.rst +++ b/source/includes/fact-findAndModify-update-comparison.rst @@ -13,7 +13,7 @@ When updating a document, |operation| and the method, you cannot specify which single document to update when multiple documents match. -- By default, |operation| method returns |return-object|. To +- By default, |operation| returns |return-object|. To obtain the updated document, use the ``new`` option. The :method:`~db.collection.update()` method returns a diff --git a/source/includes/steps-findAndModify-quorum-reads.yaml b/source/includes/steps-findAndModify-quorum-reads.yaml new file mode 100644 index 00000000000..9e1af3d6cfa --- /dev/null +++ b/source/includes/steps-findAndModify-quorum-reads.yaml @@ -0,0 +1,92 @@ +title: Create a unique index. +stepnum: 1 +ref: quorum-read-unique-index +pre: | + Create a unique index on the fields that will be used to specify an + exact match in the :method:`db.collection.findAndModify()` operation. + + This tutorial will use an exact match on the ``sku`` field. As such, + create a unique index on the ``sku`` field. +action: + language: javascript + code: | + db.products.createIndex( { sku: 1 }, { unique: true } ) +--- +title: Use ``findAndModify`` to read committed data. +stepnum: 2 +ref: quorum-read-findAndModify +pre: | + Use the :method:`db.collection.findAndModify()` method to make a + trivial update to the document you want to read and return the + modified document. To specify the document to read, you must use an + exact match query that is supported by a unique index. + + The following :method:`~db.collection.findAndModify()` operation + specifies an exact match on the uniquely indexed field ``sku`` and + increments the field named ``_dummy_field`` in the matching document. +action: + language: javascript + code: | + var updatedDocument = db.products.findAndModify( + { + query: { sku: "abc123" }, + update: { $inc: { _dummy_field: 1 } }, + new: true + } + ); +--- +title: Issue ``getLastError`` to determine quorum read. +stepnum: 3 +ref: quorum-read-gle +pre: | + To determine if the read from the + :method:`~db.collection.findAndModify()` was a true quorum read, + issue a :dbcommand:`getLastError` command with :writeconcern:`w: + "majority"`. + + While not necessary to the procedure, the + :dbcommand:`getLastError` command also includes a :ref:`wc-wtimeout` + value of ``5000`` milliseconds to prevent the operation from blocking + forever if the write cannot propagate to a majority of voting members. +action: + language: javascript + code: | + var gle = db.runCommand( { getLastError: 1, w: "majority", wtimeout: 5000 } ); + + if ( (gle.ok != 1) || (gle.err != null) ) { + print("The document returned from findAndModify() may reflect data that is not durable and subject to rollback."); + printjson(gle); + } else { + printjson(updatedDocument); + } +post: | + The :dbcommand:`getLastError` determines whether the update from + the :method:`~db.collection.findAndModify()` operation has + propagated to the majority of the replica set's voting members. + + Even in situations where two nodes in the replica set believe that + they are the primary, only one will be able to complete the write + with :writeconcern:`w: "majority"`. As such, the + :dbcommand:`getLastError` with :writeconcern:`w: "majority"` write + concern can confirm whether the client has connected to the true + primary to perform the :method:`~db.collection.findAndModify()` + operation. + + .. note:: + :ref:`wc-wtimeout` causes :dbcommand:`getLastError` to return + with an error after the specified time limit, even if the + required write concern will eventually succeed. As such, if the + :dbcommand:`getLastError` times out, it cannot determine whether + the document returned by :method:`~db.collection.findAndModify()` + is or is not the result of a quorum read. For quorum reads, + ignore the document returned by + :method:`~db.collection.findAndModify()` and repeat the + :method:`~db.collection.findAndModify()` and + :dbcommand:`getLastError`. + + Since the quorum read procedure only increments a dummy field in + the document, you can safely repeat the + :method:`~db.collection.findAndModify()` and + :dbcommand:`getLastError`, adjusting the :ref:`wc-wtimeout` as + necessary. +... diff --git a/source/includes/toc-crud-tutorials.yaml b/source/includes/toc-crud-tutorials.yaml index 665064a3e54..6ef7191a3fe 100644 --- a/source/includes/toc-crud-tutorials.yaml +++ b/source/includes/toc-crud-tutorials.yaml @@ -52,4 +52,8 @@ file: /tutorial/create-an-auto-incrementing-field description: | Describes how to create an incrementing sequence number for the ``_id`` field using a Counters Collection or an Optimistic Loop. +--- +file: /tutorial/perform-findAndModify-quorum-reads +description: | + Perform quorum reads using ``findAndModify``. ... diff --git a/source/reference/command/findAndModify.txt b/source/reference/command/findAndModify.txt index 40a24ecbb22..a5a9b2cc95b 100644 --- a/source/reference/command/findAndModify.txt +++ b/source/reference/command/findAndModify.txt @@ -390,3 +390,5 @@ The method returns the deleted document: }, "ok" : 1 } + +.. seealso:: :doc:`/tutorial/perform-findAndModify-quorum-reads` diff --git a/source/reference/method/db.collection.findAndModify.txt b/source/reference/method/db.collection.findAndModify.txt index a827b1a6eff..4ffa59d551f 100644 --- a/source/reference/method/db.collection.findAndModify.txt +++ b/source/reference/method/db.collection.findAndModify.txt @@ -286,3 +286,5 @@ The method returns the deleted document: "state" : "active", "rating" : 3 } + +.. seealso:: :doc:`/tutorial/perform-findAndModify-quorum-reads` diff --git a/source/reference/write-concern.txt b/source/reference/write-concern.txt index aff8bda6f62..9b57e2f2638 100644 --- a/source/reference/write-concern.txt +++ b/source/reference/write-concern.txt @@ -71,7 +71,8 @@ The ``w`` option requests acknowledgement that the write operation has propagated to a specified number of :program:`mongod` instances or to :program:`mongod` instances with specified tags. -The ``w`` option accepts the following values: +Using the ``w`` option, the following ``w: `` write concerns are +available: .. note:: diff --git a/source/tutorial/perform-findAndModify-quorum-reads.txt b/source/tutorial/perform-findAndModify-quorum-reads.txt new file mode 100644 index 00000000000..f8ea5f02b85 --- /dev/null +++ b/source/tutorial/perform-findAndModify-quorum-reads.txt @@ -0,0 +1,82 @@ +==================================== +Perform Quorum Reads on Replica Sets +==================================== + +.. default-domain:: mongodb + +Overview +-------- + +When reading from replica sets, in some edge cases, clients can +read stale data even when specifying a :readmode:`primary` read +preference. [#edge-cases-2-primaries]_ Clients may also see results of +writes before they are made durable and before they have propagated to +enough replica set members to avoid rollbacks. + +This tutorial outlines a procedure that uses +:method:`db.collection.findAndModify()` to read data that is not stale +and cannot be rolled back. To do so, the procedure uses the +:method:`db.collection.findAndModify()` method to modify a dummy field +in a document and issues a :dbcommand:`getLastError` command to confirm +that the :method:`db.collection.findAndModify()` operation has +propagated to enough members to avoid rollbacks. Specifically, the +procedure requires that: + +- :method:`db.collection.findAndModify()` use an **exact** match query, + and a :doc:`unique index ` **must exist** to + satisfy the query. + +- :method:`db.collection.findAndModify()` must actually modify a + document; i.e. result in a change to the document. + +- :dbcommand:`getLastError` must use the write concern + :writeconcern:`w: "majority"`. + +Prerequisites +------------- + +This tutorial reads from a collection named ``products``. Initialize +the collection using the following operation. + +.. code-block:: javascript + + db.products.insert( [ + { + _id: 1, + sku: "xyz123", + description: "hats", + available: [ { quantity: 25, size: "S" }, { quantity: 50, size: "M" } ], + _dummy_field: 0 + }, + { + _id: 2, + sku: "abc123", + description: "socks", + available: [ { quantity: 10, size: "L" } ], + _dummy_field: 0 + }, + { + _id: 3, + sku: "ijk123", + description: "t-shirts", + available: [ { quantity: 30, size: "M" }, { quantity: 5, size: "L" } ], + _dummy_field: 0 + } + ] ) + +The documents in this collection contain a dummy field named +``_dummy_field`` that will be incremented by the +:method:`db.collection.findAndModify()` in the tutorial. If the field +does not exist, the :method:`db.collection.findAndModify()` operation +will add the field to the document. The purpose of the field is to +ensure that the :method:`db.collection.findAndModify()` results in a +modification to the document. + +Procedure +--------- + +.. include:: /includes/steps/findAndModify-quorum-reads.rst + +.. [#edge-cases-2-primaries] + + .. include:: /includes/footnote-two-primaries-edge-cases.rst \ No newline at end of file From 9ead22937f15c897154014b264dfcbe2b83272d3 Mon Sep 17 00:00:00 2001 From: Kyle Suarez Date: Wed, 6 Jan 2016 14:11:07 -0500 Subject: [PATCH 2/3] DOCS-5324 no gle for findAndModify quorum reads Rather than issuing a getLastError, you can use a write concern of ``majority`` to ensure that findAndModify reads only data that cannot be rolled back. --- .../steps-findAndModify-quorum-reads.yaml | 77 +++++-------------- .../perform-findAndModify-quorum-reads.txt | 14 ++-- 2 files changed, 26 insertions(+), 65 deletions(-) diff --git a/source/includes/steps-findAndModify-quorum-reads.yaml b/source/includes/steps-findAndModify-quorum-reads.yaml index 9e1af3d6cfa..56bcd4f95fc 100644 --- a/source/includes/steps-findAndModify-quorum-reads.yaml +++ b/source/includes/steps-findAndModify-quorum-reads.yaml @@ -18,12 +18,17 @@ ref: quorum-read-findAndModify pre: | Use the :method:`db.collection.findAndModify()` method to make a trivial update to the document you want to read and return the - modified document. To specify the document to read, you must use an - exact match query that is supported by a unique index. + modified document. A write concern of :writeconcern:`{ w: "majority" } + <"majority">` is required. To specify the document to read, you must + use an exact match query that is supported by a unique index. The following :method:`~db.collection.findAndModify()` operation specifies an exact match on the uniquely indexed field ``sku`` and increments the field named ``_dummy_field`` in the matching document. + While not necessary, the write concern for this command also includes + a :ref:`wc-wtimeout` value of ``5000`` milliseconds to prevent the + operation from blocking forever if the write cannot propagate to a + majority of voting members. action: language: javascript code: | @@ -31,62 +36,20 @@ action: { query: { sku: "abc123" }, update: { $inc: { _dummy_field: 1 } }, - new: true - } + new: true, + writeConcern: { w: "majority", wtimeout: 5000 } + }, ); ---- -title: Issue ``getLastError`` to determine quorum read. -stepnum: 3 -ref: quorum-read-gle -pre: | - To determine if the read from the - :method:`~db.collection.findAndModify()` was a true quorum read, - issue a :dbcommand:`getLastError` command with :writeconcern:`w: - "majority"`. - - While not necessary to the procedure, the - :dbcommand:`getLastError` command also includes a :ref:`wc-wtimeout` - value of ``5000`` milliseconds to prevent the operation from blocking - forever if the write cannot propagate to a majority of voting members. -action: - language: javascript - code: | - var gle = db.runCommand( { getLastError: 1, w: "majority", wtimeout: 5000 } ); - - if ( (gle.ok != 1) || (gle.err != null) ) { - print("The document returned from findAndModify() may reflect data that is not durable and subject to rollback."); - printjson(gle); - } else { - printjson(updatedDocument); - } post: | - The :dbcommand:`getLastError` determines whether the update from - the :method:`~db.collection.findAndModify()` operation has - propagated to the majority of the replica set's voting members. - - Even in situations where two nodes in the replica set believe that - they are the primary, only one will be able to complete the write - with :writeconcern:`w: "majority"`. As such, the - :dbcommand:`getLastError` with :writeconcern:`w: "majority"` write - concern can confirm whether the client has connected to the true - primary to perform the :method:`~db.collection.findAndModify()` - operation. - - .. note:: - :ref:`wc-wtimeout` causes :dbcommand:`getLastError` to return - with an error after the specified time limit, even if the - required write concern will eventually succeed. As such, if the - :dbcommand:`getLastError` times out, it cannot determine whether - the document returned by :method:`~db.collection.findAndModify()` - is or is not the result of a quorum read. For quorum reads, - ignore the document returned by - :method:`~db.collection.findAndModify()` and repeat the - :method:`~db.collection.findAndModify()` and - :dbcommand:`getLastError`. + Even in situations where two nodes in the replica set believe that + they are the primary, only one will be able to complete the write with + :writeconcern:`w: "majority" <"majority">`. As such, the + :method:`~db.collection.findAndModify()` method with + :writeconcern:`"majority"` write concern will be successful only when + the client has connected to the true primary to perform the operation. - Since the quorum read procedure only increments a dummy field in - the document, you can safely repeat the - :method:`~db.collection.findAndModify()` and - :dbcommand:`getLastError`, adjusting the :ref:`wc-wtimeout` as - necessary. + Since the quorum read procedure only increments a dummy field in the + document, you can safely repeat invocations of + :method:`~db.collection.findAndModify()`, adjusting the + :ref:`wc-wtimeout` as necessary. ... diff --git a/source/tutorial/perform-findAndModify-quorum-reads.txt b/source/tutorial/perform-findAndModify-quorum-reads.txt index f8ea5f02b85..0faa43f1f6f 100644 --- a/source/tutorial/perform-findAndModify-quorum-reads.txt +++ b/source/tutorial/perform-findAndModify-quorum-reads.txt @@ -16,11 +16,9 @@ enough replica set members to avoid rollbacks. This tutorial outlines a procedure that uses :method:`db.collection.findAndModify()` to read data that is not stale and cannot be rolled back. To do so, the procedure uses the -:method:`db.collection.findAndModify()` method to modify a dummy field -in a document and issues a :dbcommand:`getLastError` command to confirm -that the :method:`db.collection.findAndModify()` operation has -propagated to enough members to avoid rollbacks. Specifically, the -procedure requires that: +:method:`db.collection.findAndModify()` method with a :ref:`write +concern ` to modify a dummy field in a document. +Specifically, the procedure requires that: - :method:`db.collection.findAndModify()` use an **exact** match query, and a :doc:`unique index ` **must exist** to @@ -29,8 +27,8 @@ procedure requires that: - :method:`db.collection.findAndModify()` must actually modify a document; i.e. result in a change to the document. -- :dbcommand:`getLastError` must use the write concern - :writeconcern:`w: "majority"`. +- :method:`db.collection.findAndModify()` must use the write concern + :writeconcern:`{ w: "majority" } <"majority">`. Prerequisites ------------- @@ -79,4 +77,4 @@ Procedure .. [#edge-cases-2-primaries] - .. include:: /includes/footnote-two-primaries-edge-cases.rst \ No newline at end of file + .. include:: /includes/footnote-two-primaries-edge-cases.rst From ac1026f4e3e7f2f35ef1d4b61635b3784684b03a Mon Sep 17 00:00:00 2001 From: Kyle Suarez Date: Tue, 12 Jan 2016 11:00:00 -0500 Subject: [PATCH 3/3] DOCS-5324 clarify findAndModify quorum read doc This explicitly details the reasons why one would require a "quorum read" over the existing majority and local read concerns. It also adds a note warning that this procedure is much more expensive than a regular read. This commit also fixes some grammar and explicitly mentions that this technique is new in MongoDB 3.2. --- .../perform-findAndModify-quorum-reads.txt | 28 +++++++++++++------ 1 file changed, 20 insertions(+), 8 deletions(-) diff --git a/source/tutorial/perform-findAndModify-quorum-reads.txt b/source/tutorial/perform-findAndModify-quorum-reads.txt index 0faa43f1f6f..1a104f28e7c 100644 --- a/source/tutorial/perform-findAndModify-quorum-reads.txt +++ b/source/tutorial/perform-findAndModify-quorum-reads.txt @@ -4,19 +4,24 @@ Perform Quorum Reads on Replica Sets .. default-domain:: mongodb +.. versionadded:: 3.2 + Overview -------- -When reading from replica sets, in some edge cases, clients can -read stale data even when specifying a :readmode:`primary` read -preference. [#edge-cases-2-primaries]_ Clients may also see results of -writes before they are made durable and before they have propagated to -enough replica set members to avoid rollbacks. +When reading from the primary of a replica set, it is possible to read +data that is stale or not durable, depending on the read concern used +[#edge-cases-2-primaries]_. With a read concern level of +:readconcern:`"local"`, a client can read data before it is +:term:`durable`; that is, before they have propagated to enough replica +set members to avoid a rollback. A read concern level of +:readconcern:`"majority"` guarantees durable reads but may return stale +data that has been overwritten by another write operation. This tutorial outlines a procedure that uses :method:`db.collection.findAndModify()` to read data that is not stale and cannot be rolled back. To do so, the procedure uses the -:method:`db.collection.findAndModify()` method with a :ref:`write +:method:`~db.collection.findAndModify()` method with a :ref:`write concern ` to modify a dummy field in a document. Specifically, the procedure requires that: @@ -24,12 +29,19 @@ Specifically, the procedure requires that: and a :doc:`unique index ` **must exist** to satisfy the query. -- :method:`db.collection.findAndModify()` must actually modify a +- :method:`~db.collection.findAndModify()` must actually modify a document; i.e. result in a change to the document. -- :method:`db.collection.findAndModify()` must use the write concern +- :method:`~db.collection.findAndModify()` must use the write concern :writeconcern:`{ w: "majority" } <"majority">`. +.. important:: + + The "quorum read" procedure has a substantial cost over simply using + a read concern of :readconcern:`"majority"` because it incurs write + latency rather than read latency. This technique should only be used + if staleness is absolutely intolerable. + Prerequisites -------------