diff --git a/source/images/compass-analyze-inventory-docs.png b/source/images/compass-analyze-inventory-docs.png new file mode 100644 index 00000000000..d7f7e554610 Binary files /dev/null and b/source/images/compass-analyze-inventory-docs.png differ diff --git a/source/images/compass-create-inventory-index.png b/source/images/compass-create-inventory-index.png new file mode 100644 index 00000000000..297ca46e8d3 Binary files /dev/null and b/source/images/compass-create-inventory-index.png differ diff --git a/source/images/compass-create-new-index.png b/source/images/compass-create-new-index.png new file mode 100644 index 00000000000..cb7f737cf0f Binary files /dev/null and b/source/images/compass-create-new-index.png differ diff --git a/source/images/compass-explain-plan-no-index-raw-json.png b/source/images/compass-explain-plan-no-index-raw-json.png new file mode 100644 index 00000000000..244a4e9437b Binary files /dev/null and b/source/images/compass-explain-plan-no-index-raw-json.png differ diff --git a/source/images/compass-explain-plan-no-index.png b/source/images/compass-explain-plan-no-index.png new file mode 100644 index 00000000000..7cbe19e678a Binary files /dev/null and b/source/images/compass-explain-plan-no-index.png differ diff --git a/source/images/compass-explain-plan-with-index-raw-json.png b/source/images/compass-explain-plan-with-index-raw-json.png new file mode 100644 index 00000000000..afef757a9f3 Binary files /dev/null and b/source/images/compass-explain-plan-with-index-raw-json.png differ diff --git a/source/images/compass-explain-plan-with-index.png b/source/images/compass-explain-plan-with-index.png new file mode 100644 index 00000000000..f7bed9c9ca4 Binary files /dev/null and b/source/images/compass-explain-plan-with-index.png differ diff --git a/source/images/compass-find-no-index-results.png b/source/images/compass-find-no-index-results.png new file mode 100644 index 00000000000..dccf3e05286 Binary files /dev/null and b/source/images/compass-find-no-index-results.png differ diff --git a/source/images/compass-show-new-index.png b/source/images/compass-show-new-index.png new file mode 100644 index 00000000000..67dac36e9cb Binary files /dev/null and b/source/images/compass-show-new-index.png differ diff --git a/source/tutorial/analyze-query-plan.txt b/source/tutorial/analyze-query-plan.txt index 30e8684113b..f22c182a3a9 100644 --- a/source/tutorial/analyze-query-plan.txt +++ b/source/tutorial/analyze-query-plan.txt @@ -10,13 +10,27 @@ Analyze Query Performance :depth: 1 :class: singlecol -The :method:`cursor.explain("executionStats") ` and -the :method:`db.collection.explain("executionStats") -` methods provide statistics about the -performance of a query. This data output can be useful in measuring if -and how a query uses an index. - -.. include:: /includes/fact-explain-collection-method.rst +.. tabs-drivers:: + + tabs: + - id: shell + content: | + The + :method:`cursor.explain("executionStats") ` + and the :method:`db.collection.explain("executionStats") + ` methods provide statistics about + the performance of a query. These statistics can be useful in + measuring if and how a query uses an index. + + .. include:: /includes/fact-explain-collection-method.rst + + - id: compass + content: | + |compass| provides an + `Explain Plan `_ + tab, which displays statistics about the performance of a + query. These statistics can be useful in measuring if and how + a query uses an index. Evaluate the Performance of a Query ----------------------------------- @@ -36,75 +50,182 @@ Consider a collection ``inventory`` with the following documents: { "_id" : 9, "item" : "t2", type: "toys", quantity: 50 } { "_id" : 10, "item" : "f4", type: "food", quantity: 75 } +.. tabs-drivers:: + + tabs: + - id: compass + content: | + The documents appear in |compass| as the following: + + .. figure:: /images/compass-analyze-inventory-docs.png + :alt: Compass Inventory collection documents + .. _analyze-no-index: Query with No Index ~~~~~~~~~~~~~~~~~~~ -The following query retrieves documents where the ``quantity`` field -has a value between ``100`` and ``200``, inclusive: +.. tabs-drivers:: -.. code-block:: javascript + tabs: + - id: shell + content: | + The following query retrieves documents where the + ``quantity`` field has a value between ``100`` and ``200``, + inclusive: - db.inventory.find( { quantity: { $gte: 100, $lte: 200 } } ) + .. cssclass:: copyable-code + .. code-block:: javascript -The query returns the following documents: + db.inventory.find( { quantity: { $gte: 100, $lte: 200 } } ) -.. code-block:: javascript + The query returns the following documents: - { "_id" : 2, "item" : "f2", "type" : "food", "quantity" : 100 } - { "_id" : 3, "item" : "p1", "type" : "paper", "quantity" : 200 } - { "_id" : 4, "item" : "p2", "type" : "paper", "quantity" : 150 } + .. code-block:: javascript -To view the query plan selected, use the -:method:`~cursor.explain("executionStats")` method: + { "_id" : 2, "item" : "f2", "type" : "food", "quantity" : 100 } + { "_id" : 3, "item" : "p1", "type" : "paper", "quantity" : 200 } + { "_id" : 4, "item" : "p2", "type" : "paper", "quantity" : 150 } -.. code-block:: javascript + To view the query plan selected, chain the + :method:`cursor.explain("executionStats") ` + cursor method to the end of the :command:`find` command: - db.inventory.find( - { quantity: { $gte: 100, $lte: 200 } } - ).explain("executionStats") + .. cssclass:: copyable-code + .. code-block:: javascript -:method:`~cursor.explain()` returns the following results: + db.inventory.find( + { quantity: { $gte: 100, $lte: 200 } } + ).explain("executionStats") -.. code-block:: javascript + :method:`~cursor.explain()` returns the following results: + + .. code-block:: javascript - { - "queryPlanner" : { - "plannerVersion" : 1, - ... - "winningPlan" : { - "stage" : "COLLSCAN", + { + "queryPlanner" : { + "plannerVersion" : 1, + ... + "winningPlan" : { + "stage" : "COLLSCAN", + ... + } + }, + "executionStats" : { + "executionSuccess" : true, + "nReturned" : 3, + "executionTimeMillis" : 0, + "totalKeysExamined" : 0, + "totalDocsExamined" : 10, + "executionStages" : { + "stage" : "COLLSCAN", + ... + }, + ... + }, ... } - }, - "executionStats" : { - "executionSuccess" : true, - "nReturned" : 3, - "executionTimeMillis" : 0, - "totalKeysExamined" : 0, - "totalDocsExamined" : 10, - "executionStages" : { - "stage" : "COLLSCAN", - ... - }, - ... - }, - ... - } - -- :data:`queryPlanner.winningPlan.stage - ` displays ``COLLSCAN`` to - indicate a collection scan. - -- :data:`executionStats.nReturned ` - displays ``3`` to indicate that the query matches and returns three - documents. - -- :data:`executionStats.totalDocsExamined - ` display ``10`` to - indicate that MongoDB had to scan ten documents (i.e. all documents - in the collection) to find the three matching documents. + + - :data:`queryPlanner.winningPlan.stage + ` displays + ``COLLSCAN`` to indicate a collection scan. + + Collection scans indicate that the + :binary:`~bin.mongod` had to scan the entire collection + document by document to identify the results. This is a + generally expensive operation and can result in slow + queries. + + - :data:`executionStats.nReturned + ` displays ``3`` to + indicate that the query matches and returns three documents. + + - :data:`executionStats.totalKeysExamined + ` displays ``0`` + to indicate that this is query is not using an index. + + - :data:`executionStats.totalDocsExamined + ` displays ``10`` + to indicate that MongoDB had to scan ten documents (i.e. + all documents in the collection) to find the three matching + documents. + + - id: compass + content: | + The following query retrieves documents where the + ``quantity`` field has a value between ``100`` and ``200``, + inclusive: + + Copy the following filter into the Compass query bar and click + :guilabel:`Find`: + + .. class:: copyable-code + .. code-block:: javascript + + { quantity: { $gte: 100, $lte: 200 } } + + The query returns the following documents: + + .. figure:: /images/compass-find-no-index-results.png + :alt: Compass no index query results + + To view the query plan selected: + + 1. Click the :guilabel:`Explain Plan` tab for the + ``test.inventory`` collection. + + #. Click :guilabel:`Explain`. + + |compass| displays the query plan as follows: + + .. figure:: /images/compass-explain-plan-no-index.png + :alt: Compass no index query plan + + .. note:: + + Because we are working with such a small dataset for the + purposes of this tutorial, the + :guilabel:`Actual Query Execution Time` displays + ``0`` seconds, even though we are not using an index. + + In a larger dataset, the difference in query + execution time between an indexed query versus a + non-indexed query would be much more substantial. + + Visual Tree + ``````````` + + - The :guilabel:`Query Performance Summary` shows the + execution stats of the query: + + - :guilabel:`Documents Returned` displays ``3`` to indicate + that the query matches and returns three documents. + + - :guilabel:`Index Keys Examined` displays ``0`` to + indicate that this query is not using an index. + + - :guilabel:`Documents Examined` displays ``10`` to indicate + that MongoDB had to scan ten documents (i.e. all documents + in the collection) to find the three matching documents. + + - Below the :guilabel:`Query Performance Summary`, |compass| + displays the ``COLLSCAN`` query stage to indicate that a + collection scan was used for this query. + + Collection scans indicate that the + :binary:`~bin.mongod` had to scan the entire collection + document by document to identify the results. This is a + generally expensive operation and can result in slow + queries. + + Raw JSON + ```````` + + The explain details can also be viewed in raw JSON format by + clicking :guilabel:`Raw JSON` below the query bar: + + .. figure:: /images/compass-explain-plan-no-index-raw-json.png + :alt: Compass no index query plan raw JSON The difference between the number of matching documents and the number of examined documents may suggest that, to improve efficiency, the @@ -118,207 +239,335 @@ Query with Index To support the query on the ``quantity`` field, add an index on the ``quantity`` field: -.. code-block:: javascript +.. tabs-drivers:: - db.inventory.createIndex( { quantity: 1 } ) + tabs: + - id: shell + content: | + .. cssclass:: copyable-code + .. code-block:: javascript -To view the query plan statistics, use the -:method:`~cursor.explain("executionStats")` method: + db.inventory.createIndex( { quantity: 1 } ) -.. code-block:: javascript + To view the query plan statistics, use the + :method:`~cursor.explain("executionStats")` method: - db.inventory.find( - { quantity: { $gte: 100, $lte: 200 } } - ).explain("executionStats") + .. code-block:: javascript -The :method:`~cursor.explain()` method returns the following results: + db.inventory.find( + { quantity: { $gte: 100, $lte: 200 } } + ).explain("executionStats") -.. code-block:: javascript + The :method:`~cursor.explain()` method returns the following + results: - { - "queryPlanner" : { - "plannerVersion" : 1, - ... - "winningPlan" : { - "stage" : "FETCH", - "inputStage" : { - "stage" : "IXSCAN", - "keyPattern" : { - "quantity" : 1 + .. code-block:: javascript + + { + "queryPlanner" : { + "plannerVersion" : 1, + ... + "winningPlan" : { + "stage" : "FETCH", + "inputStage" : { + "stage" : "IXSCAN", + "keyPattern" : { + "quantity" : 1 + }, + ... + } + }, + "rejectedPlans" : [ ] + }, + "executionStats" : { + "executionSuccess" : true, + "nReturned" : 3, + "executionTimeMillis" : 0, + "totalKeysExamined" : 3, + "totalDocsExamined" : 3, + "executionStages" : { + ... }, ... - } - }, - "rejectedPlans" : [ ] - }, - "executionStats" : { - "executionSuccess" : true, - "nReturned" : 3, - "executionTimeMillis" : 0, - "totalKeysExamined" : 3, - "totalDocsExamined" : 3, - "executionStages" : { + }, ... - }, - ... - }, - ... - } + } -- :data:`queryPlanner.winningPlan.inputStage.stage - ` displays ``IXSCAN`` to indicate - index use. + - :data:`queryPlanner.winningPlan.inputStage.stage + ` displays + ``IXSCAN`` to indicate index use. -- :data:`executionStats.nReturned ` - displays ``3`` to indicate that the query matches and returns three - documents. + - :data:`executionStats.nReturned ` + displays ``3`` to indicate that the query matches and + returns three documents. -- :data:`executionStats.totalKeysExamined - ` display ``3`` to indicate - that MongoDB scanned three index entries. + - :data:`executionStats.totalKeysExamined + ` displays ``3`` + to indicate that MongoDB scanned three index entries. The + number of keys examined match the number of documents + returned, meaning that the :binary:`~bin.mongod` only had + to examine index keys to return the results. The + :binary:`~bin.mongod` did not have to scan all of the + documents, and only the three matching documents had to be + pulled into memory. This results in a very efficient query. -- :data:`executionStats.totalDocsExamined - ` display ``3`` to indicate - that MongoDB scanned three documents. + - :data:`executionStats.totalDocsExamined + ` display ``3`` + to indicate that MongoDB scanned three documents. -When run with an index, the query scanned ``3`` index entries and ``3`` -documents to return ``3`` matching documents. Without the index, to -return the ``3`` matching documents, the query had to scan the whole -collection, scanning ``10`` documents. + - id: compass + content: | + 1. Click the :guilabel:`Indexes` tab for the + ``test.inventory`` collection. -.. _analyze-compare-performance: + 2. Click :guilabel:`Create Index`. -Compare Performance of Indexes ------------------------------- + 3. Select ``quantity`` from the + :guilabel:`Select a field name` dropdown. -To manually compare the performance of a query using more than one -index, you can use the :method:`~cursor.hint()` method in conjunction -with the :method:`~cursor.explain()` method. + 4. Select ``1 (asc)`` from the type dropdown. -Consider the following query: + 5. Click :guilabel:`Create`. -.. code-block:: javascript + .. figure:: /images/compass-create-inventory-index.png + :scale: 90 % + :alt: Create inventory index in Compass - db.inventory.find( { quantity: { $gte: 100, $lte: 300 }, type: "food" } ) + .. note:: -The query returns the following documents: + Leaving the index name field blank causes |compass| to + create a default name for the index. -.. code-block:: javascript + You can now see your newly created index in the + :guilabel:`Indexes` tab: - { "_id" : 2, "item" : "f2", "type" : "food", "quantity" : 100 } - { "_id" : 5, "item" : "f3", "type" : "food", "quantity" : 300 } + .. figure:: /images/compass-show-new-index.png + :alt: Compass show new index -To support the query, add a :doc:`compound index -`. With :doc:`compound indexes -`, the order of the fields matter. + Return to the :guilabel:`Explain Plan` tab for the + ``inventory`` collection and re-run the query from + the previous step: -For example, add the following two compound indexes. The first index -orders by ``quantity`` field first, and then the ``type`` field. The -second index orders by ``type`` first, and then the ``quantity`` field. + .. class:: copyable-code + .. code-block:: javascript -.. code-block:: javascript + { quantity: { $gte: 100, $lte: 200 } } - db.inventory.createIndex( { quantity: 1, type: 1 } ) - db.inventory.createIndex( { type: 1, quantity: 1 } ) + |compass| displays the query plan as follows: -Evaluate the effect of the first index on the query: + .. figure:: /images/compass-explain-plan-with-index.png + :alt: Compass explain plan with index -.. code-block:: javascript + Visual Tree + ``````````` - db.inventory.find( - { quantity: { $gte: 100, $lte: 300 }, type: "food" } - ).hint({ quantity: 1, type: 1 }).explain("executionStats") + - The :guilabel:`Query Performance Summary` shows the + execution stats of the query: -The :method:`~cursor.explain()` method returns the following output: + - :guilabel:`Documents Returned` displays ``3`` to indicate + that the query matches and returns three documents. -.. code-block:: javascript + - :guilabel:`Index Keys Examined` displays ``3`` + to indicate that MongoDB scanned three index entries. The + number of keys examined match the number of documents + returned, meaning that the :binary:`~bin.mongod` only had + to examine index keys to return the results. The + :binary:`~bin.mongod` did not have to scan all of the + documents, and only the three matching documents had to be + pulled into memory. This results in a very efficient + query. + + - :guilabel:`Documents Examined` displays ``3`` to indicate + that MongoDB scanned three documents. + + - On the right-hand side of the + :guilabel:`Query Performance Summary`, |compass| shows + that the query used the ``quantity`` index. + + - Below the :guilabel:`Query Performance Summary`, |compass| + displays the query stages ``FETCH`` and ``IXSCAN``. + ``IXSCAN`` indicates that the + :binary:`~bin.mongod` used an index to satisfy the query + before exeuting the ``FETCH`` stage and retrieving the + documents. + + Raw JSON + ```````` + + The explain details can also be viewed in raw JSON format by + clicking :guilabel:`Raw JSON` below the query bar: + + .. figure:: /images/compass-explain-plan-with-index-raw-json.png + :alt: Compass query plan with index raw JSON + +Without the index, the query would scan the whole collection of ``10`` +documents to return ``3`` matching documents. The query also had to +scan the entirety of each document, potentially pulling them into +memory. This results in an expensive and potentially slow query +operation. + +When run with an index, the query scanned ``3`` index entries +and ``3`` documents to return ``3`` matching documents, resulting +in a very efficient query. + +.. tabs-drivers:: + + tabs: + - id: shell + content: | + .. _analyze-compare-performance: + + Compare Performance of Indexes + ------------------------------ + + To manually compare the performance of a query using more + than one index, you can use the :method:`~cursor.hint()` + method in conjunction with the :method:`~cursor.explain()` + method. + + Consider the following query: - { - "queryPlanner" : { - ... - "winningPlan" : { - "stage" : "FETCH", - "inputStage" : { - "stage" : "IXSCAN", - "keyPattern" : { - "quantity" : 1, - "type" : 1 + .. cssclass:: copyable-code + .. code-block:: javascript + + db.inventory.find( { + quantity: { + $gte: 100, $lte: 300 }, - ... - } - } - }, - "rejectedPlans" : [ ] - }, - "executionStats" : { - "executionSuccess" : true, - "nReturned" : 2, - "executionTimeMillis" : 0, - "totalKeysExamined" : 5, - "totalDocsExamined" : 2, - "executionStages" : { - ... - } - }, - ... - } - -MongoDB scanned ``5`` index keys -(:data:`executionStats.totalKeysExamined -`) to return ``2`` matching -documents (:data:`executionStats.nReturned -`). - -Evaluate the effect of the second index on the query: + type: "food" + } ) -.. code-block:: javascript + The query returns the following documents: - db.inventory.find( - { quantity: { $gte: 100, $lte: 300 }, type: "food" } - ).hint({ type: 1, quantity: 1 }).explain("executionStats") + .. code-block:: javascript -The :method:`~cursor.explain()` method returns the following output: + { "_id" : 2, "item" : "f2", "type" : "food", "quantity" : 100 } + { "_id" : 5, "item" : "f3", "type" : "food", "quantity" : 300 } -.. code-block:: javascript + To support the query, add a :doc:`compound index + `. With :doc:`compound indexes + `, the order of the fields matter. + + For example, add the following two compound indexes. The + first index orders by ``quantity`` field first, and then the + ``type`` field. The second index orders by ``type`` first, + and then the ``quantity`` field. + + .. cssclass:: copyable-code + .. code-block:: javascript + + db.inventory.createIndex( { quantity: 1, type: 1 } ) + db.inventory.createIndex( { type: 1, quantity: 1 } ) + + Evaluate the effect of the first index on the query: + + .. cssclass:: copyable-code + .. code-block:: javascript + + db.inventory.find( + { quantity: { $gte: 100, $lte: 300 }, type: "food" } + ).hint({ quantity: 1, type: 1 }).explain("executionStats") + + The :method:`~cursor.explain()` method returns the following + output: - { - "queryPlanner" : { - ... - "winningPlan" : { - "stage" : "FETCH", - "inputStage" : { - "stage" : "IXSCAN", - "keyPattern" : { - "type" : 1, - "quantity" : 1 + .. code-block:: javascript + + { + "queryPlanner" : { + ... + "winningPlan" : { + "stage" : "FETCH", + "inputStage" : { + "stage" : "IXSCAN", + "keyPattern" : { + "quantity" : 1, + "type" : 1 + }, + ... + } + } + }, + "rejectedPlans" : [ ] + }, + "executionStats" : { + "executionSuccess" : true, + "nReturned" : 2, + "executionTimeMillis" : 0, + "totalKeysExamined" : 5, + "totalDocsExamined" : 2, + "executionStages" : { + ... + } }, ... } - }, - "rejectedPlans" : [ ] - }, - "executionStats" : { - "executionSuccess" : true, - "nReturned" : 2, - "executionTimeMillis" : 0, - "totalKeysExamined" : 2, - "totalDocsExamined" : 2, - "executionStages" : { - ... - } - }, - ... - } - -MongoDB scanned ``2`` index keys -(:data:`executionStats.totalKeysExamined -`) to return ``2`` matching -documents (:data:`executionStats.nReturned -`). - -For this example query, the compound index ``{ type: 1, quantity: 1 }`` -is more efficient than the compound index ``{ quantity: 1, type: 1 }``. - -.. seealso:: :doc:`/core/query-optimization`, :doc:`/core/query-plans`, - :doc:`/tutorial/optimize-query-performance-with-indexes-and-projections`, - :doc:`/applications/indexes` + + MongoDB scanned ``5`` index keys + (:data:`executionStats.totalKeysExamined + `) to return ``2`` + matching documents (:data:`executionStats.nReturned + `). + + Evaluate the effect of the second index on the query: + + .. cssclass:: copyable-code + .. code-block:: javascript + + db.inventory.find( + { quantity: { $gte: 100, $lte: 300 }, type: "food" } + ).hint({ type: 1, quantity: 1 }).explain("executionStats") + + The :method:`~cursor.explain()` method returns the following + output: + + .. code-block:: javascript + + { + "queryPlanner" : { + ... + "winningPlan" : { + "stage" : "FETCH", + "inputStage" : { + "stage" : "IXSCAN", + "keyPattern" : { + "type" : 1, + "quantity" : 1 + }, + ... + } + }, + "rejectedPlans" : [ ] + }, + "executionStats" : { + "executionSuccess" : true, + "nReturned" : 2, + "executionTimeMillis" : 0, + "totalKeysExamined" : 2, + "totalDocsExamined" : 2, + "executionStages" : { + ... + } + }, + ... + } + + MongoDB scanned ``2`` index keys + (:data:`executionStats.totalKeysExamined + `) to return ``2`` + matching documents (:data:`executionStats.nReturned + `). + + For this example query, the compound index ``{ type: 1, quantity: 1 }`` + is more efficient than the compound index ``{ quantity: 1, type: 1 }``. + + .. seealso:: :doc:`/core/query-optimization`, :doc:`/core/query-plans`, + :doc:`/tutorial/optimize-query-performance-with-indexes-and-projections`, + :doc:`/applications/indexes` + + - id: compass + content: | + .. seealso:: + - :doc:`Query Optimization ` + - :ref:`MongoDB Compass Documentation ` + - `Compass Query Plan Documentation `_