DOCSP-38417 - Aggregation (#69)

Author: mongoKart

snooty.toml

@@ -6,6 +6,8 @@ toc_landing_pages = [
     "/read",
     "/connect",
     "/indexes",
+    "/aggregation",
+    "/aggregation/aggregation-tutorials",
     "/security",
     "/aggregation-tutorials",
 ]

source/aggregation.txt

@@ -1,17 +1,265 @@
-To run an explain plan for this aggregation use
-`PyMongoExplain <https://pypi.org/project/pymongoexplain/>`_,
-a companion library for PyMongo. It allows you to explain any CRUD operation
-by providing a few convenience classes:
+.. _pymongo-aggregation:
 
-.. code-block:: python
+====================================
+Transform Your Data with Aggregation
+====================================
+
+.. facet::
+   :name: genre
+   :values: reference
+ 
+.. meta::
+   :keywords: code example, transform, computed, pipeline
+   :description: Learn how to use {+driver-short+} to perform aggregation operations.
+
+.. contents:: On this page
+   :local:
+   :backlinks: none
+   :depth: 2
+   :class: singlecol
+
+.. toctree::
+   :titlesonly:
+   :maxdepth: 1
+
+   /aggregation/aggregation-tutorials
+
+Overview
+--------
+
+In this guide, you can learn how to use {+driver-short+} to perform
+**aggregation operations**.
+
+Aggregation operations process data in your MongoDB collections and
+return computed results. The MongoDB Aggregation framework, which is
+part of the Query API, is modeled on the concept of data processing
+pipelines. Documents enter a pipeline that contains one or more stages,
+and this pipeline transforms the documents into an aggregated result.
+
+An aggregation operation is similar to a car factory. A car factory has
+an assembly line, which contains assembly stations with specialized
+tools to do specific jobs, like drills and welders. Raw parts enter the
+factory, and then the assembly line transforms and assembles them into a
+finished product.
+
+The **aggregation pipeline** is the assembly line, **aggregation stages** are the
+assembly stations, and **operator expressions** are the
+specialized tools.
+
+Aggregation Versus Find Operations
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+You can use find operations to perform the following actions:
+
+- Select which documents to return
+- Select which fields to return
+- Sort the results
+
+You can use aggregation operations to perform the following actions:
+
+- Perform find operations
+- Rename fields
+- Calculate fields
+- Summarize data
+- Group values
+
+Limitations
+~~~~~~~~~~~
 
-  >>> from pymongoexplain import ExplainableCollection
-  >>> ExplainableCollection(collection).aggregate(pipeline)
-  {'ok': 1.0, 'queryPlanner': [...]}
+Keep the following limitations in mind when using aggregation operations:
 
-Or, use the the ``~pymongo.database.Database.command`` method method:
+- Returned documents must not violate the
+  :manual:`BSON document size limit </reference/limits/#mongodb-limit-BSON-Document-Size>`
+  of 16 megabytes.
+- Pipeline stages have a memory limit of 100 megabytes by default. You can exceed this
+  limit by using the ``allowDiskUse`` keyword argument of the
+  ``aggregate()`` method.
+
+.. important:: $graphLookup exception
+
+   The :manual:`$graphLookup
+   </reference/operator/aggregation/graphLookup/>` stage has a strict
+   memory limit of 100 megabytes and ignores the ``allowDiskUse`` parameter.
+
+Aggregation Example
+-------------------
+
+.. note::
+  
+  This example uses the ``sample_restaurants.restaurants`` collection
+  from the :atlas:`Atlas sample datasets </sample-data>`. To learn how to create a
+  free MongoDB Atlas cluster and load the sample datasets, see :ref:`<pymongo-get-started>`.
+
+To perform an aggregation, pass a list of aggregation stages to the
+``collection.aggregate()`` method.
+
+The following code example produces a count of the number of bakeries in each borough
+of New York. To do so, it uses an aggregation pipeline with the following stages:
+
+- A :manual:`$match </reference/operator/aggregation/match/>` stage to filter for documents
+  whose ``cuisine`` field contains the value ``"Bakery"``.
+
+- A :manual:`$group </reference/operator/aggregation/group/>` stage to group the matching
+  documents by the ``borough`` field, accumulating a count of documents for each distinct
+  value.
 
 .. code-block:: python
+   :copyable: true
+
+   # Define an aggregation pipeline with a match stage and a group stage
+   pipeline = [
+       { "$match": { "cuisine": "Bakery" } },
+       { "$group": { "_id": "$borough", "count": { "$sum": 1 } } }
+   ]
+
+   # Execute the aggregation
+   aggCursor = collection.aggregate(pipeline)
+
+   # Print the aggregated results
+   for document in aggCursor:
+       print(document)
+
+The preceding code example produces output similar to the following:
+
+.. code-block:: javascript
+   
+   {'_id': 'Bronx', 'count': 71}
+   {'_id': 'Brooklyn', 'count': 173}
+   {'_id': 'Missing', 'count': 2}
+   {'_id': 'Manhattan', 'count': 221}
+   {'_id': 'Queens', 'count': 204}
+   {'_id': 'Staten Island', 'count': 20}
+
+Explain an Aggregation
+~~~~~~~~~~~~~~~~~~~~~~
+
+To view information about how MongoDB executes your operation, you can
+instruct MongoDB to **explain** it. When MongoDB explains an operation, it returns
+**execution plans** and performance statistics. An execution
+plan is a potential way MongoDB can complete an operation.
+When you instruct MongoDB to explain an operation, it returns both the
+plan MongoDB executed and any rejected execution plans.
+
+To explain an aggregation operation, you can use either the
+`PyMongoExplain <https://pypi.org/project/pymongoexplain/>`__ library or a database
+command. Select the corresponding tab below to see an example of each method.
+
+.. tabs::
+
+   .. tab:: PyMongoExplain
+      :tabid: pymongoexplain
+
+      Use pip to install the ``pymongoexplain`` library, as shown in the
+      following example:
+
+      .. code-block:: sh
+
+         python3 -m pip install pymongoexplain
+
+      The following code example runs the preceding aggregation example and prints the explanation
+      returned by MongoDB: 
+     
+      .. io-code-block::
+         :copyable: true
+     
+        .. input::
+           :language: python
+        
+           # Define an aggregation pipeline with a match stage and a group stage
+           pipeline = [
+              { "$match": { "cuisine": "Bakery" } },
+              { "$group": { "_id": "$borough", "count": { "$sum": 1 } } }
+           ]
+
+           # Execute the operation and print the explanation
+           result = ExplainableCollection(collection).aggregate(pipeline)
+           print(result)
+     
+        .. output::
+           :language: javascript
+           :visible: false
+
+           ... 
+           'winningPlan': {'queryPlan': {'stage': 'GROUP',
+                                                 'planNodeId': 3,
+                                                 'inputStage': {'stage': 'COLLSCAN',
+                                                                'planNodeId': 1,
+                                                                'filter': {'cuisine': {'$eq': 'Bakery'}},
+                                                                'direction': 'forward'}},
+                                                               ... 
+
+   .. tab:: Database Command
+      :tabid: db-command
+
+      The following code example runs the preceding aggregation example and prints the explanation
+      returned by MongoDB: 
+
+      .. io-code-block::
+         :copyable: true
+
+         .. input::
+            :language: python
+
+            # Define an aggregation pipeline with a match stage and a group stage
+            pipeline = [
+               { $match: { cuisine: "Bakery" } },
+               { $group: { _id: "$borough", count: { $sum: 1 } } }
+            ]
+
+            # Execute the operation and print the explanation
+            result = database.command("aggregate", "collection", pipeline=pipeline, explain=True)
+            print(result)
+
+         .. output::
+            :language: javascript
+
+            ...
+            'command': {'aggregate': 'collection',
+              'pipeline': [{'$match': {'cuisine': 'Bakery'}},
+                           {'$group': {'_id': '$borough',
+                                       'count': {'$sum': 1}}}],
+              'explain': True,
+            ...
+
+.. tip::
+
+   You can use Python's ``pprint`` module to make explanation results easier to read:
+
+   .. code-block:: python
+
+      import pprint
+      ...
+      pprint.pp(result)
+
+Additional Information
+----------------------
+
+MongoDB Server Manual
+~~~~~~~~~~~~~~~~~~~~~
+
+To view a full list of expression operators, see :manual:`Aggregation
+Operators. </reference/operator/aggregation/>`
+
+To learn about assembling an aggregation pipeline and view examples, see
+:manual:`Aggregation Pipeline. </core/aggregation-pipeline/>`
+
+To learn more about creating pipeline stages, see :manual:`Aggregation
+Stages. </reference/operator/aggregation-pipeline/>`
+
+To learn more about explaining MongoDB operations, see
+:manual:`Explain Output </reference/explain-results/>` and
+:manual:`Query Plans. </core/query-plans/>`
+
+Aggregation Tutorials
+~~~~~~~~~~~~~~~~~~~~~
+
+To view step-by-step explanations of common aggregation tasks, see
+:ref:`pymongo-aggregation-tutorials-landing`.
+
+API Documentation
+~~~~~~~~~~~~~~~~~
+
+For more information about executing aggregation operations with {+driver-short+},
+see the following API documentation:
 
-  >>> db.command('aggregate', 'things', pipeline=pipeline, explain=True)
-  {'ok': 1.0, 'stages': [...]}
+- `aggregate() <{+api-root+}pymongo/collection.html#pymongo.collection.Collection.aggregate>`__

source/aggregation-tutorials.txt renamed to source/aggregation/aggregation-tutorials.txt

@@ -19,11 +19,11 @@ Aggregation Tutorials
 
 .. toctree::
 
-   /aggregation-tutorials/filtered-subset/
-   /aggregation-tutorials/group-total/
-   /aggregation-tutorials/unpack-arrays/
-   /aggregation-tutorials/one-to-one-join/
-   /aggregation-tutorials/multi-field-join/
+   /aggregation/aggregation-tutorials/filtered-subset/
+   /aggregation/aggregation-tutorials/group-total/
+   /aggregation/aggregation-tutorials/unpack-arrays/
+   /aggregation/aggregation-tutorials/one-to-one-join/
+   /aggregation/aggregation-tutorials/multi-field-join/
 
 Overview
 --------

source/aggregation-tutorials/filtered-subset.txt renamed to source/aggregation/aggregation-tutorials/filtered-subset.txt

@@ -18,7 +18,7 @@ MongoDB PyMongo Documentation
    /write-operations
    /read
    /indexes
-   /aggregation-tutorials
+   /aggregation
    /security
    /tools
    /faq