DOCSP-30436: Added ignoreNullValues (#162)

Author: DBirtolo-mdb

source/configuration/write.txt

@@ -31,40 +31,57 @@ The following options for writing to MongoDB are available:
    * - Property name
      - Description
 
-   * - ``mongoClientFactory``
-     - | MongoClientFactory configuration key.
-       | You can specify a custom implementation which must implement the
-         ``com.mongodb.spark.sql.connector.connection.MongoClientFactory``
-         interface.
-       |
-       | **Default:** ``com.mongodb.spark.sql.connector.connection.DefaultMongoClientFactory``
+   * - ``collection``
+     - | **Required.**
+       | The collection name configuration.
 
    * - ``connection.uri``
      - | **Required.**
        | The connection string configuration key.
        |
        | **Default:** ``mongodb://localhost:27017/``
 
+   * - ``convertJson``
+     - | When ``true``, the connector parses the string and converts extended JSON
+         into BSON.
+       |
+       | **Default:** ``false``
+
    * - ``database``
      - | **Required.**
        | The database name configuration.
 
-   * - ``collection``
-     - | **Required.**
-       | The collection name configuration.
+   * - ``idFieldList``
+     - | Field or list of fields by which to split the collection data. To
+         specify more than one field, separate them using a comma as shown
+         in the following example:
 
+       .. code-block:: none
+          :copyable: false
+
+          "fieldName1,fieldName2"
+
+       | **Default:** ``_id``
+
+   * - ``ignoreNullValues``
+     - | When ``true``, the connector ignores any ``null`` values when writing,
+         including ``null`` values in arrays and nested documents.
+       |
+       | **Default:** ``false``
 
    * - ``maxBatchSize``
      - | Specifies the maximum number of operations to batch in bulk
          operations.
-
        |
        | **Default:** ``512``
 
-   * - ``ordered``
-     - | Specifies whether to perform ordered bulk operations.
+   * - ``mongoClientFactory``
+     - | MongoClientFactory configuration key.
+       | You can specify a custom implementation which must implement the
+         ``com.mongodb.spark.sql.connector.connection.MongoClientFactory``
+         interface.
        |
-       | **Default:** ``true``
+       | **Default:** ``com.mongodb.spark.sql.connector.connection.DefaultMongoClientFactory``
 
    * - ``operationType``
      - | Specifies the type of write operation to perform. You can set
@@ -83,26 +100,19 @@ The following options for writing to MongoDB are available:
        |
        | **Default:** ``replace``
 
-   * - ``idFieldList``
-     - | Field or list of fields by which to split the collection data. To
-         specify more than one field, separate them using a comma as shown
-         in the following example:
-
-       .. code-block:: none
-          :copyable: false
-
-          "fieldName1,fieldName2"
-
-       | **Default:** ``_id``
+   * - ``ordered``
+     - | Specifies whether to perform ordered bulk operations.
+       |
+       | **Default:** ``true``
 
-   * - ``writeConcern.w``
-     - | Specifies ``w``, a write-concern option to request acknowledgment
-         that the write operation has propogated to a specified number of
-         MongoDB instances. For a list
-         of allowed values for this option, see :manual:`WriteConcern
-         </reference/write-concern/#w-option>` in the MongoDB manual.
+   * - ``upsertDocument``
+     - | When ``true``, replace and update operations will insert the data
+         if no match exists.
        |
-       | **Default:** ``1``
+       | For time series collections, you must set ``upsertDocument`` to
+         ``false``.
+       |
+       | **Default:** ``true``
 
    * - ``writeConcern.journal``
      - | Specifies ``j``, a write-concern option to enable request for
@@ -114,6 +124,15 @@ The following options for writing to MongoDB are available:
          guide on the
          :manual:`WriteConcern j option </reference/write-concern/#j-option>`.
 
+   * - ``writeConcern.w``
+     - | Specifies ``w``, a write-concern option to request acknowledgment
+         that the write operation has propogated to a specified number of
+         MongoDB nodes. For a list
+         of allowed values for this option, see :manual:`WriteConcern
+         </reference/write-concern/#w-option>` in the MongoDB manual.
+       |
+       | **Default:** ``1``
+
    * - ``writeConcern.wTimeoutMS``
      - | Specifies ``wTimeoutMS``, a write-concern option to return an error
          when a write operation exceeds the number of milliseconds. If you
@@ -123,21 +142,6 @@ The following options for writing to MongoDB are available:
          guide on the
          :manual:`WriteConcern wtimeout option </reference/write-concern/#wtimeout>`.
 
-   * - ``upsertDocument``
-     - | When ``true``, replace and update operations will insert the data
-         if no match exists.
-       |
-       | For time series collections, you must set ``upsertDocument`` to
-         ``false``.
-       |
-       | **Default:** ``true``
-
-   * - ``convertJson``
-     - | When ``true``, the connector parses the string and converts extended JSON
-         into BSON.
-       |
-       | **Default:** ``false``
-
 .. _configure-output-uri:
 
 ``connection.uri`` Configuration Setting

source/release-notes.txt

@@ -2,6 +2,13 @@
 Release Notes
 =============
 
+MongoDB Connector for Spark 10.2.0
+----------------------------------
+
+- Added the ``ignoreNullValues`` write configuration property, which enables you
+  to control whether the connector ignores null values. In previous versions, 
+  the connector always wrote ``null`` values to MongoDB.
+
 MongoDB Connector for Spark 10.1.1
 ----------------------------------
 

source/write-to-mongodb.txt

@@ -43,4 +43,7 @@ Write to MongoDB
 .. important::
 
    If your write operation includes a field with a ``null`` value,
-   the connector writes the field name and ``null`` value to MongoDB. 
+   the connector writes the field name and ``null`` value to MongoDB. You can
+   change this behavior by setting the write configuration property
+   ``ignoreNullValues``. For more information about setting the connector's
+   write behavior, see :ref:`Write Configuration Options <spark-write-conf>`.