Skip to content

Commit 1214cea

Browse files
Chris Choschmalliso
authored andcommitted
DOCSP-10983: clarify RenameByMapping example (#43)
* DOCSP-10983: fix RenameByMapping property example
1 parent 9662680 commit 1214cea

File tree

1 file changed

+56
-32
lines changed

1 file changed

+56
-32
lines changed

source/kafka-sink-postprocessors.txt

Lines changed: 56 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -23,12 +23,10 @@ which each post processor is executed in the order provided on
2323
the ``SinkDocument``, and the result is stored in a MongoDB collection.
2424

2525
Post processors perform data modification tasks such as setting
26-
the document ``_id`` field, key or value field projection, renaming
27-
fields, and redacting sensitive information. You can use the
28-
following pre-built post processors or implement your own by extending
29-
the `PostProcessor
30-
<https://github.com/mongodb/mongo-kafka/blob/master/src/main/java/com/mongodb/kafka/connect/sink/processor/PostProcessor.java>`_
31-
class:
26+
the document ``_id`` field, message key or value projection, renaming
27+
fields, and redacting sensitive information. You can implement your own
28+
post processor by extending the `PostProcessor <https://github.com/mongodb/mongo-kafka/blob/master/src/main/java/com/mongodb/kafka/connect/sink/processor/PostProcessor.java>`_
29+
class or use one of the following pre-built ones:
3230

3331
.. list-table::
3432
:header-rows: 1
@@ -74,7 +72,7 @@ class:
7472

7573
* - RenameByMapping
7674
- | Full Path: ``com.mongodb.kafka.connect.sink.processor.field.renaming.RenameByMapping``
77-
| Renames fields that are an exact match to a specified key or value field.
75+
| Renames fields that are an exact match to a specified field name in the key or value document.
7876

7977
.. seealso:: :ref:`Renaming configuration <config-field-renaming>` and :ref:`Example <field-renaming-mapping-example>`.
8078

@@ -84,7 +82,7 @@ class:
8482

8583
.. seealso:: :ref:`Renaming configuration <config-field-renaming>` and :ref:`Example <field-renaming-regex-example>`.
8684

87-
85+
8886
You can configure the post processor chain by specifying an ordered,
8987
comma separated list of fully-qualified ``PostProcessor`` class names:
9088

@@ -155,7 +153,7 @@ provided with this connector:
155153
- | Full Path: ``com.mongodb.kafka.connect.sink.processor.id.strategy.PartialValueStrategy``
156154
| Uses a block list or allow list projection of the value structure of the ``SinkDocument``.
157155
| Defaults to a blank document if no value exists.
158-
156+
159157
* - UuidProvidedInKeyStrategy
160158
- | Full Path: ``com.mongodb.kafka.connect.sink.processor.id.strategy.UuidInKeyStrategy``
161159
| Converts the _id key field to a UUID. The value must be either a string or binary type and must conform to the `UUID format <https://en.wikipedia.org/wiki/Universally_unique_identifier#Format>`__.
@@ -463,12 +461,15 @@ This section provides example configurations for the ``RenameByMapping``
463461
and ``RenameByRegex`` post processors to show how they update field names
464462
in a sink record. The field renaming parameters specify whether to update the
465463
``key`` or ``value`` document in the record using dot notation as well as
466-
the pattern to match and replacement string in a JSON array.
464+
the pattern to match. You must specify the ``RenameByMapping`` and
465+
``RenameByRegex`` properties with your parameters in a JSON array.
467466

468467
The field renaming post processor examples use the following sample sink
469468
record:
470469

471-
**Key document**
470+
.. _sample-key-document:
471+
472+
**Sample Key Document**
472473

473474
.. code-block:: json
474475

@@ -478,7 +479,9 @@ record:
478479
"date_day": 17
479480
}
480481

481-
**Value document**
482+
.. _sample-value-document:
483+
484+
**Sample Value Document**
482485

483486
.. code-block:: json
484487

@@ -494,11 +497,13 @@ record:
494497
RenameByMapping Example
495498
^^^^^^^^^^^^^^^^^^^^^^^
496499

497-
The ``RenameByMapping`` post processor setting is an array of objects.
498-
Each object in the array contains the following JSON element keys:
500+
The ``RenameByMapping`` post processor setting specifies one or more
501+
objects that assigns fields matching a string to a new name in a Key or Value
502+
document.
499503

500-
Each object contains the text to match in the ``oldName`` element and
501-
the replacement text in the ``newName`` element.
504+
Each object contains the text to match in the ``oldName`` element and the
505+
replacement text in the ``newName`` element as described in the table
506+
below.
502507

503508
.. list-table::
504509
:header-rows: 1
@@ -509,34 +514,44 @@ the replacement text in the ``newName`` element.
509514
- Description
510515

511516
* - oldName
512-
- Contains a string that matches on the text to replace.
517+
- Specifies whether to match a key or value document and an appended
518+
string that matches the field to replace.
513519

514520
* - newName
515-
- Contains the replacement text for all matches of the string defined
521+
- Contains the replacement text for all matches of the field defined
516522
in the ``oldName`` field.
517523

518-
.. code-block:: properties
519524

520-
field.renamer.mapping=[{"oldName":"key.location","newName":"city"},{"oldName":"value.flapjacks","newName":"crepes"}]
525+
The following example property matches the "location" field of a Key
526+
document and renames it to "country":
527+
528+
.. code-block:: properties
521529

522-
The record contains the following data after applying the ``RenameByMapping``
523-
post processor:
530+
field.renamer.mapping=[{"oldName":"key.location", "newName":"country"}]
524531

525-
**Key document**
532+
This transforms the :ref:`original key document <sample-key-document>`
533+
to the following one after applying the ``RenameByMapping`` post processor:
526534

527535
.. code-block:: json
528-
:emphasize-lines: 2
529536

530537
{
531-
"city": "Provence",
538+
"country": "Provence",
532539
"date_month": "October",
533540
"date_day": 17
534541
}
535542

536-
**Value document**
543+
You can perform a similar field name assignment for value documents by
544+
specifying ``value`` with the appended field name in the ``oldName``
545+
field as follows:
546+
547+
.. code-block:: properties
548+
549+
field.renamer.mapping=[{"oldName":"value.location", "newName":"city"}]
550+
551+
This transforms the :ref:`original value document <sample-value-document>`
552+
to the following one after applying the ``RenameByMapping`` post processor:
537553

538554
.. code-block:: json
539-
:emphasize-lines: 2
540555

541556
{
542557
"crepes": {
@@ -545,6 +560,14 @@ post processor:
545560
}
546561
}
547562

563+
You can also specify multiple mappings in the ``field.renamer.mapping``
564+
property by using a stringified JSON array as shown in the following
565+
example:
566+
567+
.. code-block:: properties
568+
569+
field.renamer.mapping="[{ "oldName":"key.location", "newName":"city" }, { "oldName":"value.crepes", "newName":"flapjacks" }]"
570+
548571
.. _field-renaming-regex-example:
549572

550573
RenameByRegex
@@ -578,21 +601,22 @@ Example
578601

579602
field.renamer.mapping=[{"regexp":"^key\\.date.*$","pattern":"_","replace":"-"},{"regexp":"^value\\.crepes\\..*","pattern":"purchased","replace":"quantity"}]
580603

581-
The record contains the following data after applying the ``RenameByMapping``
582-
post processor:
604+
When the post processor is applied to the :ref:`sample key document <sample-key-document>`
605+
and the :ref:`sample value document <sample-value-document>`, it produces the
606+
following output:
583607

584-
**Key document**
608+
**Key Document**
585609

586610
.. code-block:: json
587611
:emphasize-lines: 3,4
588612

589613
{
590-
"city": "Provence",
614+
"location": "Provence",
591615
"date-month": "October",
592616
"date-day": 17
593617
}
594618

595-
**Value document**
619+
**Value Document**
596620

597621
.. code-block:: json
598622
:emphasize-lines: 3

0 commit comments

Comments
 (0)