Skip to content

Commit e474abb

Browse files
elyse-mdbcarriecwk
elyse-mdb
and
carriecwk
authored
(DOCSP-43962) [Kafka] Kafka connector nested components (#167)
* (DOCSP-43962) Unnested 1 example in tip. * (DOCSP-43962) Removing the subjunctive should * (DOCSP-42962) Unnested 1 important admonition in table. * (DOCSP-42962) Typo fix. * (DOCSP-42962) Unnested 1 important admonition in table. * (DOCSP-42962) Unnested 1 tip in table. * (DOCSP-42962) Fixing indentation error and unnesting 1 important admonition in table. * (DOCSP-42962) Spacing. * (DOCSP-42962) Build errors. * (DOCSP-42962) Unnested 1 example and 1 note in table. * (DOCSP-42962) Unnested 2 examples and 1 note in include. * (DOCSP-42962) Replace quotation marks with monospace. * (DOCSP-42962) Unnested 2 notes in table. * (DOCSP-42962) consistency. * (DOCSP-42962) Edited for consistency and unnested 2 tips. * (DOCSP-42962) Removing extra line breaks. * (DOCSP-42962) Unnested 2 important admonitions in table. * (DOCSP-42962) Fix rendering error. * (DOCSP-42962) Fixing rendering issues. * (DOCSP-42962) Unnested 2 notes and 1 example in table. * (DOCSP-42962) Unnested 3 important admonitions * (DOCSP-42962) Spacing fix. * (DOCSP-42962) Unnested 1 example and 3 tips. * (DOCSP-42962) Unnested 1 note, 2 tips, 1 important admonition. * (DOCSP-42962) Spacing. * (DOCSP-42962) Spacing. * (DOCSP-42962) Normalizing spacing and adding reference to the Data Formats page. * (DOCSP-42962) Unnested 3 tips, 1 example and 1 important admonition in table. * (DOCSP-42962) Regularizing usage of pipeline operator line breaks. * (DOCSP-42962) Spacing. * (DOCSP-42962) Spacing. * (DOCSP-42962) Normalizing spacing across pages. * Update source/source-connector/configuration-properties/kafka-topic.txt Co-authored-by: carriecwk <[email protected]> * Update source/source-connector/configuration-properties/kafka-topic.txt Co-authored-by: carriecwk <[email protected]> --------- Co-authored-by: Elyse Foreman <[email protected]> Co-authored-by: carriecwk <[email protected]>
1 parent 6bf4efa commit e474abb

20 files changed

+258
-276
lines changed

‎source/includes/copy-existing-admonition.rst

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,5 @@
1-
.. note:: Data Copy Can Produce Duplicate Events
2-
3-
If any system changes the data in the database while the source connector
4-
converts existing data from it, MongoDB may produce duplicate change
5-
stream events to reflect the latest changes. Since the change stream
6-
events on which the data copy relies are idempotent, the copied data is
7-
eventually consistent.
1+
If any system changes the data in the database while the source connector
2+
converts existing data from it, MongoDB may produce duplicate change
3+
stream events to reflect the latest changes. Since the change stream
4+
events on which the data copy relies are idempotent, the copied data is
5+
eventually consistent.

‎source/includes/externalize-secrets.rst

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,5 @@
1-
.. important:: Avoid Exposing Your Authentication Credentials
2-
3-
To avoid exposing your authentication credentials in your
4-
``connection.uri`` setting, use a
5-
`ConfigProvider <https://docs.confluent.io/current/connect/security.html#externalizing-secrets>`__
6-
and set the appropriate configuration parameters.
1+
:gold:`IMPORTANT:` To avoid exposing your authentication credentials in your
2+
``connection.uri`` setting, use a
3+
`ConfigProvider <https://docs.confluent.io/current/connect/security.html#externalizing-secrets>`__
4+
and set the appropriate configuration parameters.
75

‎source/introduction/data-formats.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
.. _kafka-data-formats:
2+
13
============
24
Data Formats
35
============

‎source/sink-connector/configuration-properties/connector-message.txt

Lines changed: 10 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -75,8 +75,9 @@ Settings
7575
When set to ``true``, the default value, the
7676
connector writes a batch of records as an ordered bulk write
7777
operation.
78-
| To learn more about bulk write operations, see the :ref:`Write
79-
Model Strategies page <sink-connector-bulk-write-ops>`.
78+
|
79+
| To learn more about bulk write operations, see
80+
:ref:`Bulk Write Operations <sink-connector-bulk-write-ops>`.
8081
|
8182
| **Default**: ``true``
8283
| **Accepted Values**: ``true`` or ``false``
@@ -111,15 +112,13 @@ Settings
111112
| The maximum number of tasks to create for this connector. The
112113
connector may create fewer than the maximum tasks specified if it
113114
cannot handle the level of parallelism you specify.
114-
115-
.. important:: Multiple Tasks May Process Messages Out of Order
116-
117-
If you specify a value greater than ``1``, the connector enables
118-
parallel processing of the tasks. If your topic has multiple
119-
partition logs, which enables the connector to read from the
120-
topic in parallel, the tasks may process the messages out of
121-
order.
122-
115+
|
116+
| :gold:`IMPORTANT:` If you specify a value greater than ``1``,
117+
the connector enables parallel processing of the tasks.
118+
If your topic has multiple partition logs, which enables
119+
the connector to read from the topic in parallel,
120+
the tasks may process the messages out of order.
121+
|
123122
| **Default**: ``1``
124123
| **Accepted Values**: An integer
125124

‎source/sink-connector/configuration-properties/error-handling.txt

Lines changed: 21 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -48,19 +48,21 @@ Settings
4848
| Whether to continue processing messages if the connector encounters
4949
an error. Allows the connector to override the ``errors.tolerance``
5050
Kafka cluster setting.
51+
|
5152
| When set to ``none``, the connector reports any error and
5253
blocks further processing of the rest of the messages.
54+
|
5355
| When set to ``all``, the connector ignores any problematic messages.
56+
|
5457
| When set to ``data``, the connector tolerates only data errors and
5558
fails on all other errors.
59+
|
5660
| To learn more about error handling strategies, see the
5761
:ref:`<kafka-sink-handle-errors>` page.
58-
59-
.. note::
60-
61-
This property overrides the `errors.tolerance <https://docs.confluent.io/platform/current/installation/configuration/connect/sink-connect-configs.html#errors-tolerance>`__
62-
property of the Connect Framework.
63-
62+
|
63+
| This property overrides the `errors.tolerance <https://docs.confluent.io/platform/current/installation/configuration/connect/sink-connect-configs.html#errors-tolerance>`__
64+
| property of the Connect Framework.
65+
|
6466
| **Default:** Inherits the value from the ``errors.tolerance``
6567
setting.
6668
| **Accepted Values**: ``"none"`` or ``"all"``
@@ -73,16 +75,14 @@ Settings
7375
failed operations to the log file. The connector classifies
7476
errors as "tolerated" or "not tolerated" using the
7577
``errors.tolerance`` or ``mongo.errors.tolerance`` settings.
76-
78+
|
7779
| When set to ``true``, the connector logs both "tolerated" and
7880
"not tolerated" errors.
7981
| When set to ``false``, the connector logs "not tolerated" errors.
80-
81-
.. note::
82-
83-
This property overrides the `errors.log.enable <https://docs.confluent.io/platform/current/installation/configuration/connect/sink-connect-configs.html#errors-log-enable>`__
84-
property of the Connect Framework.
85-
82+
|
83+
| This property overrides the `errors.log.enable <https://docs.confluent.io/platform/current/installation/configuration/connect/sink-connect-configs.html#errors-log-enable>`__
84+
| property of the Connect Framework.
85+
|
8686
| **Default:** ``false``
8787
| **Accepted Values**: ``true`` or ``false``
8888

@@ -104,7 +104,8 @@ Settings
104104
| Name of topic to use as the dead letter queue. If blank, the
105105
connector does not send any invalid messages to the dead letter
106106
queue.
107-
| For more information about the dead letter queue, see the
107+
|
108+
| To learn more about the dead letter queue, see the
108109
:ref:`Dead Letter Queue Configuration Example <sink-dead-letter-queue-configuration-example>`.
109110
|
110111
| **Default:** ``""``
@@ -116,11 +117,13 @@ Settings
116117
| **Description:**
117118
| Whether the connector should include context headers when it
118119
writes messages to the dead letter queue.
120+
|
119121
| To learn more about the dead letter queue, see the
120122
:ref:`Dead Letter Queue Configuration Example <sink-dead-letter-queue-configuration-example>`.
123+
|
121124
| To learn about the exceptions the connector defines and
122-
reports through context headers, see the
123-
:ref:`<sink-configuration-error-handling-dlq-errors>` section.
125+
reports through context headers, see
126+
:ref:`<sink-configuration-error-handling-dlq-errors>`.
124127
|
125128
| **Default:** ``false``
126129
| **Accepted Values**: ``true`` or ``false``
@@ -132,7 +135,8 @@ Settings
132135
| The number of nodes on which to replicate the dead letter queue
133136
topic. If you are running a single-node Kafka cluster, you must
134137
set this to ``1``.
135-
| For more information about the dead letter queue, see the
138+
|
139+
| To learn more about the dead letter queue, see the
136140
:ref:`Dead Letter Queue Configuration Example <sink-dead-letter-queue-configuration-example>`.
137141
|
138142
| **Default:** ``3``

‎source/sink-connector/configuration-properties/id-strategy.txt

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -76,8 +76,9 @@ Settings
7676
|
7777
| **Description:**
7878
| Whether the connector should delete documents when the key value
79-
matches a document in MongoDB and the value field is null. This
80-
setting applies when you specify an id generation strategy that
79+
matches a document in MongoDB and the value field is null.
80+
|
81+
| This setting applies when you specify an id generation strategy that
8182
operates on the key document such as ``FullKeyStrategy``,
8283
``PartialKeyStrategy``, and ``ProvidedInKeyStrategy``.
8384
|

‎source/sink-connector/configuration-properties/kafka-topic.txt

Lines changed: 9 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -42,10 +42,8 @@ Settings
4242
| **Description:**
4343
| A list of Kafka topics that the sink connector watches.
4444

45-
.. note::
46-
47-
You can define either the ``topics`` or the ``topics.regex``
48-
setting, but not both.
45+
You can define either the ``topics`` or the ``topics.regex``
46+
setting, but not both.
4947

5048
| **Accepted Values**: A comma-separated list of valid Kafka topics
5149

@@ -58,20 +56,16 @@ Settings
5856
| A regular expression that matches the Kafka topics that the sink
5957
connector watches.
6058

61-
.. example::
62-
63-
.. code-block:: properties
64-
65-
topics.regex=activity\\.\\w+\\.clicks$
59+
For example, the following regex matches topic names such as
60+
"activity.landing.clicks" and "activity.support.clicks".
61+
It does not match the topic names "activity.landing.views" and "activity.clicks".
6662

67-
This regex matches topic names such as "activity.landing.clicks"
68-
and "activity.support.clicks". It does not match the topic names
69-
"activity.landing.views" and "activity.clicks".
63+
.. code-block:: properties
7064

71-
.. note::
65+
topics.regex=activity\\.\\w+\\.clicks$
7266

73-
You can define either the ``topics`` or the ``topics.regex``
74-
setting, but not both.
67+
You can define either the ``topics`` or the ``topics.regex``
68+
setting, but not both.
7569

7670
| **Accepted Values**: A valid regular expression pattern using ``java.util.regex.Pattern``.
7771

‎source/sink-connector/configuration-properties/mongodb-connection.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,7 @@ Settings
6868
| When set to ``true``, if the connector calls a command on your
6969
MongoDB instance that's deprecated in the declared {+stable-api+}
7070
version, it raises an exception.
71+
|
7172
| You can set the API version with the ``server.api.version``
7273
configuration option. For more information on the {+stable-api+}, see
7374
the MongoDB manual entry on the
@@ -83,6 +84,7 @@ Settings
8384
| When set to ``true``, if the connector calls a command on your
8485
MongoDB instance that's not covered in the declared {+stable-api+}
8586
version, it raises an exception.
87+
|
8688
| You can set the API version with the ``server.api.version``
8789
configuration option. For more information on the {+stable-api+}, see
8890
the MongoDB manual entry on the

‎source/sink-connector/configuration-properties/mongodb-namespace.txt

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -43,14 +43,12 @@ Settings
4343
database or collection in which to sink the data. The default
4444
``DefaultNamespaceMapper`` uses values specified in the
4545
``database`` and ``collection`` properties.
46-
47-
.. seealso::
48-
49-
The connector includes an alternative class for specifying the
50-
database and collection called ``FieldPathNamespaceMapper``. See
51-
the :ref:`FieldPathNamespaceMapper settings <fieldpathnamespacemapper-settings>`
52-
for more information.
53-
46+
|
47+
| The connector includes an alternative class for specifying the
48+
| database and collection called ``FieldPathNamespaceMapper``. See
49+
| :ref:`FieldPathNamespaceMapper Settings <fieldpathnamespacemapper-settings>`
50+
| for more information.
51+
|
5452
| **Default**:
5553

5654
.. code-block:: none
@@ -149,10 +147,12 @@ You can use the following settings to customize the behavior of the
149147
| **Description**:
150148
| Whether to throw an exception when either the document is missing the
151149
mapped field or it has an invalid BSON type.
150+
|
152151
| When set to ``true``, the connector does not process documents
153152
missing the mapped field or that contain an invalid BSON type.
154153
The connector may halt or skip processing depending on the related
155154
error-handling configuration settings.
155+
|
156156
| When set to ``false``, if a document is missing the mapped field or
157157
if it has an invalid BSON type, the connector defaults to
158158
writing to the specified ``database`` and ``collection`` settings.

‎source/sink-connector/configuration-properties/post-processors.txt

Lines changed: 7 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -40,13 +40,10 @@ Settings
4040
| **Description:**
4141
| A list of post-processor classes the connector should apply to
4242
process the data before saving it to MongoDB.
43-
44-
.. seealso::
45-
46-
For more information on post-processors and examples of
47-
their usage, see the section on
48-
:doc:`Post-processors </sink-connector/fundamentals/post-processors>`.
49-
43+
|
44+
| To learn more about post-processors and see examples of
45+
| their usage, see
46+
| :doc:`Sink Connector Post Processors </sink-connector/fundamentals/post-processors>`.
5047
|
5148
| **Default**:
5249

@@ -129,12 +126,9 @@ Settings
129126
| **Description:**
130127
| The class that specifies the ``WriteModelStrategy`` the connector should
131128
use for :manual:`Bulk Writes </core/bulk-write-operations/index.html>`.
132-
133-
.. seealso::
134-
135-
For information on how to create your own strategy, see
136-
:ref:`<kafka-sink-write-model-create-custom-strategy>`.
137-
129+
|
130+
| To learn more about how to create your own strategy, see
131+
| :ref:`<kafka-sink-write-model-create-custom-strategy>`.
138132
|
139133
| **Default**:
140134

0 commit comments

Comments
 (0)