Skip to content

Commit 2356cf7

Browse files
biniona-mongodbschmalliso
authored andcommitted
(DOCSP-18848) Remove Change Stream TODO (#172)
1 parent 65fda3d commit 2356cf7

File tree

1 file changed

+29
-18
lines changed

1 file changed

+29
-18
lines changed

source/source-connector/fundamentals/change-streams.txt

Lines changed: 29 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -87,22 +87,33 @@ connector closes its change stream when you stop it.
8787
Resume Tokens
8888
~~~~~~~~~~~~~
8989

90-
Your connector stores a **resume token** to keep track of what changes
91-
it has processed. A resume token is a piece of data that references
92-
the ``_id`` field of a change event document in your MongoDB oplog.
93-
Your connector only processes relevant change event documents written to the oplog after the
94-
document referenced by its resume token.
95-
96-
If your source connector does not have a resume token, such as when you start
97-
the connector for the first time, your connector processes relevant change
98-
events written to the oplog after it first connects to MongoDB.
99-
100-
.. <TODO for Ross>: Confirm if this is after the source connector first connects
101-
to MongoDB, or after it first starts. Pretty sure it is after it first
102-
connects to MongoDB. I think this is getting in the weeds of how change streams are
103-
implemented in the Java driver and I'm not sure if this is a valuable distinction.
104-
105-
If your source connector's resume token does not correspond to any entry in your
106-
oplog, your connector has an invalid resume token. To learn how to recover from an
107-
invalid resume token,
90+
Your connector uses a **resume token** as its **offset**. An offset is a value
91+
your connector stores in an {+ak+} topic to keep track of what source data it
92+
has processed. Your connector uses its offset value when it must recover from
93+
a restart or crash. A resume token is a piece of data that references the
94+
``_id`` field of a change event document in your MongoDB oplog.
95+
96+
If your source connector does not have an offset, such as when you start
97+
the connector for the first time, your connector starts a new change stream.
98+
Once your connector receives its first change event document and publishes that
99+
document to {+ak+}, your connector stores the resume token of that document as
100+
its offset.
101+
102+
If the resume token value of your source connector's offset does not correspond to
103+
any entry in your MongoDB deployment's oplog, your connector has an invalid resume
104+
token. To learn how to recover from an invalid resume token,
108105
:ref:`see our troubleshooting guide <kafka-troubleshoot-recover-invalid-resume-token>`.
106+
107+
To learn more about resume tokens, see the following resources:
108+
109+
- :manual:`Resume a Change Stream </changeStreams/#resume-a-change-stream>`
110+
in the MongoDB manual
111+
- :manual:`Change Events </reference/change-events/#std-label-change-stream-event-id>`
112+
in the MongoDB manual
113+
114+
To learn more about offsets, see the following resources:
115+
116+
- {+kc+} ``offset.storage.topic``
117+
`configuration option documentation <https://docs.confluent.io/platform/current/installation/configuration/connect/index.html#offset-storage-topic>`__
118+
- {+kc+} ``OffsetStorageReader``
119+
`API documentation <https://kafka.apache.org/0110/javadoc/org/apache/kafka/connect/storage/OffsetStorageReader.html>`__

0 commit comments

Comments
 (0)