Skip to content

Time series updates for sharding support and updates / deletes #6085

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

jeff-allen-mongo
Copy link
Collaborator

@jeff-allen-mongo jeff-allen-mongo commented Nov 8, 2021

Copy link

@banarun banarun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for adding the docs. I just added some suggestions for details for sharding time-series collection.

Sharding
~~~~~~~~

:ref:`Time series collections <manual-timeseries-collection>` cannot
currently be sharded.
Starting in MongoDB 5.1, sharded time series collections are supported.
Copy link

@banarun banarun Nov 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably add more details here. Some of the key things to highlight would be,

  1. The changes to the shardCollection command, which now accepts timeseries parameter.
  2. The sharding admin commands (like split, moveChunk) are not supported for time-series collection.
  3. The limitations on the shard key pattern (Can only use metaField and/or timeField in the shard key). Details here: https://docs.google.com/document/d/1ljVx7gni5dg6vuLSL2li14lq13T2RqENWEGKKeliCzw/edit#heading=h.1qvrx81umcnx

@jeff-allen-mongo jeff-allen-mongo force-pushed the time-series-updates branch 9 times, most recently from 6c700e6 to 6a39d6a Compare November 8, 2021 22:09
Copy link
Collaborator Author

@jeff-allen-mongo jeff-allen-mongo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your feedback @banarun. I've updated to include additional detail per your suggestions. Could you please take a look?

Cheers

@@ -0,0 +1,3 @@
Required. The name of the field which contains the date in each
time series document. Documents in a time series collection must
have a valid BSON date as the value for the ``timeField``.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as existing copy; shouldn't need review.

metadata should rarely, if ever, change.

The name of the specified field may not be ``_id`` or the same as
the ``timeseries.timeField``. The field can be of any type.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as existing copy; shouldn't need review.

from the same source.

If you do not specify ``timeseries.metaField``, consider the time
span between all measurements that are inserted in the collection.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as existing copy; shouldn't need review.

Copy link

@banarun banarun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Just a couple of minor comments.

Are we using this ticket to also track the upgrade/downgrade produce? If yes, I think there are some FCV implications that we need to mention in the docs.

`````````````````````````````````````

When sharding time series collections, you can only specify the
``metaField``, ``timeField``, or both in the shard key. No other fields,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can also add metaField (or sub-fields of a metaField) to clarify that the field need not be just the metaField?

<sharding-hashed-sharding>` or a :ref:`ranged shard key
<sharding-ranged>`.

- ``timeField`` can only be a :ref:`ranged shard key
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The timeField can only be at the end of the shard key pattern.

@jeff-allen-mongo jeff-allen-mongo changed the title Time series updates Time series updates for sharding support and updates / deletes Nov 9, 2021
@jeff-allen-mongo jeff-allen-mongo merged commit 37d1680 into mongodb:master Nov 9, 2021
mongo-cr-bot pushed a commit that referenced this pull request Jan 30, 2024
#6085)

* remove group from QE supported agg stages

* add taxonomy

* add group back to list

* remove ref to deleted section

* clarify group on unencrypted fields

(cherry picked from commit 8d63816)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants