DRIVERS-709: Unified Test Format #846
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
prashantmital
suggested changes
Aug 21, 2020
jmikola
changed the title
mbroadst
reviewed
Aug 27, 2020
jmikola
commented
Oct 7, 2020
source/unified-test-format/tests/valid-pass/poc-transactions-mongos-pin-auto.yml
Show resolved
Hide resolved
ShaneHarvey
reviewed
Oct 7, 2020
source/unified-test-format/tests/valid-pass/poc-transactions-mongos-pin-auto.yml
Show resolved
Hide resolved
alcaeus
approved these changes
Oct 8, 2020
p-mongo
reviewed
Oct 9, 2020
p-mongo
reviewed
Oct 9, 2020
p-mongo
approved these changes
Oct 9, 2020
kevinAlbs
pushed a commit
to kevinAlbs/specifications
that referenced
this pull request
Nov 9, 2021
* DRIVERS-709 wip * consolidate test format, simplify assert syntax, docs for matching * bulkWrite errors, schemaVersion, and configureFailPoint APM exclusion * Revise schema version and move allowMultipleMongoses to top-level * Define terms and start test execution steps * Readability improvements and TODO items * Suggest YAML nodes and anchors for collection/database names * Fix link text and version formatting * indent Server Fail Points and describe process for sharded-replicaset * Test implementation and various other clarifications * Language edits from PR feedback * Require explicit database and collection names and add extra notes * observeEvents and useMultipleMongoses client entity options * Example insertOne test * special operations for sessions spec tests * open questions * open question for representing options in operation.arguments * Rename schema and test file to convey they are outdated * clarify schema version comparisons and compatibility * lsid assertions require observed commands with lsid fields * Rename topologies link related spec tickets * top-level description field * change stream iteration, open questions, and errorCode assertion * Future Work, createchangeStream operation, and review fixes * Update last modified and advisors * Fix typos * Changes for events, GridFS, and error assertions Close existing Open Questions and add new Future Work sections. * runOnRequirements, expectedEventsForClient, refer to YAML objects * Sync JSON file with YAML to add description field * Require arrays, rename "expect" prefix, clarify "sharded" requirement * Remove note about clearing state between test files * Note that server versions are compared numerically * Fix typo * Clarify type of observeEvents * Require BSON types be expressed as strings for $$type operator * Clarify test.outcome and comparison rules * Make formatting of "primary" RP consistent with WC and RC * Add Goals, clarify schemaVersion, and Future Work entries * Elaborate on schema version * Clarify (un)supported entity types * Allow equivalent language types for BSON types in entity map * Raise errors for unsupported operations and arguments * Fix link syntax * Feedback from Oleg * $$sessionLsid error handling and heartbeatFrequencyMS tweaks * Clarify BSON type support and advise against using deprecated types * Clarify server version string comparison rules * Revise version docs, MUST reset state, explain SHOULD in Design Rationale * Improve Evaluating Matches docs * Test runners MUST raise an error for incompatible files * Future work for operations/arguments and schema version * Prohibit $$unsetOrMatches for array elements * Optional arrays must be non-empty, revise expectedEvent and $$matchesHexBytes * Update schema for current spec syntax * Update Makefile for new schema * expectedError.isError can only be true * Add invalid test files for testing the schema itself * Prohibit additionalProperties in test objects * Requirements for model objects and iterables and clarify failPoint Also restructures Entity Test Operations sections * Fix RST syntax and typos * Specify descriptions for invalid schema tests * Initial valid-fail and valid-pass tests * Clarify iteration rules * Don't mix testRunner ops with error/result expectations * Additional POC tests * Clarifications for CRUD operations * Advise using $$unsetOrMatches for InsertOneResult * POC for session tests * POC for transaction tests * Fix insertOne syntax * Use complete fullDocument assertions in change stream tests * Clarify rules for enabling event listeners * Fix event assertions for ChangeStream tests * Fix syntax of callback argument for withTransaction tests * Remove stream entity and add proxy operations for GridFS * Reorganized entity operation content and advise not using watch * Fix retryable reads test and add additional find test * Fix retryable writes test to use correct collection object * Fix syntax errors in session tests * Fix typos in transaction tests * Specify useMultipleMongoses:false for failPoint tests * Fix outcome assertion in session pinning test * Add related issue and iterateUntilDocumentOrError clarification * Rename returnDocument test and fix syntax error * Require sharded-replicaset for retryable write session test * Fix typo * Remove "Related Issues" section and clear "Change Log" * Fix schema validation error in returnDocument-enum-invalid.yml * Generate JSON for YAML tests * Add schema checks to Travis CI
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://jira.mongodb.org/browse/DRIVERS-709