Skip to content

PYTHON-3388 Propagate Original Error for Write Errors Labeled NoWritesPerformed #1117

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
Dec 2, 2022

Conversation

juliusgeo
Copy link
Contributor

No description provided.

@juliusgeo juliusgeo marked this pull request as ready for review November 16, 2022 00:07
@juliusgeo juliusgeo requested a review from blink1073 November 16, 2022 00:36
blink1073
blink1073 previously approved these changes Nov 16, 2022
Copy link
Member

@blink1073 blink1073 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@@ -836,6 +837,8 @@ def __init__(
# This will be used later if we fork.
MongoClient._clients[self._topology._topology_id] = self

self._indefinite_error: Optional[WriteConcernError] = None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't use an instance variable to track this because it makes MongoClient not thread safe.

if self._indefinite_error and exc.has_error_label("NoWritesPerformed"):
raise self._indefinite_error from exc
if isinstance(exc, WriteConcernError):
self._indefinite_error = exc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_indefinite_error is not needed. last_error already tracks the error that should be propagated.

Copy link
Contributor Author

@juliusgeo juliusgeo Nov 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

last_error just tracks the last error, it does not track the last WriteConcernError, which is what we want to do. This ticket's purpose is to have different behavior for normal retryable errors compared to indefinite errors like WriteConcernError. Changing the logic to use last_error makes it not raise the error correctly:

  File "/Users/julius/Work/mongo-python-driver/test/test_retryable_writes.py", line 648, in test_returns_original_error_code
    client.test.test.insert_one({"_id": 1})
AssertionError: WriteConcernError not raised

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you think we need to track the last WriteConcernError? This change should not track WriteConcernErrors any differently than other errors. This change is about errors labeled with NoWritesPerformed which may or may not be a WriteConcernError.

@blink1073
Copy link
Member

Mypy failure:

pymongo/mongo_client.py:1416: error: Incompatible types in assignment
(expression has type "PyMongoError", variable has type
"Optional[WriteConcernError]")  [assignment]
                            self._indefinite_error = exc

raise self._indefinite_error from exc
if isinstance(exc, WriteConcernError):
self._indefinite_error = exc
with self._indefinite_error_lock:
Copy link
Member

@ShaneHarvey ShaneHarvey Nov 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding a lock here does not help at all. It's still not thread safe. We can't use an instance variable to track something that should be a local variable.

assert last_error is not None
raise last_error from exc
else:
raise
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a missing condition here. If the current error is NoWritesPerformed and the last error is None, then we must raise the current error even though it's labelled NoWritesPerformed. This is True for all the cases above with assert last_error is not None too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing the assertion and refactoring fixed this.

Copy link
Member

@ShaneHarvey ShaneHarvey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add the test from mongodb/specifications#1349?

@juliusgeo
Copy link
Contributor Author

/opt/hostedtoolcache/Python/3.11.0/x64/bin/pre-commit run --show-diff-on-failure --color=always --all-files --hook-stage=manual
[52](https://github.com/mongodb/mongo-python-driver/actions/runs/3518982269/jobs/5898456163#step:4:54)
[INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
[53](https://github.com/mongodb/mongo-python-driver/actions/runs/3518982269/jobs/5898456163#step:4:55)
[INFO] Initializing environment for https://github.com/psf/black.
[54](https://github.com/mongodb/mongo-python-driver/actions/runs/3518982269/jobs/5898456163#step:4:56)
[INFO] Initializing environment for https://github.com/PyCQA/isort.
[55](https://github.com/mongodb/mongo-python-driver/actions/runs/3518982269/jobs/5898456163#step:4:57)
[INFO] Initializing environment for https://gitlab.com/pycqa/flake8.
[56](https://github.com/mongodb/mongo-python-driver/actions/runs/3518982269/jobs/5898456163#step:4:58)
An unexpected error has occurred: CalledProcessError: command: ('/usr/bin/git', 'fetch', 'origin', '--tags')
[57](https://github.com/mongodb/mongo-python-driver/actions/runs/3518982269/jobs/5898456163#step:4:59)
return code: 128
[58](https://github.com/mongodb/mongo-python-driver/actions/runs/3518982269/jobs/5898456163#step:4:60)
expected return code: 0
[59](https://github.com/mongodb/mongo-python-driver/actions/runs/3518982269/jobs/5898456163#step:4:61)
stdout: (none)
[60](https://github.com/mongodb/mongo-python-driver/actions/runs/3518982269/jobs/5898456163#step:4:62)
stderr:
[61](https://github.com/mongodb/mongo-python-driver/actions/runs/3518982269/jobs/5898456163#step:4:63)
    fatal: could not read Username for 'https://gitlab.com/': No such device or address

I do not think this pre-commit failure is related.

@blink1073
Copy link
Member

The pre-commit failure was addressed in #1122

@juliusgeo juliusgeo merged commit 26efc0f into mongodb:master Dec 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants