Skip to content

bpo-42815: Fix issue when thread doesn't copy context of parent thread #24074

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 6 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions Lib/test/test_decimal.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
import unittest
import numbers
import locale
import contextvars
from test.support import (run_unittest, run_doctest, is_resource_enabled,
requires_IEEE_754, requires_docstrings,
requires_legacy_unicode_capi)
Expand Down Expand Up @@ -1616,8 +1617,8 @@ def test_threading(self):
self.finish1 = threading.Event()
self.finish2 = threading.Event()

th1 = threading.Thread(target=thfunc1, args=(self,))
th2 = threading.Thread(target=thfunc2, args=(self,))
th1 = threading.Thread(target=thfunc1, args=(self,), context=contextvars.Context())
th2 = threading.Thread(target=thfunc2, args=(self,), context=contextvars.Context())

th1.start()
th2.start()
Expand Down
26 changes: 26 additions & 0 deletions Lib/test/test_threading.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
from test.support.import_helper import import_module
from test.support.script_helper import assert_python_ok, assert_python_failure

import contextvars
import random
import sys
import _thread
Expand Down Expand Up @@ -928,6 +929,31 @@ def test_debug_deprecation(self):
b'is deprecated and will be removed in Python 3.12')
self.assertIn(msg, err)

def test_contextvars(self):
context_var = contextvars.ContextVar("context_var")
context_var.set("default")

result = None

def target():
nonlocal result
result = context_var.get()

t = threading.Thread(target=target)
t.start()
t.join()

self.assertEqual(result, "default")

custom_ctx = contextvars.Context()
custom_ctx.run(lambda: context_var.set("custom"))

t = threading.Thread(target=target, context=custom_ctx)
t.start()
t.join()

self.assertEqual(result, "custom")


class ThreadJoinOnShutdown(BaseTestCase):

Expand Down
11 changes: 8 additions & 3 deletions Lib/threading.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
from time import monotonic as _time
from _weakrefset import WeakSet
from itertools import islice as _islice, count as _count
from contextvars import copy_context as _copy_context
try:
from _collections import deque as _deque
except ImportError:
Expand Down Expand Up @@ -818,7 +819,7 @@ class Thread:
_initialized = False

def __init__(self, group=None, target=None, name=None,
args=(), kwargs=None, *, daemon=None):
args=(), kwargs=None, *, daemon=None, context=None):
"""This constructor should always be called with keyword arguments. Arguments are:

*group* should be None; reserved for future extension when a ThreadGroup
Expand Down Expand Up @@ -863,6 +864,10 @@ class is implemented.
else:
self._daemonic = current_thread().daemon
self._ident = None
if context is not None:
self._context = context
else:
self._context = _copy_context()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is _copy_context() fast/efficient when contextvars is not used at all?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is information from PEP-567:

The Context mapping is implemented using an immutable dictionary. This allows for a O(1) implementation of the copy_context() function.

I believe it is fast/efficient to use copy_context.

if _HAVE_THREAD_NATIVE_ID:
self._native_id = None
self._tstate_lock = None
Expand Down Expand Up @@ -943,11 +948,11 @@ def run(self):
"""
try:
if self._target is not None:
self._target(*self._args, **self._kwargs)
self._context.run(self._target, *self._args, **self._kwargs)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's one why to fix the issue. Another would be to modify the contextvars module to detect that it runs in a new thread. For exemple, this change doesn't cover the case of threads spawned by C libraries which then use the Python C API to run Python code. Or functions calling _thread.start_new_thread() directly.

I don't know which option is the best. At least, this change is reasonable small.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, maybe I add bad description for this issue. I will try to make things more clear. Every thread has it's own context and they do not share same context. This PR tries fix problem when child thread use empty context instead of copy of a parent context.

finally:
# Avoid a refcycle if the thread is running a function with
# an argument that has a member that points to the thread.
del self._target, self._args, self._kwargs
del self._context, self._target, self._args, self._kwargs

def _bootstrap(self):
# Wrapper around the real bootstrap code that ignores
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Fix issue when new thread doesn't copy context of a parent thread. Patch
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This description is not very helpful. I would prefer to rephrase it to explain that threads no longer share contextvars variables with their parent thread, but the context is now copied.

IMO this change is important enough to be documented in a .. versionchanged:: 3.11 markup in https://docs.python.org/dev/library/threading.html#threading.Thread documentation.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually every thread has it's own context and they do not share same context. This PR tries fix problem when child thread use empty context instead of copy of a parent context.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rephrase the NEWS entry and add the requested versionchanged markup.

provided by Yurii Karabas.
2 changes: 1 addition & 1 deletion Modules/Setup
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@ _symtable symtablemodule.c
#array -DPy_BUILD_CORE_MODULE arraymodule.c # array objects
#cmath cmathmodule.c _math.c -DPy_BUILD_CORE_MODULE # -lm # complex math library functions
#math mathmodule.c _math.c -DPy_BUILD_CORE_MODULE # -lm # math library functions, e.g. sin()
#_contextvars _contextvarsmodule.c # Context Variables
_contextvars _contextvarsmodule.c # Context Variables
#_struct -DPy_BUILD_CORE_MODULE _struct.c # binary structure packing/unpacking
#_weakref _weakref.c # basic weak reference support
#_testcapi _testcapimodule.c # Python C API test module
Expand Down