Skip to content

Commit fa220ec

Browse files
hongweipengrhettinger
authored andcommitted
Raise a RuntimeError when tee iterator is consumed from different threads (GH-15567)
1 parent 13f37f2 commit fa220ec

File tree

3 files changed

+12
-1
lines changed

3 files changed

+12
-1
lines changed

Doc/library/itertools.rst

+2-1
Original file line numberDiff line numberDiff line change
@@ -643,7 +643,8 @@ loops that truncate the stream.
643643

644644
Once :func:`tee` has made a split, the original *iterable* should not be
645645
used anywhere else; otherwise, the *iterable* could get advanced without
646-
the tee objects being informed.
646+
the tee objects being informed. the :func:`tee` iterator can not be consumed
647+
from different threads, even if an underlying iterator is thread-safe.
647648

648649
This itertool may require significant auxiliary storage (depending on how
649650
much temporary data needs to be stored). In general, if one iterator uses
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Raise a RuntimeError when itertools.tee() iterator is consumed from different
2+
threads. Patch by hongweipeng.

Modules/itertoolsmodule.c

+8
Original file line numberDiff line numberDiff line change
@@ -452,6 +452,7 @@ typedef struct {
452452
teedataobject *dataobj;
453453
int index; /* 0 <= index <= LINKCELLS */
454454
PyObject *weakreflist;
455+
unsigned long thread_id;
455456
} teeobject;
456457

457458
static PyTypeObject teedataobject_type;
@@ -680,6 +681,11 @@ tee_next(teeobject *to)
680681
{
681682
PyObject *value, *link;
682683

684+
if (to->thread_id != PyThread_get_thread_ident()) {
685+
PyErr_SetString(PyExc_RuntimeError,
686+
"tee() iterator can not be consumed from different threads.");
687+
return NULL;
688+
}
683689
if (to->index >= LINKCELLS) {
684690
link = teedataobject_jumplink(to->dataobj);
685691
if (link == NULL)
@@ -713,6 +719,7 @@ tee_copy(teeobject *to, PyObject *Py_UNUSED(ignored))
713719
newto->dataobj = to->dataobj;
714720
newto->index = to->index;
715721
newto->weakreflist = NULL;
722+
newto->thread_id = to->thread_id;
716723
PyObject_GC_Track(newto);
717724
return (PyObject *)newto;
718725
}
@@ -745,6 +752,7 @@ tee_fromiterable(PyObject *iterable)
745752

746753
to->index = 0;
747754
to->weakreflist = NULL;
755+
to->thread_id = PyThread_get_thread_ident();
748756
PyObject_GC_Track(to);
749757
done:
750758
Py_XDECREF(it);

0 commit comments

Comments
 (0)