Crash when free-threaded Python is built with --with-trace-refs

[XuehaiPan]

Crash report

What happened?

I'm using ThreadPoolExecutor to write tests for free-threading enabled C extensions.

• deps(python): enable Python 3.13t support metaopt/optree#137

When running concurrent threads larger than num_workers, CPython crashes. CPython crashes when num_futures >= 2 if the test function is complex enough.

The Python is installed via the following configuration with PyDebug enabled:

./configure --prefix="${PWD}/pydev" \
    --enable-ipv6 --with-openssl="$(brew --prefix openssl)" \
    --with-system-expat --with-system-libmpdec \
    --with-assertions --with-pydebug --with-trace-refs \
    --disable-gil

from concurrent.futures import ThreadPoolExecutor

NUM_WORKERS = 32
NUM_FUTURES = 1024

def concurrent_run(func):
    with ThreadPoolExecutor(max_workers=NUM_WORKERS) as executor:
        for _ in range(NUM_FUTURES):
            executor.submit(func)


concurrent_run(lambda : None)

Python 3.13.0rc2+ experimental free-threading build (heads/3.13:112b1704fa6, Sep 13 2024, 15:34:48) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from concurrent.futures import ThreadPoolExecutor
>>> NUM_WORKERS = 32
... NUM_FUTURES = 1024
... 
... def concurrent_run(func):
...     with ThreadPoolExecutor(max_workers=NUM_WORKERS) as executor:
...         for _ in range(NUM_FUTURES):
...             executor.submit(func)
...             
>>> concurrent_run(lambda : None)
Assertion failed: (value == REFCHAIN_VALUE), function _PyRefchain_Remove, file object.c, line 195.
Fatal Python error: Aborted

Thread 0x0000000173e2b000 (most recent call first):
  File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 304 in __enter__
  File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 528 in release
  File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/concurrent/futures/thread.py", line 87 in _worker
  File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 992 in run
  File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 1041 in _bootstrap_inner
  File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 1012 in _bootstrap

Thread 0x0000000172e1f000 (most recent call first):
  File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/concurrent/futures/thread.py", line 89 in _worker
  File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 992 in run
  File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 1041 in _bootstrap_inner
  File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 1012 in _bootstrap

Thread 0x0000000171e13000 (most recent call first):
  File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/concurrent/futures/thread.py", line 89 in _worker
  File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 992 in run
  File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 1041 in _bootstrap_inner
  File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 1012 in _bootstrap

Current thread 0x0000000170e07000 (most recent call first):
Assertion failed: (PyCode_Check(f->f_executable)), function _PyFrame_GetCode, file pycore_frame.h, line 81.

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Output from running 'python -VV' on the command line:

Python 3.13.0rc2+ experimental free-threading build (heads/3.13:112b1704fa6, Sep 13 2024, 15:34:48) [Clang 15.0.0 (clang-1500.3.9.4)]

Linked PRs

• gh-124043: Disallow mixing --with-trace-refs and --disable-gil #124078
• [3.13] gh-124043: Disallow mixing --with-trace-refs and --disable-gil (GH-124078) #124138

[ZeroIntensity]

At a glance, looks like this could be a data race. I'll take a look.

[ZeroIntensity]

Ah, --with-trace-refs is unsupported under free-threading, so this is a non-issue. As a double check, I confirmed that this doesn't occur with tracerefs disabled. I wonder if configure should disallow mixing --with-trace-refs and --disable-gil, as this isn't the first case of this happening. See #122921, for example.

@encukou, this needs pending.

[colesbury]

Let's disable --with-trace-refs until it works so that more people don't run into this

[ZeroIntensity]

Let's disable --with-trace-refs until it works so that more people don't run into this

I've created #124078 for this.

[ZeroIntensity]

Should this get closed, or are we planning on merging the backport before 3.13.0?

[encukou]

There's no free-threading before 1.13, so I don't think we should no backports there.
But, do we close issues before the PR is merged? (I assume that it'll be for 3.13.1, next week.)

[ZeroIntensity]

But, do we close issues before the PR is merged?

I'm not totally sure, I've seen core devs close issues when the only remaining PR is for 3.13, because the branch is locked (but I've also seen other core devs leave issues open, so I don't know).

[ZeroIntensity]

I think everything is done here. Thank you for the report, @XuehaiPan!

[terryjreedy]

I think issues should be left open until all backports done. The right sidebar of PRs have a Development button to make closing the (3.13) PR close the issue. I posted on Discord release-discussions.