Skip to content

Speed up test_weakref in the free-threaded build #120065

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
colesbury opened this issue Jun 4, 2024 · 0 comments
Closed

Speed up test_weakref in the free-threaded build #120065

colesbury opened this issue Jun 4, 2024 · 0 comments
Labels
tests Tests in the Lib/test dir topic-free-threading

Comments

@colesbury
Copy link
Contributor

colesbury commented Jun 4, 2024

The test_weakref test is one of the slowest tests when running in the free-threaded build.

The problem is the default period for the collect_in_thread() function is too short (100 µs). This isn't too much of a problem for the default build because the GIL switch interval is 5 ms, which effectively makes the smaller period irrelevant.

With the free-threaded build, this the 100 µs period means that the test spends the majority of its time calling gc.collect() nearly non-stop.

We should increase the period to 5 ms or so. On my machine, this decreases up the overall test_weakref time from 1 minute to 8 seconds.

@contextlib.contextmanager
def collect_in_thread(period=0.0001):
"""
Ensure GC collections happen in a different thread, at a high frequency.
"""
please_stop = False
def collect():
while not please_stop:
time.sleep(period)
gc.collect()
with support.disable_gc():
t = threading.Thread(target=collect)
t.start()
try:
yield
finally:
please_stop = True
t.join()

Linked PRs

@colesbury colesbury added tests Tests in the Lib/test dir topic-free-threading labels Jun 4, 2024
colesbury added a commit to colesbury/cpython that referenced this issue Jun 4, 2024
This matches the default GIL switch interval. It greatly speeds up the
free-threaded build: previously, it spent nearly all its time in
`gc.collect()`.
colesbury added a commit to colesbury/cpython that referenced this issue Jun 4, 2024
colesbury added a commit that referenced this issue Jun 5, 2024
This matches the default GIL switch interval. It greatly speeds up the
free-threaded build: previously, it spent nearly all its time in
`gc.collect()`.
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jun 5, 2024
…GH-120068)

This matches the default GIL switch interval. It greatly speeds up the
free-threaded build: previously, it spent nearly all its time in
`gc.collect()`.
(cherry picked from commit 4bba1c9)

Co-authored-by: Sam Gross <[email protected]>
colesbury added a commit that referenced this issue Jun 5, 2024
…0068) (#120110)

This matches the default GIL switch interval. It greatly speeds up the
free-threaded build: previously, it spent nearly all its time in
`gc.collect()`.
(cherry picked from commit 4bba1c9)

Co-authored-by: Sam Gross <[email protected]>
barneygale pushed a commit to barneygale/cpython that referenced this issue Jun 5, 2024
…#120068)

This matches the default GIL switch interval. It greatly speeds up the
free-threaded build: previously, it spent nearly all its time in
`gc.collect()`.
noahbkim pushed a commit to hudson-trading/cpython that referenced this issue Jul 11, 2024
…#120068)

This matches the default GIL switch interval. It greatly speeds up the
free-threaded build: previously, it spent nearly all its time in
`gc.collect()`.
estyxx pushed a commit to estyxx/cpython that referenced this issue Jul 17, 2024
…#120068)

This matches the default GIL switch interval. It greatly speeds up the
free-threaded build: previously, it spent nearly all its time in
`gc.collect()`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tests Tests in the Lib/test dir topic-free-threading
Projects
None yet
Development

No branches or pull requests

1 participant