-
-
Notifications
You must be signed in to change notification settings - Fork 32.8k
gh-116738: Make cProfile module thread-safe #138229
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
@yoney - LGTM. I think it's worth doing the work to make |
@mpage Thanks for the review! I think collecting profiling data per thread and merging it lazily before reporting could work. This kind of refactoring would likely benefit For now, this PR addresses the immediate issue that causes crashes. While the recursive-count might still be incorrect, the other counts like call-count are accurate, and this fix resolves the crash. I think it makes sense to land this now and consider the follow-up separately. |
Misc/NEWS.d/next/Core_and_Builtins/2025-08-28-09-29-46.gh-issue-116738.yLZJpV.rst
Outdated
Show resolved
Hide resolved
…e-116738.yLZJpV.rst
Thanks @yoney for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14. |
(cherry picked from commit 8554c0917e25a7abe12b3000f1589b6566c91a25) Co-authored-by: Alper <[email protected]> Co-authored-by: Kumar Aditya <[email protected]>
GH-138575 is a backport of this pull request to the 3.14 branch. |
This change uses critical sections to make
cProfile (_lsprof)
thread-safe when the GIL is disabled. It does this by applying the@critical_section
clinic directive toProfilerObject
.Protecting
ProfilerObject
prevents crashes, and the thread sanitizer does not report errors for the new FT test. However, the stats still show incorrect recursive call information.This happens because of limitations in the
cProfile
module, and these issues are not directly related to the FT Python build. By adjustingsys.setswitchinterval()
, you can see similar incorrect recursive call reports even in a non-FT Python build.This is a quick fix to prevent crashes. It is also a request for comments on whether we should make bigger changes to the
cProfile
module to handle multithreading better, such as collecting profile data per thread.cc: @mpage, @colesbury, @Yhg1s