Skip to content

Improvements to test.support.interpreters.Interpreter #132775

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
ericsnowcurrently opened this issue Apr 21, 2025 · 1 comment
Open

Improvements to test.support.interpreters.Interpreter #132775

ericsnowcurrently opened this issue Apr 21, 2025 · 1 comment
Assignees
@ericsnowcurrently
Labels
3.14 bugs and security fixes stdlib Python modules in the Lib dir topic-subinterpreters type-feature A feature request or enhancement

Comments

@ericsnowcurrently
Copy link
Member

ericsnowcurrently commented Apr 21, 2025 •
edited by bedevere-app bot
Loading

Interpreter.call() should be able to support arbitrary callables, full args, and return values.

Linked PRs
@ericsnowcurrently ericsnowcurrently added the type-bug An unexpected behavior, bug, or error label Apr 21, 2025
@ericsnowcurrently ericsnowcurrently self-assigned this Apr 21, 2025
@ericsnowcurrently ericsnowcurrently added stdlib Python modules in the Lib dir topic-subinterpreters 3.14 bugs and security fixes labels Apr 21, 2025
This was referenced Apr 25, 2025
Merged
Closed
Merged
Merged
Merged
ericsnowcurrently added a commit that referenced this issue Apr 28, 2025
@ericsnowcurrently
gh-132775: Cleanup Related to crossinterp.c Before Further Changes (g…
6f04325 
…h-132974)

This change consists of adding tests and moving code around, with some renaming thrown in.
@bedevere-app bedevere-app bot mentioned this issue Apr 28, 2025
Merged
ericsnowcurrently added a commit that referenced this issue Apr 28, 2025
@ericsnowcurrently
gh-132775: Add _PyModule_GetFilenameObject() and _PyModule_GetFilenam…
c172382 
…eUTF8() (gh-132979)

They are derived from the existing `PyModule_GetFilenameObject().

They are used by a later change related to pickle and handling __main__.
ericsnowcurrently added a commit that referenced this issue Apr 28, 2025
@ericsnowcurrently
gh-132775: Drop PyUnstable_InterpreterState_GetMainModule() (gh-132978)
fe462f5 
We replace it with _Py_GetMainModule(), and add _Py_CheckMainModule(), but both in the internal-only C-API.  We also add _PyImport_GetModulesRef(), which is the equivalent of _PyImport_GetModules(), but which increfs before the lock is released.

This is used by a later change related to pickle and handling __main__.
ericsnowcurrently added a commit that referenced this issue Apr 28, 2025
@ericsnowcurrently
gh-132775: Add _PyBytes_GetXIData() (gh-133101)
606003f 
This is the base for several other XIData wrappers, like pickle and marshal. It is essentially a refactor of the existing bytes XIData code.
This was referenced Apr 28, 2025
Merged
Merged
ericsnowcurrently added a commit that referenced this issue Apr 28, 2025
@ericsnowcurrently
gh-132775: Add _PyMarshal_GetXIData() (gh-133108)
bdd23c0 
Note that the bulk of this change is tests.
ericsnowcurrently added a commit that referenced this issue Apr 29, 2025
@ericsnowcurrently
gh-132775: Add _PyCode_ReturnsOnlyNone() (gh-132981)
96a7fb9 
The function indicates whether or not the function has a return statement.

This is used by a later change related treating some functions like scripts.
@bedevere-app bedevere-app bot mentioned this issue Apr 29, 2025
Merged
@markshannon
Copy link
Member

Could you elaborate what the "minor issues" are?
Either as a task list or sub issues?

It isn't clear to me why, for example, #132981 is needed, or if it is the right solution to whatever the problem is.

ericsnowcurrently added a commit that referenced this issue Apr 30, 2025
@ericsnowcurrently
gh-132775: Add _PyCode_GetVarCounts() (gh-133128)
94b4fcd 
This helper is useful in a variety of ways, including in demonstrating how the different counts relate to one another.

It will be used in a later change to help identify if a function is "stateless", meaning it doesn't have any free vars or globals.

Note that a majority of this change is tests.
@bedevere-app bedevere-app bot mentioned this issue Apr 30, 2025
Merged
ericsnowcurrently added a commit that referenced this issue Apr 30, 2025
@ericsnowcurrently
gh-132775: Add _PyPickle_GetXIData() (gh-133107)
cb35c11 
There's some extra complexity due to making sure we we get things right when handling functions and classes defined in the __main__ module.  This is also reflected in the tests, including the addition of extra functions in test.support.import_helper.
ericsnowcurrently added a commit to ericsnowcurrently/cpython that referenced this issue May 1, 2025
@ericsnowcurrently
Revert "pythongh-132775: Add _PyCode_GetVarCounts() (pythongh-133128)"
1740ecb 
This reverts commit 94b4fcd.
ericsnowcurrently added a commit that referenced this issue May 1, 2025
@ericsnowcurrently
Revert "gh-132775: Add _PyCode_GetVarCounts() (gh-133128)" (gh-133232)
811edcf 
The change broke the s390 builds, so I'm reverting it while I investigate.

This reverts commit 94b4fcd.
ericsnowcurrently added a commit to ericsnowcurrently/cpython that referenced this issue May 1, 2025
@ericsnowcurrently
pythongh-132775: Unrevert "Add _PyCode_GetVarCounts()"
d9c3672 
This reverts commit 811edcf (pythongh-133128).
@bedevere-app bedevere-app bot mentioned this issue May 1, 2025
Merged
ericsnowcurrently added a commit that referenced this issue May 5, 2025
@ericsnowcurrently
gh-132775: Add _PyCode_VerifyStateless() (gh-133221)
d270bb5 
"Stateless" code is a function or code object which does not rely on external state or internal state.
It may rely on arguments and builtins, but not globals or a closure. I've left a comment in
pycore_code.h that provides more detail.

We also add _PyFunction_VerifyStateless(). The new functions will be used in several later changes
that facilitate "sharing" functions and code objects between interpreters.
This was referenced May 5, 2025
Closed
Merged
ericsnowcurrently added a commit that referenced this issue May 5, 2025
@ericsnowcurrently
gh-132775: Do Not Set __name__ to __main__ With _PyPickle_GetXIData() (…
e961611 
…gh-133472)

This is a follow-up to gh-133107.  I realized that we could end up with an
infinite recursion if we try to run a function from __main__ in a subinterpreter.
ericsnowcurrently added a commit that referenced this issue May 5, 2025
@ericsnowcurrently
gh-132775: Add _PyCode_GetXIData() (gh-133475)
ea59873
This was referenced May 5, 2025
Merged
Merged
Open
Draft
Draft
@ericsnowcurrently ericsnowcurrently changed the title Minor InterpreterPoolExecutor Issues Improvements to test.support.interpreters.Interpreter May 6, 2025
@ericsnowcurrently ericsnowcurrently added type-feature A feature request or enhancement and removed type-bug An unexpected behavior, bug, or error labels May 6, 2025
@ericsnowcurrently ericsnowcurrently mentioned this issue May 6, 2025
Open
encukou added a commit to encukou/cpython that referenced this issue May 6, 2025
@encukou
Revert "pythongh-132775: Add _PyCode_VerifyStateless() (pythongh-133221…
e7d7e26 
…)"

This reverts commit d270bb5.

Note that the test functions in _code_definitions are not reverted.
@bedevere-app bedevere-app bot mentioned this issue May 6, 2025
Merged
hugovk pushed a commit that referenced this issue May 6, 2025
@encukou
gh-132775: Revert "gh-132775: Add _PyCode_VerifyStateless() (gh-133221)…
3c73cf5 
…" (#133497)
ericsnowcurrently added a commit to ericsnowcurrently/cpython that referenced this issue May 6, 2025
@ericsnowcurrently
Revert "pythongh-132775: Revert "pythongh-132775: Add _PyCode_VerifyS…
eab4c13 
…tateless() (pythongh-133221)" (python#133497)"

This reverts commit 3c73cf5.
This was referenced May 6, 2025
Merged
Merged
ericsnowcurrently added a commit that referenced this issue May 8, 2025
@ericsnowcurrently
gh-132775: Unrevert "Add _PyCode_VerifyStateless()" (gh-133528)
27128e4 
This reverts commit 3c73cf5 (gh-133497), which itself reverted
the original commit d270bb5 (gh-133221).

We reverted the original change due to failing android tests.
The checks in _PyCode_CheckNoInternalState() were too strict,
so we've relaxed them.
ericsnowcurrently added a commit that referenced this issue May 8, 2025
@ericsnowcurrently
[3.14] gh-132775: Unrevert "Add _PyCode_VerifyStateless()" (gh-133625)
c39bc81 
This reverts commit 3c73cf5 (gh-133497), which itself reverted
the original commit d270bb5 (gh-133221).

We reverted the original change due to failing android tests.
The checks in _PyCode_CheckNoInternalState() were too strict,
so we've relaxed them.
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 8, 2025
@ericsnowcurrently @miss-islington
pythongh-132775: Add _PyCode_GetScriptXIData() (pythongh-133480)
0a920cd 
This converts functions, code, str, bytes, bytearray, and memoryview objects to PyCodeObject,
and ensure that the object looks like a script.  That means no args, no return, and no closure.
_PyCode_GetPureScriptXIData() takes it a step further and ensures there are no globals.

We also add _PyObject_SupportedAsScript() to the internal C-API.
(cherry picked from commit c81fa2b)

Co-authored-by: Eric Snow <[email protected]>
@bedevere-app bedevere-app bot mentioned this issue May 8, 2025
Merged
ChVeen pushed a commit to ChVeen/cpython that referenced this issue May 8, 2025
@ericsnowcurrently
pythongh-132775: Add _PyCode_GetScriptXIData() (pythongh-133480)
c81fa2b 
This converts functions, code, str, bytes, bytearray, and memoryview objects to PyCodeObject,
and ensure that the object looks like a script.  That means no args, no return, and no closure.
_PyCode_GetPureScriptXIData() takes it a step further and ensures there are no globals.

We also add _PyObject_SupportedAsScript() to the internal C-API.
ericsnowcurrently added a commit that referenced this issue May 8, 2025
@miss-islington @ericsnowcurrently
[3.14] gh-132775: Add _PyCode_GetScriptXIData() (gh-133676)
1059548 
This converts functions, code, str, bytes, bytearray, and memoryview objects to PyCodeObject,
and ensure that the object looks like a script.  That means no args, no return, and no closure.
_PyCode_GetPureScriptXIData() takes it a step further and ensures there are no globals.

We also add _PyObject_SupportedAsScript() to the internal C-API.

(cherry picked from commit c81fa2b, AKA gh-133480)

Co-authored-by: Eric Snow <[email protected]>
ericsnowcurrently added a commit that referenced this issue May 12, 2025
@ericsnowcurrently
gh-132775: Add _PyFunction_GetXIData() (gh-133481)
8cf4947
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 12, 2025
@ericsnowcurrently @miss-islington
pythongh-132775: Add _PyFunction_GetXIData() (pythongh-133481)
36ef950 
(cherry picked from commit 8cf4947)

Co-authored-by: Eric Snow <[email protected]>
@bedevere-app bedevere-app bot mentioned this issue May 12, 2025
Merged
ericsnowcurrently added a commit that referenced this issue May 12, 2025
@miss-islington @ericsnowcurrently
[3.14] gh-132775: Add _PyFunction_GetXIData() (gh-133955)
3467656 
(cherry picked from commit 8cf4947, AKA gh-133481)

Co-authored-by: Eric Snow <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ericsnowcurrently ericsnowcurrently

Labels
3.14 bugs and security fixes stdlib Python modules in the Lib dir topic-subinterpreters type-feature A feature request or enhancement
Projects
Subinterpreters
Status: Todo
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ericsnowcurrently @markshannon