Skip to content

"python3 -m pegen python" got broken after v3.14.0a7 #133555

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
alexprengere opened this issue May 7, 2025 · 1 comment
Closed

"python3 -m pegen python" got broken after v3.14.0a7 #133555

alexprengere opened this issue May 7, 2025 · 1 comment
Labels
3.14 bugs and security fixes build The build process and cross-build topic-parser type-bug An unexpected behavior, bug, or error

Comments

@alexprengere
Copy link
Contributor

alexprengere commented May 7, 2025

Bug report

Bug description:

I am currently experimenting with the cpython parser/tokenizer (I want to propose a new feature at some point, but I want to come up with an implementation as well).
Following the parser documentation, from the current main (3dfed23), I can generate a C parser from the Tools/peg_generator directory:

$ python3.13 -m pegen c  ../../Grammar/python.gram  ../../Grammar/Tokens
# parse.c generated, works fine

But the Python version is not working:

$ python3.13 -m pegen python ../../Grammar/python.gram  
pegen.grammar.GrammarError: Dangling reference to rule 'TSTRING_START'
For full traceback, use -v

On v3.14.0a7 it works fine, so I guess this changed when PEP750 implementation was merged.
After investigating a bit, it looks like this is caused by using Python3.13 to run this, when I should use the version compiled from main (that includes the new tokens).
For f-strings, a workaround was merged in #125588 to add the missing tokens in parser_generator.py, so if that is OK I will open a PR to do the same for t-strings.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

@alexprengere alexprengere added the type-bug An unexpected behavior, bug, or error label May 7, 2025
alexprengere added a commit to alexprengere/cpython that referenced this issue May 7, 2025
@picnixz picnixz added 3.14 bugs and security fixes build The build process and cross-build labels May 7, 2025
alexprengere added a commit to alexprengere/cpython that referenced this issue May 7, 2025
alexprengere added a commit to alexprengere/cpython that referenced this issue May 7, 2025
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 8, 2025
…thonGH-133557)

(cherry picked from commit b48599b)

Co-authored-by: Alex Prengère <[email protected]>
Co-authored-by: Adam Turner <[email protected]>
AA-Turner added a commit that referenced this issue May 8, 2025
…H-133557) (#133630)

gh-133555: Allow regenerating the parser with Python < 3.14 (GH-133557)
(cherry picked from commit b48599b)

Co-authored-by: Alex Prengère <[email protected]>
Co-authored-by: Adam Turner <[email protected]>
@alexprengere
Copy link
Contributor Author

Fixed by #133557, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.14 bugs and security fixes build The build process and cross-build topic-parser type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

3 participants