Skip to content

"python3 -m pegen python" got broken after v3.14.0a7 #133555

Closed
@alexprengere

Description

@alexprengere

Bug report

Bug description:

I am currently experimenting with the cpython parser/tokenizer (I want to propose a new feature at some point, but I want to come up with an implementation as well).
Following the parser documentation, from the current main (3dfed23), I can generate a C parser from the Tools/peg_generator directory:

$ python3.13 -m pegen c  ../../Grammar/python.gram  ../../Grammar/Tokens
# parse.c generated, works fine

But the Python version is not working:

$ python3.13 -m pegen python ../../Grammar/python.gram  
pegen.grammar.GrammarError: Dangling reference to rule 'TSTRING_START'
For full traceback, use -v

On v3.14.0a7 it works fine, so I guess this changed when PEP750 implementation was merged.
After investigating a bit, it looks like this is caused by using Python3.13 to run this, when I should use the version compiled from main (that includes the new tokens).
For f-strings, a workaround was merged in #125588 to add the missing tokens in parser_generator.py, so if that is OK I will open a PR to do the same for t-strings.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.14bugs and security fixesbuildThe build process and cross-buildtopic-parsertype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions