Skip to content

Commit 24a7c29

Browse files
authored
[3.9] bpo-42123: Run the parser two times and only enable invalid rules on the second run (GH-22111) (GH-23011)
* Implement running the parser a second time for the errors messages The first parser run is only responsible for detecting whether there is a `SyntaxError` or not. If there isn't the AST gets returned. Otherwise, the parser is run a second time with all the `invalid_*` rules enabled so that all the customized error messages get produced. (cherry picked from commit bca7014)
1 parent c4b58ce commit 24a7c29

File tree

6 files changed

+71
-51
lines changed

6 files changed

+71
-51
lines changed

Grammar/python.gram

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -535,7 +535,7 @@ yield_expr[expr_ty]:
535535

536536
arguments[expr_ty] (memo):
537537
| a=args [','] &')' { a }
538-
| incorrect_arguments
538+
| invalid_arguments
539539
args[expr_ty]:
540540
| a=','.(starred_expression | named_expression !'=')+ b=[',' k=kwargs {k}] { _PyPegen_collect_call_seqs(p, a, b, EXTRA) }
541541
| a=kwargs { _Py_Call(_PyPegen_dummy_name(p),
@@ -620,7 +620,7 @@ t_atom[expr_ty]:
620620

621621

622622
# From here on, there are rules for invalid syntax with specialised error messages
623-
incorrect_arguments:
623+
invalid_arguments:
624624
| args ',' '*' { RAISE_SYNTAX_ERROR("iterable argument unpacking follows keyword argument unpacking") }
625625
| a=expression for_if_clauses ',' [args | expression for_if_clauses] {
626626
RAISE_SYNTAX_ERROR_KNOWN_LOCATION(a, "Generator expression must be parenthesized") }
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
Run the parser two times. On the first run, disable all the rules that only
2+
generate better error messages to gain performance. If there's a parse
3+
failure, run the parser a second time with those enabled.

0 commit comments

Comments
 (0)