Commit 87c3936
committed
Tokenizer/PHP: efficiency fix
Reminder: the PHP::processAdditional()` method walks _back_ from the end of the file to the beginning.
The type handling retokenization layer is triggered for each `&`, `|` and `)` the tokenizer encounters.
When something is recognized as a valid type declaration, the relevant tokens will all be retokenized in one go the first time the type handling layer is triggered, which means that - as the type tokens will have been retokenized already -, the type handling layer will not be triggered again for any of the other type related tokens in the type.
However, if the type is *not* recognized as a valid type, the type handling layer will keep getting retriggered and will (correctly) keep concluding this is not a valid type.
The change in this PR, prevents the type handling layer from doing any work when it is retriggered on a token which was previously already seen and concluded to be, either not part of a type or part of an invalid type.
This should make the tokenizer marginally faster for complex types containing an error, like `(A&B|(C&D)|(E&F)`.1 parent 07477a7 commit 87c3936
1 file changed
+12
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2602 | 2602 | | |
2603 | 2603 | | |
2604 | 2604 | | |
2605 | | - | |
| 2605 | + | |
| 2606 | + | |
| 2607 | + | |
2606 | 2608 | | |
2607 | 2609 | | |
2608 | 2610 | | |
| |||
3038 | 3040 | | |
3039 | 3041 | | |
3040 | 3042 | | |
| 3043 | + | |
| 3044 | + | |
| 3045 | + | |
| 3046 | + | |
| 3047 | + | |
| 3048 | + | |
3041 | 3049 | | |
3042 | 3050 | | |
3043 | 3051 | | |
| |||
3250 | 3258 | | |
3251 | 3259 | | |
3252 | 3260 | | |
| 3261 | + | |
| 3262 | + | |
| 3263 | + | |
3253 | 3264 | | |
3254 | 3265 | | |
3255 | 3266 | | |
| |||
0 commit comments