[ASTGen] Add experimental feature to use ASTGen in lieu of parsing types #66033
Conversation
DougGregor
requested review from
zoecarver,
CodaFi,
ahoppen,
bnbarham,
rintaro,
hborla,
slavapestov,
xedin and
tshortli
as code owners
|
@CodaFi you're gonna like this... |
|
@swift-ci please smoke test |
| /// is NULL, something went wrong. | ||
| template<typename T> | ||
| using ASTFromSyntaxTreeCallback = std::pair<T*, const void *>( | ||
| void *sourceFile, const void *sourceLoc |
There was a problem hiding this comment.
We really need to figure out a better way to pass things around than using void *.
There was a problem hiding this comment.
Yes, this is pretty dreadful. Typedefs would improve on things a little bit. @bnbarham keeps threatening to add them
There was a problem hiding this comment.
#66044 for the common types (though this would be the first SourceFile I think?). Decls/exprs/typerefs/etc are a little more difficult since the visitor just returns a ASTNode for each visit at the moment. Even disregarding that I'm not sure we want to bridge every single AST node type that we need.
But it would be nice to do something to avoid:
- 682ac98 - forgot
.rawValue - https://github.com/apple/swift/pull/66044/files#diff-05684b1c93b20e9444846f61a3575d650a447713a129b477ecf5a518828d17e9L187 -
namewas not aBridgedIdentifierbut rather aUnresolvedDeclRefExpr. Finding that involves looking at the visit forVariableDeclSyntax-> visits itspattern-> visitsIdentifierPatternSyntax-> callsSwiftIdentifierExpr_create-> creates aUnresolvedDeclRefExpr.
| sourceFilePtr: UnsafeRawPointer, | ||
| sourceLocationPtr: UnsafePointer<UInt8>?, | ||
| type: Node.Type | ||
| type: Node.Type, | ||
| wantOutermost: Bool = false |
There was a problem hiding this comment.
Could you add a comment what, wantOutermost does? Also, I think it would also be good to document the other parameters.
There was a problem hiding this comment.
Yes, I will. I should really move this API somewhere else (e.g., on ExportedSourceFile) because it's in an odd place now that I made it non-private.
|
@swift-ci please smoke test |
1 similar comment
|
@swift-ci please smoke test |
Introduce a new experimental feature `ASTGenTypes` that uses ASTGen to translate the Swift syntax tree (produced by the new Swift parser) into C++ `TypeRepr` nodes instead of having the C++ parser create the nodes. The approach here is to intercept the C++ parser's `parseType` operation to find the Swift syntax node at the given position (where the lexer currently is) and have ASTGen translate that into the corresponding C++ AST node. Then, we spin the lexer forward to the token immediately following the end of the syntax node and continue parsing.
The pointers we were vending to C++ had low bits set because they were AST node entries rather than the raw TypeReprs.
There is a modeling difference between the swift-syntax tree and the C++ type representation (TypeRepr) that is a little odd here, so we end up parsing the ellipsis on the C++ side rather than looking "up" the syntax tree to find it.
Bridge the simple type attributes like `@autoclosure` into an `AttributedTypeRepr`. We don't validate the arguments, and we don't handle more complicated attributes like `@convention(c)` just yet.
Protocol composition types such as `P & Q` are similar to nested types like `A.B` because the innermost type syntax node at the given position doesn't cover the whole type. Adjust by looking back up the tree. This all feels like a hack, and there should be a better way. While here, fix the source ranges passed in to create `CompositionTypeRepr`. We were tripping assertions due to missing source-location information.
Instead of "spinning" the C++ lexer, consuming tokens uptil we get past the point where ASTGen told us the end of the syntax node was, just reset the lexer to exactly that position. This is more efficient, but also fixes problems where we would end up skipping past a `>` that had been split.
DougGregor
force-pushed
the
parse-types-via-astgen
branch
from
9dcfbf2 to
7224b81
Compare
|
@swift-ci please smoke test |
|
I'm going to merge this and we'll iterate to get it into a nicer form before writing more parsing logic |
Introduce a new experimental feature
ASTGenTypesthat uses ASTGen to translate the Swift syntax tree (produced by the new Swift parser) into C++TypeReprnodes instead of having the C++ parser create the nodes.The approach here is to intercept the C++ parser's
parseTypeoperation to find the Swift syntax node at the given position (where the lexer currently is) and have ASTGen translate that into the corresponding C++ AST node. Then, we spin the lexer forward to the token immediately following the end of the syntax node and continue parsing.Introduce a collection of small fixes to the mapping of type syntax nodes into
TypeReprs:inout,consuming, andisolatedAnyand treat it as the empty composition type_identifier as the empty identifierWith all of these changes, a hacked compiler that enables this experimental flag all the time successfully handles 236 of 245 tests in
test/Sema.