isIdentifierText in scanner.ts doesn't match Identifier parsing anymore

[ajafff]

TypeScript Version: 3.6.0-dev.20190804

Search Terms:

Expected behavior:

isIdentifierText should match what scan does.

Actual behavior:

The implementation was not updated to allow for astral symbols and escaped characters. This might cause some inconsistencies, for example when using an astral character in compilerOptions.reactNamespace.

Related Issues:
#32096, #32701, cc @weswigham

[weswigham]

It's already up to date (and had to be to parse the example in the PR)? isIdentifierText uses isIdentifierStart and isIdentifierPart which in turn use isUnicodeIdentifierStart and isUnicodeIdentifierPart which in turn use unicodeESNextIdentifierStart and unicodeESNextIdentifierPart which were added in that PR expressly to support the higher unicode codepoints that are now allows as identifier bits.

[ajafff]

But it still uses charCodeAt instead of codePointAt and advances the index by 1 instead of charSize(ch)

[weswigham]

Oh, fair. As-is, I suppose it's just relying on the later half of an extended code point maybe being a valid part on its own. Anyone know of a unicode character which is a valid identifier part but whose lower 8 bits, interpreted as a character, are not (to write the test with)?

[weswigham]

Found one~