Add support for semantic functionality in macro expansion reference documents

[lokesh-tr]

Reimplementation of #1610 after #1631

This PR migrates several requests made to sourcekitd to support Semantic Functionality in Macro Expansion Reference Documents.

What has been migrated and works?

1. cursorInfo
2. documentDiagnosticReport
3. openInterface
4. semanticTokens
5. relatedIdentifiers (used by documentSymbolHighlight)
6. Nested Macro Expansions

What has been migrated but doesn’t work?

• inlayHints (variableTypeInfo) probably needs a fix in sourcekitd

What won't be migrated since they don't apply for reference documents?

• Rename (Symbol)
• Code Completion Session
• RefactoringResponse.refactoring (i.e. Semantic Refactor)

Note:

• A fix for the percent encoding issue will be done in the sourcekit-lsp PR and this PR will only work in conjunction with that.
• Test cases for semantic functionality will be available in a follow-up PR

Moreover, this PR also does the following:

1. Updates outdated doc comments from Support expansion of nested macros #1631
2. Fixes an issue where wrong file and position passed in PeekDocumentsRequest from Support expansion of nested macros #1631

---

Expansion of Swift Macros in Visual Studio Code - Google Summer Of Code 2024
@lokesh-tr @ahoppen @adam-fowler

[lokesh-tr]

Since test cases are passing for nested macro expansion. I would like to assume that the issue with semantic functionality not working is due to vs code / extension.

[lokesh-tr]

I will try to come up with test cases for semantic functionality.

[lokesh-tr]

@ahoppen This feature is complete now, I will work on the test cases.

It would be nice if you can merge this and I can work on the test cases in a separate PR.

Everything works now :)

[ahoppen]

This looks great! I think that the changes to MacroExpansionReferenceURLData aren’t needed if #1636 ends up fixing the URL encoding issue with VS Code.

[lokesh-tr]

@ahoppen ready for review

[ahoppen]

Looks great!

[ahoppen]

@swift-ci Please test

[ahoppen]

@swift-ci Please test Windows

[lokesh-tr]

@ahoppen what exactly causes the CI to fail? Is it due to the .enableUpcomingFeature("BareSlashRegexLiterals") ? What to do to make the CI to succeed?

[ahoppen]

#1638 should fix the Swift 5.10 build by just not using bare slash regex literals. That way we don’t need to update the package manifest. Could you undo your package manifest change? The PR should be fine in CI without it because it has a Swift 6 compiler with bare slash regex literals enabled.

[lokesh-tr]

@ahoppen I updated the PR as you said. Can you please trigger the CI once again?

[ahoppen]

@swift-ci Please test

[ahoppen]

@swift-ci Please test Windows