SemanticTokens stops working after a switch from Parallel to Serial cancels a pending request

[jharjung]

I'm not sure whether the problem lies with the client or the server (or me?), but I'm observing the following behavior using VSCode 1.65.2 as the editor:

1. Open a document in my language and type out some code
2. Delete the new code and type it again
3. Repeat a couple times

It doesn't have to be a type-delete-repeat cycle, that's just an example. Typing out some text in a string literal does it too. Any amount of normal paced keyboard typing will do it.

I've got tracing turned on, on both the VSCode side and on the server side, so I can see the requests and responses going back and forth. There are several rapid fire didChange requests while typing, intermixed with occasional semanticTokens requests - and I've got a CompletionHandler, so there are a few of those requests too. Each didChange appears to swap the scheduler from Parallel to Serial, and each semanticTokens or completion swaps it back from Serial to Parallel.

Everything works fine for a bit, but eventually we hit this scenario:

### Server
OmniSharp.Extensions.LanguageServer.Shared.LspRequestRouter: Debug: Finding descriptors for textDocument/semanticTokens/full
OmniSharp.Extensions.JsonRpc.ProcessScheduler: Debug: Swapping from Serial to Parallel
OmniSharp.Extensions.JsonRpc.ProcessScheduler: Debug: Completing existing request process type Serial
OmniSharp.Extensions.JsonRpc.ProcessScheduler: Debug: Queueing Parallel:textDocument/semanticTokens/full:75 request for processing

OmniSharp.Extensions.LanguageServer.Shared.LspRequestRouter: Debug: Finding descriptors for textDocument/didChange
OmniSharp.Extensions.JsonRpc.ProcessScheduler: Debug: Swapping from Parallel to Serial
OmniSharp.Extensions.JsonRpc.ProcessScheduler: Debug: Cancelling any outstanding requests (switch from parallel to serial)
OmniSharp.Extensions.JsonRpc.ProcessScheduler: Debug: Completing existing request process type Parallel
OmniSharp.Extensions.JsonRpc.ProcessScheduler: Debug: Queueing Serial:textDocument/didChange request for processing

### Client
[Trace - 9:46:25 AM] Sending request 'textDocument/semanticTokens/full - (44)'.
[Trace - 9:46:25 AM] Sending notification 'textDocument/didChange'.
[Trace - 9:46:25 AM] Received request 'window/workDoneProgress/create - (81)'.
[Trace - 9:46:25 AM] Sending response 'window/workDoneProgress/create - (81)'. Processing request took 0ms
[Trace - 9:46:25 AM] Received notification '$/progress'.
[Trace - 9:46:25 AM] Received notification 'textDocument/publishDiagnostics'.
[Trace - 9:46:25 AM] Received notification '$/progress'.
[Trace - 9:46:25 AM] Sending notification 'textDocument/didChange'.

There's a sematicTokens request in the queue, when a didChange request comes in and cancels it. There is no corresponding Received response 'textDocument/semanticTokens/full - (44)' line, and no error, in the client trace log. There is no error in the server trace log. There is nothing to indicate the server informing the client that the request was cancelled. The request is just abandoned.

After that, the client will never send another semanticTokens request for that document. The didChange and completion requests continue to come in, as well as requests for my other handlers (hover, signatureHelp, etc.), but never another semanticTokens request.

In all of the successful sequences prior to semanticTokens 75, the most recent semanticTokens request had either completed or not yet been queued prior to the swap from Parallel to Serial.

If I open a second document, I do get semanticTokens requests for that one (until typing in that document eventually gets it into the same situation).

It takes a bit of keyboard typing to provoke this, but it's not rare at all. I would say on average it takes between 10 and 30 seconds of typing.

These are the package versions I'm using (latest release as of 3/20/2022):

• OmniSharp.Extensions.LanguageServer: 0.19.5
• vscode-languageclient: 7.0.0
• vscode-jsonrpc: 6.0.0

I've also tried it on these prerelease versions, with the same outcome:

• vscode-languageclient: 8.0.0-next.13
• vscode-jsonrpc: 8.0.0-next.7

There's a 8.0.0-next.14 out for vscode-languageclient now, but it requires VS Code 1.66 which hasn't been released, so I haven't tried that one.

I've tried this with both Full and Incremental text document sync, and with both Full and Delta semantic tokens. I see the same behavior in all combinations.

[jharjung]

Found this in docs/jsonrpc.md:

• options.WithSupportsContentModified()
  • Sets the into a special model that is used more for Language Server Protocol.
  • In this mode any Serial request will cause any outstanding Parallel requests will be cancelled with an error message.

Sure enough, setting options.WithContentModifiedSupport(false) prevents the request from getting cancelled, which allows SemanticTokens to continue functioning as it should. I'm unable to provoke the observed breaking behavior with that flag set to false.

[zreedy]

I also observe this behavior, is there a reason that this flag is not set by default? The documentation seems to indicate that this flag is intended to be set to true in OmniSharp. Additionally, what conditions constitute a ContentModifiedException? I do not believe I've observed this being thrown from anywhere other than a SemanticTokens request. The source code regarding this exception is rather impenetrable; best I can figure is that it's related to serial vs parallel processing of JSON RPC commands?