Provide guidance how language servers should support Virtual Workspaces

[davidanthoff]

The new Virtual Workspace feature is great! Could you provide guidance how we might support it from our language server?

At the moment, our LSP implementation reads files from disc all the time. I've long hoped that instead of using direct file IO we could just do calls from the server to the client via LSP to read file content, and if that supported custom URI schemes, I guess this would work. But at the moment it is not clear to me how we could get our LS to support a virtual workspace.

[tamuratak]

Related to microsoft/language-server-protocol/issues/336

[jrieken]

I've long hoped that instead of using direct file IO we could just do calls from the server to the client via LSP to read file content,

The VSCode API has this since a while. There is vscode.workspace.fs which should allow you to do the basic file operations like read, write, stat, readdir, delete etc.

[davidanthoff]

Yes, I'm aware of the vscode.workspace.fs API. But I can only use that from within the extension, right? Not from within our language server implementation. Essentially I'm thinking that we need the equivalent of vscode.workspace.fs in the language server, i.e. a LSP version that has JSON RPC methods for exactly those kind of file IO operations?

[jrieken]

Not from within our language server implementation. Essentially I'm thinking that we need the equivalent of vscode.workspace.fs in the language server

Yes - only through some custom protocol messages it can be used

[davidanthoff]

Would it make sense to extend the LSP to include those kind of protocol messages, and have the LanguageClient class provide a default implementation on the extension side that uses vscode.workspace.fs? It seems that almost any language server implementation that wants to work in a virtual workspace would need this, right?

[dbaeumer]

There is work under way that will add FS support to LSP.

[heejaechang]

@dbaeumer I hope that includes fs access to outside of workspace root files (it is common libraries to be installed outside of workspace root) and file watcher support for those files as well. also, not all LSP server is asynchronous, so hopefully, those API support synchronous API as well like fs.

[dbaeumer]

@heejaechang what range of files the provider will support depends on the provider itself and is out of the control of the protocol or VS Code API. For example the file system provider for file supports files outside the workspace since it can read it. The vscode-vfs can only provide access to the files on GitHub for example and therefore under the workspace root. It can't support reading files outside the repository.

As all process to process API we can only make this async.

[jakebailey]

That's going to end up being quite problematic for Pylance/pyright, where we are deeply synchronous during the analysis. I'm wondering how TypeScript is able to handle this, as they have a very similar architecture and are unlikely to be able to rearchitect to be async any time soon.

(Further, our cancellation itself is based on FS accesses to deal with the lack of async message handling.)

[dbaeumer]

I do have a similar problem with ESLint and I solved it by yielding as much as possible. This can be done without making all the code async (ESLint itself was sync until version 7). Doing this allows for example for normal cancellation and the json-rpc libraries do buffer all messages internally to make this easier. We even do the yielding approach in VS Code itself. When we colorize a file we yield every n lines so that events and other things can be processed. In Pylance/pyright you might be able to easily yield after every file which would only make the top level code async.

I haven't looked at TypeScript since quite some time but I do know that they introduced the concept of workers to be able to yield more efficiently. But you need to talk to the TS people.