x/tools/gopls: regression in semantic tokens (Data field is JSON "null")

[adonovan]

@neild reports that neovim with gopls at master produces an error similar to neovim/neovim#21387. I suspect https://go.dev/cl/577655 introduced a shortcut path with a nil Data slice. This patch appears to be an effective workaround:

diff --git a/gopls/internal/server/semantic.go b/gopls/internal/server/semantic.go
index ca3df78e10..7f7a3474d3 100644
--- a/gopls/internal/server/semantic.go
+++ b/gopls/internal/server/semantic.go
@@ -38,7 +38,7 @@ func (s *server) semanticTokens(ctx context.Context, td protocol.TextDocumentIde
                // Previously, an error was returned here to achieve the same effect, but
                // that had the side effect of very noisy "semantictokens are disabled"
                // logs on every keystroke.
-               return new(protocol.SemanticTokens), nil
+               return &protocol.SemanticTokens{Data: []uint32{}}, nil

[findleyr]

Thanks all for catching this (right before we were about to cut the prerelease, too!).

I suspect adding "omitempty" would also fix this, and may be more principled.

[adonovan]

Yes, the patch was just evidence. I think the cleanest solution would be to make sure that every slice in our JSON schema is marked omitempty.

[gopherbot]

Change https://go.dev/cl/591415 mentions this issue: gopls/internal/server: return a non-nil slice for empty token result

[findleyr]

It looks like omitempty may not be appropriate here, because the fields are not marked as optional in the schema.

For now, went with the fix suggested in Alan's patch: just return a non-nil empty slice.