Semantic tokens support

[michaelpj]

This adds support for semantic tokens. Concretely, that means:

• A bevy of new types, in SemanticTokens.hs.
• Four new methods.
• A number of utility functions for working with semantic tokens, based
  on what I expect any user of this functionality would need
  (absolute/relative token positions, encoding tokens into arrays, and
  producing deltas)

---

Notes:

• I have no idea what I'm doing with the method stuff, I just kept going until I ran out of compiler errors, hope I got everything.
• The string/number enums are extremely tedious to write: you have to list all the constructors multiple times for the instances etc. Maybe worth some TH at some point.
• Possibly the semantic tokens utilities should live somewhere else, but I'm not sure where. Various other modules in lsp-types seem to have utility functions in them, so it doesn't seem wildly out of place.
• The performance of the diffing algorithm isn't great (see the test). I'm not sure if this is a problem in practice, but we can probably put something higher performance in there later if people actually use it.

[banacorn]

This looks great, can't wait to give it a try!

[banacorn]

I think both SemanticTokenAbsolute and SemanticTokenRelative need to have their record fields renamed (or prefixed).

I get clashes when I have length from Prelude in my code, and I can't seem to resolve it with those record field extension pragmas.

Edit: @wz1000 suggests that they should be imported qualified

[banacorn]

Question: do I need to turn anything "on" in the server configs to enable this? Because I can't trigger any of my semantic highlighting handlers:

• server side:
  • I have all four methods as static handlers
• client side (vscode):
  • I have editor.semanticHighlighting.enabled: true
  • I have a working TextMate grammar already

[wz1000]

@banacorn Language.LSP.Types is meant to be imported qualified, we don't use prefixes for any of the fields unless they are haskell keywords ("data" -> "xdata", "type" -> "xtype").

[michaelpj]

It's possible that I failed to hook something up properly. I should write a test that actually sends some requests and responses!

[jneira]

@michaelpj i am afraid that this needs a rebase (plus the test you commented)

[michaelpj]

Thanks for the ping, I'm just not succeeding in spending much time on OSS recently! I'll try and get it polished up soon.

[michaelpj]

Okay, pushed an update. Has some more tests, but could do with even more. Things that I'd appreciate eyes on:

• Have I changed all the right places? I found Call hierarchy support #332 very helpful, since I learned e.g. about Registration, which I had missed!
• There are some fiddly utilities for encoding tokens, it would be good if someone could check them against the spec.
• The diffing is still bad. I'm inclined to not care for now.
• Actually using it! @banacorn it sounded like you'd actually tried using this in something? Is that available somewhere so I could try it?

[michaelpj]

I have no idea what's up with the Windows CI, cabal is having some incomprehensible difficulties and trying to pick a version of some that it's constrained not to do by the cabal.project. And someone just did that on master so I'm not sure why it's happening. If anyone with a Windows machine could try and reproduce that would be helpful, since I can't reproduce it without just running the CI lots of times...

[jneira]

Trying the build locally 🤞

[jneira]

• removing constraints from cabal.project worked for me locally

[michaelpj]

Thanks!

[jneira]

@michaelpj the pr with support for ghc-9.0.1 does not have the problem with some, so i merged it (it was pending since several days ago) and i will rebase yours, if you dont mind

[michaelpj]

Sure. I accidentally reverted your constraint removal when I got rid of that extra commit :) Feel free to rebase, or I will tomorrow.

[michaelpj]

Okay, CI is passing now.

[jneira]

lgtm, thanks for tests and comments
will merge soon if nobody disagree