Syntax highlighting should be full-file only

[silverwind]

Description

Diff lines are passed individually line-by-line to chroma, which its a massive source of highlighting bugs because the highlighter needs to see the whole file to highlight correctly. We should instead:

• Highlight both the old and new files individually, store the resulting HTML in Maps of (lineText, lineHTML).
• For each content line in the diff, look into the maps and if there is an exact match, replace the line with the highlighted HTML line.

This assumes chroma does not output HTML tags that span multiple lines, but I think this may already be the case.

[lunny]

Is it possible to highlight a specific part of a file if we know the file type? Since markdown allows highlighting within embedded code sections, this could be an effective way to focus on relevant content. Instead of rendering the entire file, we could adopt a similar approach to diff views, where only a portion of the file is displayed by default.

[silverwind]

Is it possible to highlight a specific part of a file if we know the file type

No, chroma assumes it is being passed a full file (basically all syntax highlighters work that way). Any random section of a file is not guaranteed to highlight correctly, only the whole file content is.

[lunny]

Is it possible to highlight a specific part of a file if we know the file type

No, chroma assumes it is being passed a full file (basically all syntax highlighter work that way). Any random section of a file is not guaranteed to highlight correctly, only the whole file content is.

So do you mean all code sections embedded in the issue markdown are not guaranteed to highlight correctly?

[silverwind]

You mean those inline code blocks that render when linking to code in discussions? They are also affected. They need to pass the whole content for highlighting purpose and then only render a subsection of the highlighted HTML. This kind of rendering is best-effort thought because if the snippet is incomplete code, there will be highlighting bugs.

Basically our interface with chroma should be to only ever pass full files to it, and then reduce the returned highlighted lines to what we need to display.

[wxiaoguang]

A related PR but stale: fix highlight problem of git diff #24336

[warasilapm]

@silverwind I noticed that multi-line block comments in my C code were not highlighting correctly.

Any chance this is a symptom of this issue? I missed the linked issue #23176. Probably just another case of this.

[silverwind]

Yes that is exactly what this issue is about. 90% of highlighting bugs are this issue, the remaining 10% are bugs in the chroma dependency 😉.

[warasilapm]

Does this issue need a champion? I'm not particularly handy with go but I could find some time or see if I can nerdsnipe a friend who is. Thoughts on using #24336 as a starting point?

[silverwind]

I don't think anyone is working on this currently, so feel free to. #24336 seems to have the right idea but from a quick glance, I'm concerned that it did not remove any line-by-line diff code. Imho, all such code needs to be replaced.