A syntax definition may trigger text manipulation

[deathaxe]

Description of the bug

A major issue with Vue Syntax Highlight 4.0.1 has been observed, which causes the following text snippet

a) to be pasted with line 7 ('error': status === 'error') being wrongly indented
b) and "\n in line 8 being deleted when re-indenting line 7 by hitting tab at the beginning of the line.

<template>
  <figure class="image">
    <img
      :src="image"
      :class="{
        'waiting': status === 'waiting',
        'error': status === 'error'
      }"
    >
  </figure>
</template>

The result can not be undone via ctrl+z and may even cause ST to crash when saving the file if LSP-vue is present.

It appears embedded JavaScript syntax somehow causes syntax engine to get into a weird state, causing the text to get corrupted.

source: https://discord.com/channels/280102180189634562/280157083356233728/1070248043938590720

The good news: This bug is worked around by vuejs/vue-syntax-highlight#216

The bad news:

1. This fix has not been released, yet.
2. It may happen to other syntaxes as well, given JavaScript being embedded in HTML and various other syntaxes.

The one way or the other: This kind of weird text manipulation which also breaks the undo stack should not happen under all circumstances.

Steps to reproduce

1. Start ST in SAFE MODE
2. Extract Vue.zip to Data\Packages\Vue
3. Open a new view
4. Set syntax to Vue Component (Broken)
5. Paste the given template code
6. Navigate to the beginning of line 7
7. Hit tab
8. Observe "Actual behavior"
9. Open another view
10. Set syntax to Vue Component
11. Paste the given template code
12. Observe "Expected behavior"

Expected behavior

After pasting code

After hitting tab (even though not needed)

The commit which fixes the bug is vuejs/vue-syntax-highlight@78f5c47

The crucial part of the fix is to embed scope:source.js#expression-statement instead of scope:source.js.

Actual behavior

After pasting code

After hitting tab

... the closing " and the following newline are deleted.

ST's console prints:

command: insert {"characters": "\t"}

Sublime Text build number

4147

Operating system & version

Windows 11

(Linux) Desktop environment and/or window manager

No response

Additional information

No response

OpenGL context information

No response

[BenjaminSchaaf]

This was a pain to track down. The cause ended up being mundane: embed escapes are handled specially when lexing a line, but this wasn't done when backtracking to a previous line. So when it got to }"\n the " (the escape pattern) and consequently the \n were dropped, resulting in the backtracking being off by 2.

[FichteFoll]

Good news. I also ran into the bug for the first time yesterday and triggered a crash after trying to undo it (with LSP-volar active, though; not sure if that's a necessity).

[BenjaminSchaaf]

Fixed in build 4148.