Large numbers of <>'s in a file cause massive slowdown

[huonw]

E.g. it takes several seconds for a keypress to appear at the bottom of a file with 400 copies of fn foo<T>() {}. Editing the same file after s/<T>/_T_/ (to preserve the size) is fine.

[huonw]

I only noticed this recently, and after updating rust-mode, so maybe this was caused by #79. cc @MicahChalmer

[tomjakubowski]

I'd noticed a general huge slowdown in rust-mode (particularly for long-lived buffers), but I attributed it to this (it'd make sense for a GC problem to only manifest itself in long-lived buffers): #85

I haven't noticed any slowdown since updating to a version of rust-mode with that patch merged.

[tomjakubowski]

Blech. I just made a file of 400 fn foo<T>() {} (I initially read that as 40 angle bracket pairs) and, as you said, it's painfully slow. Sorry for the noise!

[MicahChalmer]

Yes, I can reproduce the issue. Setting rust-match-angle-brackets to nil for the buffer (which disables the matching introduced by #79) makes the slowness go away for me.

At least part of the problem seems to be that rust-font-lock-extend-region extends too far. If you scroll to the end of the 400-line example discussed here, every keypress calls rust-font-lock-extend-region with the syntactic fontification not applied all the way to the region it's trying to re-fontify, so it ends up telling it to do the whole buffer. That means it redoes the angle bracket detection algorithm on all 400 < and > pairs every time.

[xitep]

i can reproduce this massive slowdown as well. no brackets need to be in play. it just suffices to create a file in the style of:

// comment one
// comment two
// ... 500 more of these

fn main () {
  // editing becomes slow :(
}

// another comment one
// ... the more the slower

it looks like the while loop in rust-conditional-re-search-forward is the culprit.

[MicahChalmer]

@xitep that surprises me, since there's nothing in those comments that should be found by any of the rust-conditional-re-search-forward searches. I tried a list of comments and could not reproduce the slowdown--there have to be < or > characters.

If the problem was rust-conditional-re-search-forward then I still wouldn't expect just comments to cause a problem. But I would expect to see the issue in raw strings, since they use that function as well. But I don't see that either. This is what I see when I try to reproduce the issue:

Repeat // comment one 500 times: does not slow down
Repeat fn foo() { r##"raw string"## } 500 times: does not slow down
Repeat fn foo<T>() {} 500 times: slows down, but speeds back up if rust-match-angle-brackets is set to nil

[xitep]

@MicahChalmer indeed, obviously there was something really wrong with my test before. sorry for that.

[nikomatsakis]

@MicahChalmer any thoughts here? this is definitely a problem; seems like it should be possible to cap the searchback in some way (at least temporarily) to alleviate this, no?

[MicahChalmer]

I've been trying to fix this when I have time.

There's definitely an issue with rust-font-lock-extend-region causing it to re-fontify too much over and over again, as I mentioned above. It seems like font-lock-syntactically-fontified can end up getting stuck and not advancing as the syntactic fontification proceeds to later points in the file. At this point I'm still unsure of the cause. But even if this was fixed, the initial fontification still pauses noticeably on what is not really that large of a file, so the actual searchback does probably need to be sped up as well.

One thought thought I had for trimming the searchback was to introduce a cache (similar in nature to syntax-ppss-cache) so that calls to rust-is-in-expression-context, rust-is-lt-char-operator can be memoized by position in the buffer. The cache would have to be cleared for all positions after the change. There's a whole lot of mechanics in the emacs code to make this work for syntax-ppss itself--hopefully it can be done in a simpler way here.

I'll put in a PR when I think I have a fix for it.

[nikomatsakis]

@MicahChalmer do you think we should revert the PR in the meantime? I guess I'd say it depends on how close you are to a fix.

[MicahChalmer]

Just submitted #89 to fix it.

I didn't have to do anything as complicated as the caching idea above (I actually did try it, and it didn't help the speed much.) Fortunately, just fixing up rust-font-lock-extend-region makes it much better.