Segfault in Vec::clone during analysis pass

[sfackler]

After upgrading to the new nightly, RLS seems to be processing through crates in my dependency graph as usual, but then eventually gets stuck on some crate and allocates memory until OSX decides it has to hard reboot (!).

[sfackler]

There doesn't seem to be anything interesting in the debug logs, just job queue starts and ends:

...
 INFO 2018-05-16T17:48:38Z: cargo::core::compiler::job_queue: start: sorcery v0.19.0 (registry `ssh://github.palantir.build/rust-lang/internal-rust-release-index`) => Target(lib)/Profile(dev) => Host
DEBUG 2018-05-16T17:48:42Z: cargo::core::compiler::fingerprint: write fingerprint: /Volumes/git/epoch2/target/rls/debug/.fingerprint/sorcery-bcd574a7a55b5aad/lib-sorcery-bcd574a7a55b5aad
 INFO 2018-05-16T17:48:42Z: cargo::core::compiler::job_queue: end: sorcery v0.19.0 (registry `ssh://github.palantir.build/rust-lang/internal-rust-release-index`) => Target(lib)/Profile(dev) => Host
[Info  - 10:50:40 AM] Connection to server got closed. Server will restart.
DEBUG 2018-05-16T17:50:40Z: rls::server: Language Server starting up. Version: 0.127.0-nightly (d2ade31 2018-05-08)
DEBUG 2018-05-16T17:50:40Z: cargo::core::workspace: find_root - is root /Volumes/git/epoch2/Cargo.toml
...

sorcery is a direct dependency of the root binary of this workspace if that's relevant.

[sfackler]

This may not be a regression in the latest nightly, but rather a difference between having analysis available and not? I checked out older nightlies and they seem to have the problem, and I did clean out my target directory just before this happened.

[sfackler]

Looks like a segfault in Vec::clone:

* thread #17, stop reason = EXC_BAD_ACCESS (code=2, address=0x1231fffe0)
  * frame #0: 0x00007fff5af1f18e libsystem_platform.dylib`_platform_memmove$VARIANT$Haswell + 622
    frame #1: 0x000000010eadad62 libstd-28258e62c4d16cf7.dylib`_$LT$alloc..vec..Vec$LT$T$GT$$u20$as$u20$core..clone..Clone$GT$::clone::hecd4f9d76c928061 + 114
    frame #2: 0x000000010ead9585 libstd-28258e62c4d16cf7.dylib`_$LT$alloc..string..String$u20$as$u20$core..clone..Clone$GT$::clone::h726cabf4e3ca7044 + 21
    frame #3: 0x0000000109f4a42d rls`_$LT$core..option..Option$LT$$RF$$u27$a$u20$T$GT$$GT$::cloned::h4e177fd999085e3e + 253
    frame #4: 0x0000000109fe469f rls`_$LT$alloc..vec..Vec$LT$T$GT$$u20$as$u20$core..clone..Clone$GT$::clone::he615a451a9bdcc76 + 239
    frame #5: 0x0000000109ed7d20 rls`rls::build::rustc::rustc::h79865c83392a65b1 + 3616
    frame #6: 0x0000000109e9ad6a rls`_$LT$rls..build..cargo..RlsExecutor$u20$as$u20$cargo..core..compiler..Executor$GT$::exec::h46cf566d7a7d9aaa + 11626
    frame #7: 0x000000010a6dcfa1 rls`_$LT$F$u20$as$u20$cargo..core..compiler..job..FnBox$LT$A$C$$u20$R$GT$$GT$::call_box::hd9529c258378f15e + 3329
    frame #8: 0x000000010a641ffd rls`_$LT$F$u20$as$u20$cargo..core..compiler..job..FnBox$LT$A$C$$u20$R$GT$$GT$::call_box::hf5f8484153e75a14 + 61
    frame #9: 0x000000010a641ffd rls`_$LT$F$u20$as$u20$cargo..core..compiler..job..FnBox$LT$A$C$$u20$R$GT$$GT$::call_box::hf5f8484153e75a14 + 61
    frame #10: 0x000000010a6420bf rls`cargo::core::compiler::job::Job::run::h7c549aea76e47e55 + 31
    frame #11: 0x000000010a65072b rls`_$LT$F$u20$as$u20$crossbeam..FnBox$GT$::call_box::hfb9802c95d8f7c79 + 187
    frame #12: 0x000000010eaa504f libstd-28258e62c4d16cf7.dylib`__rust_maybe_catch_panic + 31
    frame #13: 0x000000010a858542 rls`_$LT$F$u20$as$u20$alloc..boxed..FnBox$LT$A$GT$$GT$::call_box::h2333c649ab02dba2 + 146
    frame #14: 0x000000010ea921c8 libstd-28258e62c4d16cf7.dylib`std::sys_common::thread::start_thread::h36843e6546b9b405 + 136
    frame #15: 0x000000010ea66649 libstd-28258e62c4d16cf7.dylib`std::sys::unix::thread::Thread::new::thread_start::hee8a38e1bb8b4b30 + 9
    frame #16: 0x00007fff5af25661 libsystem_pthread.dylib`_pthread_body + 340
    frame #17: 0x00007fff5af2550d libsystem_pthread.dylib`_pthread_start + 377
    frame #18: 0x00007fff5af24bf9 libsystem_pthread.dylib`thread_start + 13

[sfackler]

I built a local copy of RLS with debuginfo to try to get a better stacktrace, and things seem to be much happier with it - It's using 200MB of RAM rather than 8GB and finishes analysis just fine. Was something fixed between the nightly RLS and master?

[Geobert]

Using nightly-x86_64-pc-windows-msvc updated - rustc 1.27.0-nightly (f0fdaba04 2018-05-15) in vscode-rls

fatal runtime error: memory allocation failed
[Info  - 20:09:33] Connection to server got closed. Server will restart.
fatal runtime error: memory allocation failed
[Info  - 20:09:38] Connection to server got closed. Server will restart.
fatal runtime error: memory allocation failed
[Info  - 20:09:42] Connection to server got closed. Server will restart.
fatal runtime error: memory allocation failed
[Info  - 20:09:45] Connection to server got closed. Server will restart.
fatal runtime error: memory allocation failed
[Error - 20:09:49] Connection to server got closed. Server will not be restarted.

using master I got:

[Info  - 20:39:13] Connection to server got closed. Server will restart.
[Info  - 20:39:13] Connection to server got closed. Server will restart.
[Info  - 20:39:13] Connection to server got closed. Server will restart.
[Info  - 20:39:13] Connection to server got closed. Server will restart.
[Error - 20:39:13] Connection to server got closed. Server will not be restarted.

[olix0r]

I'm seeing this issue as well on nightly-2018-05-15 -- indexing happens until it just gets stuck on one crate, and then I see memory continue to grow (10s and 100s of GB) and CPU usage elevates (looks like a core gets pegged).

I reverted back to (arbitrarily) nightly-2018-05-08 and I no longer see this behavior.

[doxxx]

Same problem, reverting to nightly-2018-05-08 also solved it for me.

[whmountains]

It's still broken for me. Afterrustup install nightly-2018-05-08. I deleted the rls and target directories. Anything else I need to do?

[nrc]

Hopefully this should be fixed in today's nightly (2018-05-16).

[olix0r]

@nrc thank you! Confirmed that rls appears stable for me on:

  nightly-x86_64-apple-darwin updated - rustc 1.27.0-nightly (2f2a11dfc 2018-05-16)

@whmountains you may need to explicitly configure the channel to use. for example, in vscode:

    "rust-client.channel": "nightly-YYYY-MM-DD",

that said, plain old nightly works for me now

[sfackler]

Yep, things seem to be working again with the new nightly!