Skip to content

std: Implement TLS for wasm32-unknown-unknown #54951

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 13, 2018

Conversation

alexcrichton
Copy link
Member

This adds an implementation of thread local storage for the
wasm32-unknown-unknown target when the atomics feature is
implemented. This, however, comes with a notable caveat of that it
requires a new feature of the standard library, wasm-bindgen-threads,
to be enabled.

Thread local storage for wasm (when atomics are enabled and there's
actually more than one thread) is powered by the assumption that an
external entity can fill in some information for us. It's not currently
clear who will fill in this information nor whose responsibility it
should be long-term. In the meantime there's a strategy being gamed out
in the wasm-bindgen project specifically, and the hope is that we can
continue to test and iterate on the standard library without committing
to a particular strategy yet.

As to the details of wasm-bindgen's strategy, LLVM doesn't currently
have the ability to emit custom global values (thread locals in a
WebAssembly.Module) so we leverage the wasm-bindgen CLI tool to do
it for us. To that end we have a few intrinsics, assuming two global values:

  • __wbindgen_current_id - gets the current thread id as a 32-bit
    integer. It's wasm-bindgen's responsibility to initialize this
    per-thread and then inform libstd of the id. Currently wasm-bindgen
    performs this initialization as part of the start function.
  • __wbindgen_tcb_{get,set} - in addition to a thread id it's assumed
    that there's a global available for simply storing a pointer's worth
    of information (a thread control block, which currently only contains
    thread local storage). This would ideally be a native global
    injected by LLVM, but we don't have a great way to support that right
    now.

To reiterate, this is all intended to be unstable and purely intended
for testing out Rust on the web with threads. The story is very likely
to change in the future and we want to make sure that we're able to do
that!

@rust-highfive
Copy link
Contributor

r? @bluss

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Oct 10, 2018
@alexcrichton
Copy link
Member Author

r? @SimonSapin

@rust-highfive rust-highfive assigned SimonSapin and unassigned bluss Oct 10, 2018
@rust-highfive
Copy link
Contributor

The job x86_64-gnu-llvm-5.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[00:04:13] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:04:14] tidy error: /checkout/src/libstd/sys/wasm/thread_local_atomics.rs:42: TODO is deprecated; use FIXME
[00:04:14] tidy error: /checkout/src/libstd/sys/wasm/thread.rs:98: TODO is deprecated; use FIXME
[00:04:15] some tidy checks failed
[00:04:15] 
[00:04:15] 
[00:04:15] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:04:15] 
[00:04:15] 
[00:04:15] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:04:15] Build completed unsuccessfully in 0:00:42
[00:04:15] Build completed unsuccessfully in 0:00:42
[00:04:15] make: *** [tidy] Error 1
[00:04:15] Makefile:79: recipe for target 'tidy' failed

The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:2203cbf0
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
---
travis_time:end:0072fa74:start=1539152573695837699,finish=1539152573699600477,duration=3762778
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:00c48a92
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:360961b8
travis_time:start:360961b8
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:32b53a88
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@rust-highfive
Copy link
Contributor

The job x86_64-gnu-llvm-5.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[00:04:15] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:04:16] tidy error: /checkout/src/libstd/sys/wasm/thread_local_atomics.rs:42: TODO is deprecated; use FIXME
[00:04:17] some tidy checks failed
[00:04:17] 
[00:04:17] 
[00:04:17] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:04:17] 
[00:04:17] 
[00:04:17] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:04:17] Build completed unsuccessfully in 0:00:48
[00:04:17] Build completed unsuccessfully in 0:00:48
[00:04:17] Makefile:79: recipe for target 'tidy' failed
[00:04:17] make: *** [tidy] Error 1

The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:09d5c42f
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
---
travis_time:end:04e1f27c:start=1539153692745016423,finish=1539153692750064385,duration=5047962
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:1072f38c
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:091b0c84
travis_time:start:091b0c84
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:034a6106
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@SimonSapin
Copy link
Contributor

I know some of those words, but I feel very much not qualified to review this and don’t know who would be.

@alexcrichton
Copy link
Member Author

@sfackler sorry I don't mean to dump all reviews on you, but you might be most familiar with all this as well, mind taking a look? If you don't have time though just let me know!

if key >= MAX_KEYS {
panic!("cannot allocate space for more TLS keys");
}
return key + 1 // offset by 1 so we never hand out 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we want to hand out 0?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah this is a bit cryptic (and perhaps backwards), but it's due to this block where out platform-agnostic thread local code can't deal with a TLS descriptor of 0 so it just asks for another one if that happens

panic!("TLS on wasm with atomics not implemented yet");
pub unsafe fn create(dtor: Option<unsafe extern fn(*mut u8)>) -> Key {
drop(dtor); // FIXME: need to figure out how to hook thread exit to run this
let key = NEXT_KEY.fetch_add(1, SeqCst);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can technically overflow after 4 billion repeated failures but that may not be worth worrying about?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heh I'm not too worried about overflow but it's easy enough to add a store beneath this in the failure case back to MAX_KEYS, so done now!

This adds an implementation of thread local storage for the
`wasm32-unknown-unknown` target when the `atomics` feature is
implemented. This, however, comes with a notable caveat of that it
requires a new feature of the standard library, `wasm-bindgen-threads`,
to be enabled.

Thread local storage for wasm (when `atomics` are enabled and there's
actually more than one thread) is powered by the assumption that an
external entity can fill in some information for us. It's not currently
clear who will fill in this information nor whose responsibility it
should be long-term. In the meantime there's a strategy being gamed out
in the `wasm-bindgen` project specifically, and the hope is that we can
continue to test and iterate on the standard library without committing
to a particular strategy yet.

As to the details of `wasm-bindgen`'s strategy, LLVM doesn't currently
have the ability to emit custom `global` values (thread locals in a
`WebAssembly.Module`) so we leverage the `wasm-bindgen` CLI tool to do
it for us. To that end we have a few intrinsics, assuming two global values:

* `__wbindgen_current_id` - gets the current thread id as a 32-bit
  integer. It's `wasm-bindgen`'s responsibility to initialize this
  per-thread and then inform libstd of the id. Currently `wasm-bindgen`
  performs this initialization as part of the `start` function.
* `__wbindgen_tcb_{get,set}` - in addition to a thread id it's assumed
  that there's a global available for simply storing a pointer's worth
  of information (a thread control block, which currently only contains
  thread local storage). This would ideally be a native `global`
  injected by LLVM, but we don't have a great way to support that right
  now.

To reiterate, this is all intended to be unstable and purely intended
for testing out Rust on the web with threads. The story is very likely
to change in the future and we want to make sure that we're able to do
that!
@sfackler
Copy link
Member

@bors r+

@bors
Copy link
Collaborator

bors commented Oct 11, 2018

📌 Commit cbe9f33 has been approved by sfackler

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 11, 2018
@bors
Copy link
Collaborator

bors commented Oct 13, 2018

⌛ Testing commit cbe9f33 with merge 3431fa629099c29a8ade8fc1ad5eb9019e076527...

@bors
Copy link
Collaborator

bors commented Oct 13, 2018

💔 Test failed - status-travis

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Oct 13, 2018
@rust-highfive
Copy link
Contributor

The job x86_64-gnu-distcheck of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
[02:20:14] warning: spurious network error (2 tries remaining): curl error: Could not resolve host: github.com
[02:20:14] ; class=Net (12)
[02:20:35] warning: spurious network error (1 tries remaining): curl error: Could not resolve host: github.com
[02:20:35] ; class=Net (12)
[02:20:55] error: failed to load source for a dependency on `rand`
[02:20:55] Caused by:
[02:20:55]   Unable to update registry `https://github.com/rust-lang/crates.io-index`
[02:20:55] 
[02:20:55] Caused by:
[02:20:55] Caused by:
[02:20:55]   failed to fetch `https://github.com/rust-lang/crates.io-index`
[02:20:55] 
[02:20:55] Caused by:
[02:20:55]   curl error: Could not resolve host: github.com
[02:20:55] ; class=Net (12)
[02:20:55] 
[02:20:55] 
[02:20:55] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "generate-lockfile" "--manifest-path" "/checkout/obj/build/tmp/distcheck-src/rust-src/lib/rustlib/src/rust/src/libstd/Cargo.toml"
[02:20:55] 
[02:20:55] 
[02:20:55] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test distcheck
[02:20:55] Build completed unsuccessfully in 2:17:46
---
travis_time:end:06aca06c:start=1539413165398035422,finish=1539413165404781177,duration=6745755
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:003a863f
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:04a7c2d0
travis_time:start:04a7c2d0
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:23062fa2
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@alexcrichton
Copy link
Member Author

@bors: retry

@bors bors removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Oct 13, 2018
@bors bors added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Oct 13, 2018
@bors
Copy link
Collaborator

bors commented Oct 13, 2018

⌛ Testing commit cbe9f33 with merge f517527...

bors added a commit that referenced this pull request Oct 13, 2018
std: Implement TLS for wasm32-unknown-unknown

This adds an implementation of thread local storage for the
`wasm32-unknown-unknown` target when the `atomics` feature is
implemented. This, however, comes with a notable caveat of that it
requires a new feature of the standard library, `wasm-bindgen-threads`,
to be enabled.

Thread local storage for wasm (when `atomics` are enabled and there's
actually more than one thread) is powered by the assumption that an
external entity can fill in some information for us. It's not currently
clear who will fill in this information nor whose responsibility it
should be long-term. In the meantime there's a strategy being gamed out
in the `wasm-bindgen` project specifically, and the hope is that we can
continue to test and iterate on the standard library without committing
to a particular strategy yet.

As to the details of `wasm-bindgen`'s strategy, LLVM doesn't currently
have the ability to emit custom `global` values (thread locals in a
`WebAssembly.Module`) so we leverage the `wasm-bindgen` CLI tool to do
it for us. To that end we have a few intrinsics, assuming two global values:

* `__wbindgen_current_id` - gets the current thread id as a 32-bit
  integer. It's `wasm-bindgen`'s responsibility to initialize this
  per-thread and then inform libstd of the id. Currently `wasm-bindgen`
  performs this initialization as part of the `start` function.
* `__wbindgen_tcb_{get,set}` - in addition to a thread id it's assumed
  that there's a global available for simply storing a pointer's worth
  of information (a thread control block, which currently only contains
  thread local storage). This would ideally be a native `global`
  injected by LLVM, but we don't have a great way to support that right
  now.

To reiterate, this is all intended to be unstable and purely intended
for testing out Rust on the web with threads. The story is very likely
to change in the future and we want to make sure that we're able to do
that!
@bors
Copy link
Collaborator

bors commented Oct 13, 2018

☀️ Test successful - status-appveyor, status-travis
Approved by: sfackler
Pushing f517527 to master...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants