Skip to content

libstd/sys/unix/thread.rs panic while running cargo 0.22.0 #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
nielx opened this issue Jul 1, 2018 · 2 comments
Closed

libstd/sys/unix/thread.rs panic while running cargo 0.22.0 #3

nielx opened this issue Jul 1, 2018 · 2 comments
Labels

Comments

@nielx
Copy link
Owner

nielx commented Jul 1, 2018

Since rust 1.21.0 there, the ​standard library panics when there are os errors on thread.join.

In cargo such a panic occurs when compiling larger crates.
The error is 'failed to join thread: No such process (os error -2147454963).'

It is unlikely that the underlying cause started since cargo 0.22.0, but rather that it has occured before but it never caused an error because it was hidden as a debug_assert.

Possible causes:

  • The pthread implementation of Haiku does something odd
  • Cargo or one of its dependencies do something odd (like doing something that relies on undefined behavior).
  • Haiku has specific API calls that need to be handled by libstd.

Currently worked around with this commit.

@nielx nielx added the bug label Jul 1, 2018
nielx pushed a commit that referenced this issue Nov 3, 2019
Fixed: error: unnecessary trailing semicolon
nielx pushed a commit that referenced this issue Jun 26, 2020
update from origin 2020-06-15
@nielx
Copy link
Owner Author

nielx commented Aug 28, 2020

Patch for Haiku submitted: https://review.haiku-os.org/c/haiku/+/3178

@nielx
Copy link
Owner Author

nielx commented Sep 1, 2020

Will be fixed in Haiku R1 beta 3. Until that point, any Rust releases that fall within the pre-beta 3 timeframe will be released with the workaround.

@nielx nielx closed this as completed Sep 1, 2020
haiku-bot pushed a commit to haiku/haiku that referenced this issue Sep 1, 2020
When a thread is created, it is expected that some other thread (usually the
creating thread) will want to make sure it completes. This is done using the
pthread_join() or wait_for_thread() calls.

It is possible that threads end before another thread waits for its completion.
That's why there is a dead thread list for each team, which holds thread ids
and their exit status so that a call to pthread_join() or wait_for_thread() in
the future can complete succesfully.

The dead thread list was limited to 32 threads per team. If there would be
more, the oldest thread would be kicked off. This could cause issues in
situations where a team would create more than 32 threads, and would start
waiting for their result after they have finished. Some of the calls would fail
because the threads would no longer be in the dead list.

This specifically caused problems for cargo (the Rust package manager), which
could depending on the number of dependencies, could create more than 32
threads. See: nielx/rust#3

This change removes the limit of dead threads within a team. Note that there is
a risk that a badly written program that does not detach or joins its threads
can make this an endless list, but the impact is relatively small (dead threads
only occupy a bit of kernel memory).

Change-Id: I0135dd54e10ee48a529f23228d21237d4f1a74e2
Reviewed-on: https://review.haiku-os.org/c/haiku/+/3178
Reviewed-by: Adrien Destugues <[email protected]>
nielx pushed a commit that referenced this issue Dec 7, 2020
Don't run `resolve_vars_if_possible` in `normalize_erasing_regions`

Neither `@eddyb` nor I could figure out what this was for. I changed it to `assert_eq!(normalized_value, infcx.resolve_vars_if_possible(&normalized_value));` and it passed the UI test suite.

<details><summary>

Outdated, I figured out the issue - `needs_infer()` needs to come _after_ erasing the lifetimes

</summary>

Strangely, if I change it to `assert!(!normalized_value.needs_infer())` it panics almost immediately:

```
query stack during panic:
#0 [normalize_generic_arg_after_erasing_regions] normalizing `<str::IsWhitespace as str::pattern::Pattern>::Searcher`
#1 [needs_drop_raw] computing whether `str::iter::Split<str::IsWhitespace>` needs drop
#2 [mir_built] building MIR for `str::<impl str>::split_whitespace`
#3 [unsafety_check_result] unsafety-checking `str::<impl str>::split_whitespace`
#4 [mir_const] processing MIR for `str::<impl str>::split_whitespace`
#5 [mir_promoted] processing `str::<impl str>::split_whitespace`
#6 [mir_borrowck] borrow-checking `str::<impl str>::split_whitespace`
#7 [analysis] running analysis passes on this crate
end of query stack
```

I'm not entirely sure what's going on - maybe the two disagree?

</details>

For context, this came up while reviewing rust-lang#77467 (cc `@lcnr).`

Possibly this needs a crater run?

r? `@nikomatsakis`
cc `@matthewjasper`
nielx pushed a commit that referenced this issue Feb 16, 2021
HWAddressSanitizer support

#  Motivation
Compared to regular ASan, HWASan has a [smaller overhead](https://source.android.com/devices/tech/debug/hwasan). The difference in practice is that HWASan'ed code is more usable, e.g. Android device compiled with HWASan can be used as a daily driver.

# Example
```
fn main() {
    let xs = vec![0, 1, 2, 3];
    let _y = unsafe { *xs.as_ptr().offset(4) };
}
```
```
==223==ERROR: HWAddressSanitizer: tag-mismatch on address 0xefdeffff0050 at pc 0xaaaad00b3468
READ of size 4 at 0xefdeffff0050 tags: e5/00 (ptr/mem) in thread T0
    #0 0xaaaad00b3464  (/root/main+0x53464)
    #1 0xaaaad00b39b4  (/root/main+0x539b4)
    #2 0xaaaad00b3dd0  (/root/main+0x53dd0)
    #3 0xaaaad00b61dc  (/root/main+0x561dc)
    #4 0xaaaad00c0574  (/root/main+0x60574)
    #5 0xaaaad00b6290  (/root/main+0x56290)
    #6 0xaaaad00b6170  (/root/main+0x56170)
    #7 0xaaaad00b3578  (/root/main+0x53578)
    #8 0xffff81345e70  (/lib64/libc.so.6+0x20e70)
    #9 0xaaaad0096310  (/root/main+0x36310)

[0xefdeffff0040,0xefdeffff0060) is a small allocated heap chunk; size: 32 offset: 16
0xefdeffff0050 is located 0 bytes to the right of 16-byte region [0xefdeffff0040,0xefdeffff0050)
allocated here:
    #0 0xaaaad009bcdc  (/root/main+0x3bcdc)
    #1 0xaaaad00b1eb0  (/root/main+0x51eb0)
    #2 0xaaaad00b20d4  (/root/main+0x520d4)
    #3 0xaaaad00b2800  (/root/main+0x52800)
    #4 0xaaaad00b1cf4  (/root/main+0x51cf4)
    #5 0xaaaad00b33d4  (/root/main+0x533d4)
    #6 0xaaaad00b39b4  (/root/main+0x539b4)
    #7 0xaaaad00b61dc  (/root/main+0x561dc)
    #8 0xaaaad00b3578  (/root/main+0x53578)
    #9 0xaaaad0096310  (/root/main+0x36310)

Thread: T0 0xeffe00002000 stack: [0xffffc0590000,0xffffc0d90000) sz: 8388608 tls: [0xffff81521020,0xffff815217d0)
Memory tags around the buggy address (one tag corresponds to 16 bytes):
  0xfefcefffef80: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xfefcefffef90: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xfefcefffefa0: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xfefcefffefb0: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xfefcefffefc0: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xfefcefffefd0: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xfefcefffefe0: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xfefcefffeff0: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
=>0xfefceffff000: a2  a2  05  00  e5 [00] 00  00  00  00  00  00  00  00  00  00
  0xfefceffff010: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xfefceffff020: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xfefceffff030: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xfefceffff040: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xfefceffff050: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xfefceffff060: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xfefceffff070: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xfefceffff080: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
Tags for short granules around the buggy address (one tag corresponds to 16 bytes):
  0xfefcefffeff0: ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..
=>0xfefceffff000: ..  ..  c5  ..  .. [..] ..  ..  ..  ..  ..  ..  ..  ..  ..  ..
  0xfefceffff010: ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..
See https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html#short-granules for a description of short granule tags
Registers where the failure occurred (pc 0xaaaad00b3468):
    x0  e500efdeffff0050  x1  0000000000000004  x2  0000ffffc0d8f5a0  x3  0200efff00000000
    x4  0000ffffc0d8f4c0  x5  000000000000004f  x6  00000ffffc0d8f36  x7  0000efff00000000
    x8  e500efdeffff0050  x9  0200efff00000000  x10 0000000000000000  x11 0200efff00000000
    x12 0200effe000006b0  x13 0200effe000006b0  x14 0000000000000008  x15 00000000c00000cf
    x16 0000aaaad00a0afc  x17 0000000000000003  x18 0000000000000001  x19 0000ffffc0d8f718
    x20 ba00ffffc0d8f7a0  x21 0000aaaad00962e0  x22 0000000000000000  x23 0000000000000000
    x24 0000000000000000  x25 0000000000000000  x26 0000000000000000  x27 0000000000000000
    x28 0000000000000000  x29 0000ffffc0d8f650  x30 0000aaaad00b3468
```

# Comments/Caveats
* HWASan is only supported on arm64.
* I'm not sure if I should add a feature gate or piggyback on the existing one for sanitizers.
* HWASan requires `-C target-feature=+tagged-globals`. That flag should probably be set transparently to the user. Not sure how to go about that.

# TODO
* Need more tests.
* Update documentation.
* Fix symbolization.
* Integrate with CI
nielx pushed a commit that referenced this issue Dec 17, 2021
…rk-Simulacrum

Override `Iterator::advance(_back)_by` for `array::IntoIter`

Because I happened to notice that `nth` is currently getting codegen'd as a loop even for `Copy` types: <https://rust.godbolt.org/z/fPqv7Gvs7>

<details>
<summary>LLVM before and after</summary>

Rust:

```rust
#[no_mangle]
pub fn array_intoiter_nth(it: &mut std::array::IntoIter<i32, 100>, n: usize) -> Option<i32> {
    it.nth(n)
}
```

Current nightly:
```llvmir
define { i32, i32 } `@array_intoiter_nth(%"core::array::iter::IntoIter<i32,` 100_usize>"* noalias nocapture align 8 dereferenceable(416) %it, i64 %n) unnamed_addr #0 personality i32 (i32, i32, i64, %"unwind::libunwind::_Unwind_Exception"*, %"unwind::libunwind::_Unwind_Context"*)* `@rust_eh_personality` !dbg !6 {
start:
  %_3.i.i.i4.i.i = getelementptr inbounds %"core::array::iter::IntoIter<i32, 100_usize>", %"core::array::iter::IntoIter<i32, 100_usize>"* %it, i64 0, i32 0, i32 0
  %_4.i.i.i5.i.i = getelementptr inbounds %"core::array::iter::IntoIter<i32, 100_usize>", %"core::array::iter::IntoIter<i32, 100_usize>"* %it, i64 0, i32 0, i32 1
  %_4.i.i.i.i.i.i = load i64, i64* %_4.i.i.i5.i.i, align 8, !alias.scope !10
  %.not.i.i = icmp eq i64 %n, 0, !dbg !15
  %_3.i.i.i.i.pre.i = load i64, i64* %_3.i.i.i4.i.i, align 8, !dbg !40, !alias.scope !41
  br i1 %.not.i.i, label %bb4.i, label %bb4.preheader.i.i, !dbg !42

bb4.preheader.i.i:                                ; preds = %start
  %umax.i = tail call i64 `@llvm.umax.i64(i64` %_3.i.i.i.i.pre.i, i64 %_4.i.i.i.i.i.i) #3, !dbg !43
  %0 = sub i64 %umax.i, %_3.i.i.i.i.pre.i, !dbg !43
  br label %bb4.i.i, !dbg !43

bb4.i.i:                                          ; preds = %bb3.i.i.i.i, %bb4.preheader.i.i
  %_3.i.i.i.i.i.i = phi i64 [ %2, %bb3.i.i.i.i ], [ %_3.i.i.i.i.pre.i, %bb4.preheader.i.i ], !dbg !52
  %iter.sroa.0.016.i.i = phi i64 [ %1, %bb3.i.i.i.i ], [ 0, %bb4.preheader.i.i ]
  %1 = add nuw i64 %iter.sroa.0.016.i.i, 1, !dbg !54
  %exitcond.not.i = icmp eq i64 %iter.sroa.0.016.i.i, %0, !dbg !52
  br i1 %exitcond.not.i, label %core::iter::traits::iterator::Iterator::nth.exit, label %bb3.i.i.i.i, !dbg !43

bb3.i.i.i.i:                                      ; preds = %bb4.i.i
  %2 = add nuw i64 %_3.i.i.i.i.i.i, 1, !dbg !63
  store i64 %2, i64* %_3.i.i.i4.i.i, align 8, !dbg !66, !alias.scope !75
  %exitcond.not.i.i = icmp eq i64 %1, %n, !dbg !15
  br i1 %exitcond.not.i.i, label %bb4.i, label %bb4.i.i, !dbg !42

bb4.i:                                            ; preds = %bb3.i.i.i.i, %start
  %_3.i.i.i.i.i = phi i64 [ %_3.i.i.i.i.pre.i, %start ], [ %2, %bb3.i.i.i.i ], !dbg !84
  %3 = icmp ult i64 %_3.i.i.i.i.i, %_4.i.i.i.i.i.i, !dbg !84
  br i1 %3, label %bb3.i.i.i, label %core::iter::traits::iterator::Iterator::nth.exit, !dbg !89

bb3.i.i.i:                                        ; preds = %bb4.i
  %4 = add nuw i64 %_3.i.i.i.i.i, 1, !dbg !90
  store i64 %4, i64* %_3.i.i.i4.i.i, align 8, !dbg !93, !alias.scope !96
  %5 = getelementptr inbounds %"core::array::iter::IntoIter<i32, 100_usize>", %"core::array::iter::IntoIter<i32, 100_usize>"* %it, i64 0, i32 1, i64 %_3.i.i.i.i.i, !dbg !105
  %6 = load i32, i32* %5, align 4, !dbg !131, !alias.scope !141, !noalias !144
  br label %core::iter::traits::iterator::Iterator::nth.exit, !dbg !149

core::iter::traits::iterator::Iterator::nth.exit: ; preds = %bb4.i.i, %bb4.i, %bb3.i.i.i
  %.sroa.3.0.i = phi i32 [ %6, %bb3.i.i.i ], [ undef, %bb4.i ], [ undef, %bb4.i.i ], !dbg !40
  %.sroa.0.0.i = phi i32 [ 1, %bb3.i.i.i ], [ 0, %bb4.i ], [ 0, %bb4.i.i ], !dbg !40
  %7 = insertvalue { i32, i32 } undef, i32 %.sroa.0.0.i, 0, !dbg !150
  %8 = insertvalue { i32, i32 } %7, i32 %.sroa.3.0.i, 1, !dbg !150
  ret { i32, i32 } %8, !dbg !151
}
```

With this PR:
```llvmir
define { i32, i32 } `@array_intoiter_nth(%"core::array::iter::IntoIter<i32,` 100_usize>"* noalias nocapture align 8 dereferenceable(416) %it, i64 %n) unnamed_addr #0 personality i32 (...)* `@__CxxFrameHandler3` {
start:
  %0 = getelementptr inbounds %"core::array::iter::IntoIter<i32, 100_usize>", %"core::array::iter::IntoIter<i32, 100_usize>"* %it, i64 0, i32 0, i32 1
  %_2.i.i.i.i = load i64, i64* %0, align 8, !alias.scope !6, !noalias !13
  %1 = getelementptr inbounds %"core::array::iter::IntoIter<i32, 100_usize>", %"core::array::iter::IntoIter<i32, 100_usize>"* %it, i64 0, i32 0, i32 0
  %_3.i.i.i.i = load i64, i64* %1, align 8, !alias.scope !16
  %2 = sub i64 %_2.i.i.i.i, %_3.i.i.i.i
  %3 = icmp ult i64 %2, %n
  %.0.sroa.speculated.i.i.i.i.i = select i1 %3, i64 %2, i64 %n
  %_10.i.i = add i64 %.0.sroa.speculated.i.i.i.i.i, %_3.i.i.i.i
  store i64 %_10.i.i, i64* %1, align 8, !alias.scope !16
  %.not.i = xor i1 %3, true
  %4 = icmp ult i64 %_10.i.i, %_2.i.i.i.i
  %or.cond.i = select i1 %.not.i, i1 %4, i1 false
  br i1 %or.cond.i, label %bb3.i.i.i, label %_ZN4core4iter6traits8iterator8Iterator3nth17hcbc727011e9e2a3bE.exit

bb3.i.i.i:                                        ; preds = %start
  %5 = add nuw i64 %_10.i.i, 1
  store i64 %5, i64* %1, align 8, !alias.scope !17
  %6 = getelementptr inbounds %"core::array::iter::IntoIter<i32, 100_usize>", %"core::array::iter::IntoIter<i32, 100_usize>"* %it, i64 0, i32 1, i64 %_10.i.i
  %7 = load i32, i32* %6, align 4, !alias.scope !26, !noalias !29
  br label %_ZN4core4iter6traits8iterator8Iterator3nth17hcbc727011e9e2a3bE.exit

_ZN4core4iter6traits8iterator8Iterator3nth17hcbc727011e9e2a3bE.exit: ; preds = %start, %bb3.i.i.i
  %.sroa.3.0.i = phi i32 [ undef, %start ], [ %7, %bb3.i.i.i ]
  %.sroa.0.0.i = phi i32 [ 0, %start ], [ 1, %bb3.i.i.i ]
  %8 = insertvalue { i32, i32 } undef, i32 %.sroa.0.0.i, 0
  %9 = insertvalue { i32, i32 } %8, i32 %.sroa.3.0.i, 1
  ret { i32, i32 } %9
}
```
</details>
nielx pushed a commit that referenced this issue Dec 27, 2022
[beta] Adjust miri to still be optional
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant