Skip to content

Conversation

@madsmtm
Copy link
Contributor

@madsmtm madsmtm commented Sep 16, 2025

At least when the linker is driven by rustc. See the comments in libmimalloc-sys/src/lib.rs for details.

Finally fixes #41, see https://github.com/madsmtm/test-malloc-overriding for the tests I've performed.

Builds upon #145, hence why this is marked as a draft.

@octavonce
Copy link
Collaborator

Also great stuff 💯

When you're ready with this we can merge

At least when the final linker invocation is driven by `rustc`.
@madsmtm
Copy link
Contributor Author

madsmtm commented Sep 17, 2025

Rebased, I think this is ready now.

@madsmtm madsmtm marked this pull request as ready for review September 17, 2025 15:15
@octavonce octavonce merged commit 0a32a38 into purpleprotocol:master Sep 17, 2025
5 checks passed
@madsmtm madsmtm deleted the true-override branch September 17, 2025 15:54
Zalathar added a commit to Zalathar/rust that referenced this pull request Nov 11, 2025
…zelmann

Simplify `jemalloc` setup

In the past, `#[used]` had to appear in the top-level crate to have a consistent effect on the linker. This has been fixed a while ago for ELF with the introduction of the `symbols.o` file in rust-lang#95604, and more recently for Mach-O in rust-lang#133832, which means that libraries can now implement the required workarounds themselves. This allows moving these `#[used]` declarations out of our `main.rs`.

Specifically, I have moved them into `tikv-jemalloc-sys` where they belong in tikv/jemallocator#109 and done the same for `mimalloc` in purpleprotocol/mimalloc_rust#146 (in case we want to experiment with switching to that one day).

Test with:
```sh
./x build library src/tools/rustdoc src/tools/clippy --set rust.jemalloc=true

# macOS
lldb -- ./build/host/stage1/bin/rustc -vV
(lldb) b _rjem_je_zone_register
(lldb) run
# Should breakpoint, this means that the allocator was properly linked

# Linux
lldb -- ./build/host/stage1/bin/rustc -vV
(lldb) b malloc
(lldb) run
# Should breakpoint, inspect that the `malloc` symbol comes from the `rustc` binary and not from `libc`
```

try-job: `aarch64-gnu`
try-job: `dist-aarch64-linux`
try-job: `dist-x86_64-musl`
try-job: `dist-x86_64-apple`
try-job: `dist-aarch64-apple`
Zalathar added a commit to Zalathar/rust that referenced this pull request Nov 11, 2025
…zelmann

Simplify `jemalloc` setup

In the past, `#[used]` had to appear in the top-level crate to have a consistent effect on the linker. This has been fixed a while ago for ELF with the introduction of the `symbols.o` file in rust-lang#95604, and more recently for Mach-O in rust-lang#133832, which means that libraries can now implement the required workarounds themselves. This allows moving these `#[used]` declarations out of our `main.rs`.

Specifically, I have moved them into `tikv-jemalloc-sys` where they belong in tikv/jemallocator#109 and done the same for `mimalloc` in purpleprotocol/mimalloc_rust#146 (in case we want to experiment with switching to that one day).

Test with:
```sh
./x build library src/tools/rustdoc src/tools/clippy --set rust.jemalloc=true

# macOS
lldb -- ./build/host/stage1/bin/rustc -vV
(lldb) b _rjem_je_zone_register
(lldb) run
# Should breakpoint, this means that the allocator was properly linked

# Linux
lldb -- ./build/host/stage1/bin/rustc -vV
(lldb) b malloc
(lldb) run
# Should breakpoint, inspect that the `malloc` symbol comes from the `rustc` binary and not from `libc`
```

try-job: `aarch64-gnu`
try-job: `dist-aarch64-linux`
try-job: `dist-x86_64-musl`
try-job: `dist-x86_64-apple`
try-job: `dist-aarch64-apple`
Zalathar added a commit to Zalathar/rust that referenced this pull request Nov 12, 2025
…zelmann

Simplify `jemalloc` setup

In the past, `#[used]` had to appear in the top-level crate to have a consistent effect on the linker. This has been fixed a while ago for ELF with the introduction of the `symbols.o` file in rust-lang#95604, and more recently for Mach-O in rust-lang#133832, which means that libraries can now implement the required workarounds themselves. This allows moving these `#[used]` declarations out of our `main.rs`.

Specifically, I have moved them into `tikv-jemalloc-sys` where they belong in tikv/jemallocator#109 and done the same for `mimalloc` in purpleprotocol/mimalloc_rust#146 (in case we want to experiment with switching to that one day).

Test with:
```sh
./x build library src/tools/rustdoc src/tools/clippy --set rust.jemalloc=true

# macOS
lldb -- ./build/host/stage1/bin/rustc -vV
(lldb) b _rjem_je_zone_register
(lldb) run
# Should breakpoint, this means that the allocator was properly linked

# Linux
lldb -- ./build/host/stage1/bin/rustc -vV
(lldb) b malloc
(lldb) run
# Should breakpoint, inspect that the `malloc` symbol comes from the `rustc` binary and not from `libc`
```

try-job: `aarch64-gnu`
try-job: `dist-aarch64-linux`
try-job: `dist-x86_64-musl`
try-job: `dist-x86_64-apple`
try-job: `dist-aarch64-apple`
Zalathar added a commit to Zalathar/rust that referenced this pull request Nov 12, 2025
…zelmann

Simplify `jemalloc` setup

In the past, `#[used]` had to appear in the top-level crate to have a consistent effect on the linker. This has been fixed a while ago for ELF with the introduction of the `symbols.o` file in rust-lang#95604, and more recently for Mach-O in rust-lang#133832, which means that libraries can now implement the required workarounds themselves. This allows moving these `#[used]` declarations out of our `main.rs`.

Specifically, I have moved them into `tikv-jemalloc-sys` where they belong in tikv/jemallocator#109 and done the same for `mimalloc` in purpleprotocol/mimalloc_rust#146 (in case we want to experiment with switching to that one day).

Test with:
```sh
./x build library src/tools/rustdoc src/tools/clippy --set rust.jemalloc=true

# macOS
lldb -- ./build/host/stage1/bin/rustc -vV
(lldb) b _rjem_je_zone_register
(lldb) run
# Should breakpoint, this means that the allocator was properly linked

# Linux
lldb -- ./build/host/stage1/bin/rustc -vV
(lldb) b malloc
(lldb) run
# Should breakpoint, inspect that the `malloc` symbol comes from the `rustc` binary and not from `libc`
```

try-job: `aarch64-gnu`
try-job: `dist-aarch64-linux`
try-job: `dist-x86_64-musl`
try-job: `dist-x86_64-apple`
try-job: `dist-aarch64-apple`
rust-timer added a commit to rust-lang/rust that referenced this pull request Nov 12, 2025
Rollup merge of #146627 - madsmtm:jemalloc-simplify, r=jdonszelmann

Simplify `jemalloc` setup

In the past, `#[used]` had to appear in the top-level crate to have a consistent effect on the linker. This has been fixed a while ago for ELF with the introduction of the `symbols.o` file in #95604, and more recently for Mach-O in #133832, which means that libraries can now implement the required workarounds themselves. This allows moving these `#[used]` declarations out of our `main.rs`.

Specifically, I have moved them into `tikv-jemalloc-sys` where they belong in tikv/jemallocator#109 and done the same for `mimalloc` in purpleprotocol/mimalloc_rust#146 (in case we want to experiment with switching to that one day).

Test with:
```sh
./x build library src/tools/rustdoc src/tools/clippy --set rust.jemalloc=true

# macOS
lldb -- ./build/host/stage1/bin/rustc -vV
(lldb) b _rjem_je_zone_register
(lldb) run
# Should breakpoint, this means that the allocator was properly linked

# Linux
lldb -- ./build/host/stage1/bin/rustc -vV
(lldb) b malloc
(lldb) run
# Should breakpoint, inspect that the `malloc` symbol comes from the `rustc` binary and not from `libc`
```

try-job: `aarch64-gnu`
try-job: `dist-aarch64-linux`
try-job: `dist-x86_64-musl`
try-job: `dist-x86_64-apple`
try-job: `dist-aarch64-apple`
github-actions bot pushed a commit to rust-lang/miri that referenced this pull request Nov 12, 2025
Simplify `jemalloc` setup

In the past, `#[used]` had to appear in the top-level crate to have a consistent effect on the linker. This has been fixed a while ago for ELF with the introduction of the `symbols.o` file in rust-lang/rust#95604, and more recently for Mach-O in rust-lang/rust#133832, which means that libraries can now implement the required workarounds themselves. This allows moving these `#[used]` declarations out of our `main.rs`.

Specifically, I have moved them into `tikv-jemalloc-sys` where they belong in tikv/jemallocator#109 and done the same for `mimalloc` in purpleprotocol/mimalloc_rust#146 (in case we want to experiment with switching to that one day).

Test with:
```sh
./x build library src/tools/rustdoc src/tools/clippy --set rust.jemalloc=true

# macOS
lldb -- ./build/host/stage1/bin/rustc -vV
(lldb) b _rjem_je_zone_register
(lldb) run
# Should breakpoint, this means that the allocator was properly linked

# Linux
lldb -- ./build/host/stage1/bin/rustc -vV
(lldb) b malloc
(lldb) run
# Should breakpoint, inspect that the `malloc` symbol comes from the `rustc` binary and not from `libc`
```

try-job: `aarch64-gnu`
try-job: `dist-aarch64-linux`
try-job: `dist-x86_64-musl`
try-job: `dist-x86_64-apple`
try-job: `dist-aarch64-apple`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

override feature is unsafe or broken on many platforms and should be labeled as such

2 participants