Implement Integer funnel shifts

[sayantn]

Parent: #145686
ACP: rust-lang/libs-team#642

This implements funnel shifts on primitive integer types. Adds support in cg_llvm and Miri

Thanks @folkertdev for the fixes, and the cg_gcc and cg_clif implementation

cc @rust-lang/libs-api

[rustbot]

r? @tgross35

rustbot has assigned @tgross35.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[rustbot]

Some changes occurred to the intrinsics. Make sure the CTFE / Miri interpreter
gets adapted for the changes, if necessary.

cc @rust-lang/miri, @RalfJung, @oli-obk, @lcnr

Some changes occurred to the CTFE / Miri interpreter

cc @rust-lang/miri

Some changes occurred to the CTFE machinery

cc @RalfJung, @oli-obk, @lcnr

[folkertdev]

can you update the comment/signature of the simd versions of this intrinsic to also reflect that it is safe to call?

[sayantn]

Do we want to do that? There are other backends where making such guarantees might require more effort. For scalars, it's quite easy, so we can make the guarantee

[RalfJung]

Please explicitly specify what happens when the shift is larger than the size of a / larger than the size of the concatenation. As written, it seems to imply that the return value is 0 in the latter case.

Looking at the docs, the shift amount seems to be modulo the size. That then makes me wonder why you talk about shifting in zeroes since those zeroes can never be visible?

Maybe that's what you mean with "wrapping on overflow" but that is not clear enough. (What gets wrapped how, and how do you define overflow? There's multiple definitions of what it means for a shift to overflow.)

[folkertdev]

yeah, the

with wrapping on overflow

bit isn't clear enough I think. For funnel shifts the shift amount is modulo'd by the total number of bits in the type. so, a funnel shift on a u8 by 12 is equivalent to one by 12 % u8::BITS.

Probably a good idea to just have an example of that in the doc comment as well.

[tgross35]

I see there is a tracking issue, but not an ACP for this. Could you create one? It's an issue template at https://github.com/rust-lang/libs-team/issues.

[rustbot]

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

[rustbot]

Some changes occurred in compiler/rustc_codegen_gcc

cc @antoyo, @GuillaumeGomez

Some changes occurred in compiler/rustc_codegen_cranelift

cc @bjorn3

[rust-log-analyzer]

The job aarch64-gnu-llvm-19-2 failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

[RUSTC-TIMING] allocbenches test:true 8.140
[RUSTC-TIMING] corebenches test:true 9.039
[RUSTC-TIMING] alloctests test:true 14.091
[RUSTC-TIMING] alloctests test:true 13.381
error: VerifierErrors([VerifierError { location: inst10, context: Some("v13 = icmp.i32 eq v11, v7  ; v7 = 0"), message: "arg 1 (v7) has type i8, expected i32" }])

error: cranelift verify error:
       function u0:1990(i8, i8, i32) -> i8 system_v {
                                       block0(v0: i8, v1: i8, v2: i32):
                                           v3 -> v0
                                           v4 -> v1
fn core::num::<impl u8>::funnel_shl(_1: u8, _2: u8, _3: u32) -> u8 {
    debug self => _1;
    debug rhs => _2;
    debug n => _3;
    let mut _0: u8;

    bb0: {
        _0 = std::intrinsics::funnel_shl::<u8>(move _1, move _2, move _3) -> [return: bb1, unwind unreachable];
    }

    bb1: {
        return;
    }
}

                                           v5 -> v2
                                           nop 
                                           jump block1
       
                                       block1:
                                           nop 
       @0000                               v6 = iconst.i32 8
       @0000                               v7 = iconst.i8 0
       @0000                               v8 = urem.i32 v5, v6  ; v6 = 8
       @0000                               v9 = ishl.i8 v3, v8
       @0000                               v10 = isub v6, v8  ; v6 = 8
       @0000                               v11 = urem v10, v6  ; v6 = 8
       @0000                               v12 = ushr.i8 v4, v11
       @0000                               v13 = icmp eq v11, v7  ; v7 = 0
       ;~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
       ; error: inst10 (v13 = icmp.i32 eq v11, v7  ; v7 = 0): arg 1 (v7) has type i8, expected i32
       
       @0000                               v14 = select v13, v7, v12  ; v7 = 0
       @0000                               v15 = bor v14, v9
                                           v16 -> v15
       @0000                               jump block2
       
                                       block2:
       @0000                               nop 
       @0001                               return v16
       }
       
       ; 1 verifier error detected (see above). Compilation aborted.


thread 'rustc' panicked at /cargo/registry/src/index.crates.io-1949cf8c6b5b557f/cranelift-codegen-0.121.0/src/verifier/mod.rs:1892:22:
internal error: entered unreachable code
stack backtrace:
   0: __rustc::rust_begin_unwind
             at /rustc/788da80fcfcef3f34c90def5baa32813e39a1a41/library/std/src/panicking.rs:697:5
   1: core::panicking::panic_fmt
             at /rustc/788da80fcfcef3f34c90def5baa32813e39a1a41/library/core/src/panicking.rs:75:14
   2: core::panicking::panic
             at /rustc/788da80fcfcef3f34c90def5baa32813e39a1a41/library/core/src/panicking.rs:145:5
   3: <cranelift_codegen::verifier::Verifier>::run
   4: cranelift_codegen::verifier::verify_function::<&cranelift_codegen::settings::Flags>
   5: rustc_codegen_cranelift::base::codegen_fn
   6: rustc_codegen_cranelift::driver::aot::module_codegen
   7: rustc_codegen_cranelift::driver::aot::run_aot::{closure#3}::{closure#0}
   8: <&mut rustc_data_structures::sync::parallel::par_map<(usize, &rustc_middle::mir::mono::CodegenUnit), alloc::vec::Vec<(usize, &rustc_middle::mir::mono::CodegenUnit)>, rustc_data_structures::marker::IntoDynSyncSend<rustc_codegen_cranelift::driver::aot::OngoingModuleCodegen>, alloc::vec::Vec<rustc_data_structures::marker::IntoDynSyncSend<rustc_codegen_cranelift::driver::aot::OngoingModuleCodegen>>, rustc_codegen_cranelift::driver::aot::run_aot::{closure#3}::{closure#0}>::{closure#0}::{closure#3} as core::ops::function::FnMut<((usize, &rustc_middle::mir::mono::CodegenUnit),)>>::call_mut
   9: <alloc::vec::into_iter::IntoIter<(usize, &rustc_middle::mir::mono::CodegenUnit)> as core::iter::traits::iterator::Iterator>::try_fold::<(), core::iter::traits::iterator::Iterator::find_map::check<(usize, &rustc_middle::mir::mono::CodegenUnit), rustc_data_structures::marker::IntoDynSyncSend<rustc_codegen_cranelift::driver::aot::OngoingModuleCodegen>, &mut rustc_data_structures::sync::parallel::par_map<(usize, &rustc_middle::mir::mono::CodegenUnit), alloc::vec::Vec<(usize, &rustc_middle::mir::mono::CodegenUnit)>, rustc_data_structures::marker::IntoDynSyncSend<rustc_codegen_cranelift::driver::aot::OngoingModuleCodegen>, alloc::vec::Vec<rustc_data_structures::marker::IntoDynSyncSend<rustc_codegen_cranelift::driver::aot::OngoingModuleCodegen>>, rustc_codegen_cranelift::driver::aot::run_aot::{closure#3}::{closure#0}>::{closure#0}::{closure#3}>::{closure#0}, core::ops::control_flow::ControlFlow<rustc_data_structures::marker::IntoDynSyncSend<rustc_codegen_cranelift::driver::aot::OngoingModuleCodegen>>>
  10: <alloc::vec::Vec<rustc_data_structures::marker::IntoDynSyncSend<rustc_codegen_cranelift::driver::aot::OngoingModuleCodegen>> as alloc::vec::spec_from_iter::SpecFromIter<rustc_data_structures::marker::IntoDynSyncSend<rustc_codegen_cranelift::driver::aot::OngoingModuleCodegen>, core::iter::adapters::filter_map::FilterMap<alloc::vec::into_iter::IntoIter<(usize, &rustc_middle::mir::mono::CodegenUnit)>, rustc_data_structures::sync::parallel::par_map<(usize, &rustc_middle::mir::mono::CodegenUnit), alloc::vec::Vec<(usize, &rustc_middle::mir::mono::CodegenUnit)>, rustc_data_structures::marker::IntoDynSyncSend<rustc_codegen_cranelift::driver::aot::OngoingModuleCodegen>, alloc::vec::Vec<rustc_data_structures::marker::IntoDynSyncSend<rustc_codegen_cranelift::driver::aot::OngoingModuleCodegen>>, rustc_codegen_cranelift::driver::aot::run_aot::{closure#3}::{closure#0}>::{closure#0}::{closure#3}>>>::from_iter
  11: <rustc_session::session::Session>::time::<alloc::vec::Vec<rustc_codegen_cranelift::driver::aot::OngoingModuleCodegen>, rustc_codegen_cranelift::driver::aot::run_aot::{closure#3}>
  12: rustc_codegen_cranelift::driver::aot::run_aot
  13: <rustc_codegen_cranelift::CraneliftCodegenBackend as rustc_codegen_ssa::traits::backend::CodegenBackend>::codegen_crate
  14: <rustc_session::session::Session>::time::<alloc::boxed::Box<dyn core::any::Any>, rustc_interface::passes::start_codegen::{closure#0}>
  15: rustc_interface::passes::start_codegen
  16: <rustc_interface::queries::Linker>::codegen_and_build_linker
  17: <std::thread::local::LocalKey<core::cell::Cell<*const ()>>>::with::<rustc_middle::ty::context::tls::enter_context<<rustc_middle::ty::context::GlobalCtxt>::enter<rustc_interface::passes::create_and_enter_global_ctxt<core::option::Option<rustc_interface::queries::Linker>, rustc_driver_impl::run_compiler::{closure#0}::{closure#2}>::{closure#2}::{closure#0}, core::option::Option<rustc_interface::queries::Linker>>::{closure#1}, core::option::Option<rustc_interface::queries::Linker>>::{closure#0}, core::option::Option<rustc_interface::queries::Linker>>
  18: <rustc_middle::ty::context::TyCtxt>::create_global_ctxt::<core::option::Option<rustc_interface::queries::Linker>, rustc_interface::passes::create_and_enter_global_ctxt<core::option::Option<rustc_interface::queries::Linker>, rustc_driver_impl::run_compiler::{closure#0}::{closure#2}>::{closure#2}::{closure#0}>
  19: <rustc_interface::passes::create_and_enter_global_ctxt<core::option::Option<rustc_interface::queries::Linker>, rustc_driver_impl::run_compiler::{closure#0}::{closure#2}>::{closure#2} as core::ops::function::FnOnce<(&rustc_session::session::Session, rustc_middle::ty::context::CurrentGcx, alloc::sync::Arc<rustc_data_structures::jobserver::Proxy>, &std::sync::once_lock::OnceLock<rustc_middle::ty::context::GlobalCtxt>, &rustc_data_structures::sync::worker_local::WorkerLocal<rustc_middle::arena::Arena>, &rustc_data_structures::sync::worker_local::WorkerLocal<rustc_hir::Arena>, rustc_driver_impl::run_compiler::{closure#0}::{closure#2})>>::call_once::{shim:vtable#0}
  20: <alloc::boxed::Box<dyn for<'a> core::ops::function::FnOnce<(&'a rustc_session::session::Session, rustc_middle::ty::context::CurrentGcx, alloc::sync::Arc<rustc_data_structures::jobserver::Proxy>, &'a std::sync::once_lock::OnceLock<rustc_middle::ty::context::GlobalCtxt<'a>>, &'a rustc_data_structures::sync::worker_local::WorkerLocal<rustc_middle::arena::Arena<'a>>, &'a rustc_data_structures::sync::worker_local::WorkerLocal<rustc_hir::Arena<'a>>, rustc_driver_impl::run_compiler::{closure#0}::{closure#2}), Output = core::option::Option<rustc_interface::queries::Linker>>> as core::ops::function::FnOnce<(&rustc_session::session::Session, rustc_middle::ty::context::CurrentGcx, alloc::sync::Arc<rustc_data_structures::jobserver::Proxy>, &std::sync::once_lock::OnceLock<rustc_middle::ty::context::GlobalCtxt>, &rustc_data_structures::sync::worker_local::WorkerLocal<rustc_middle::arena::Arena>, &rustc_data_structures::sync::worker_local::WorkerLocal<rustc_hir::Arena>, rustc_driver_impl::run_compiler::{closure#0}::{closure#2})>>::call_once
  21: rustc_interface::passes::create_and_enter_global_ctxt::<core::option::Option<rustc_interface::queries::Linker>, rustc_driver_impl::run_compiler::{closure#0}::{closure#2}>
  22: <scoped_tls::ScopedKey<rustc_span::SessionGlobals>>::set::<rustc_interface::util::run_in_thread_with_globals<rustc_interface::util::run_in_thread_pool_with_globals<rustc_interface::interface::run_compiler<(), rustc_driver_impl::run_compiler::{closure#0}>::{closure#1}, ()>::{closure#0}, ()>::{closure#0}::{closure#0}::{closure#0}, ()>
  23: rustc_span::create_session_globals_then::<(), rustc_interface::util::run_in_thread_with_globals<rustc_interface::util::run_in_thread_pool_with_globals<rustc_interface::interface::run_compiler<(), rustc_driver_impl::run_compiler::{closure#0}>::{closure#1}, ()>::{closure#0}, ()>::{closure#0}::{closure#0}::{closure#0}>
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

error: the compiler unexpectedly panicked. this is a bug.

note: using internal features is not supported and expected to cause internal compiler errors when used incorrectly

note: please attach the file at `/checkout/obj/build/aarch64-unknown-linux-gnu/stage2-tools/cg_clif/build/sysroot_tests/rustc-ice-2025-08-24T21_24_17-33416.txt` to your bug report

note: compiler flags: -C panic=abort -Z panic-abort-tests -Z codegen-backend=cranelift -C embed-bitcode=no -C debuginfo=2 -C symbol-mangling-version=v0 -Z randomize-layout -Z unstable-options -Z macro-backtrace -C split-debuginfo=off -C link-arg=-L/usr/lib/llvm-19/lib -C llvm-args=-import-instr-limit=10 -C link-args=-Wl,-z,origin -C link-args=-Wl,-rpath,$ORIGIN/../lib -Z force-unstable-if-unmarked

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
end of query stack
fn core::num::<impl u128>::funnel_shl(_1: u128, _2: u128, _3: u32) -> u128 {
    debug self => _1;
    debug rhs => _2;
    debug n => _3;
    let mut _0: u128;

    bb0: {
        _0 = std::intrinsics::funnel_shl::<u128>(move _1, move _2, move _3) -> [return: bb1, unwind unreachable];
    }

    bb1: {
        return;
    }
}

error: VerifierErrors([VerifierError { location: inst10, context: Some("v13 = icmp.i32 eq v11, v7  ; v7 = 0"), message: "arg 1 (v7) has type i64, expected i32" }])

error: cranelift verify error:
       function u0:2720(i64, i64, i32) -> i64 system_v {
                                       block0(v0: i64, v1: i64, v2: i32):
                                           v3 -> v0
                                           v4 -> v1
                                           v5 -> v2
                                           nop 
                                           jump block1
       
                                       block1:
                                           nop 
       @0000                               v6 = iconst.i32 64
       @0000                               v7 = iconst.i64 0
       @0000                               v8 = urem.i32 v5, v6  ; v6 = 64
       @0000                               v9 = ishl.i64 v3, v8
       @0000                               v10 = isub v6, v8  ; v6 = 64
       @0000                               v11 = urem v10, v6  ; v6 = 64
       @0000                               v12 = ushr.i64 v4, v11
       @0000                               v13 = icmp eq v11, v7  ; v7 = 0
       ;~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
       ; error: inst10 (v13 = icmp.i32 eq v11, v7  ; v7 = 0): arg 1 (v7) has type i64, expected i32
       
       @0000                               v14 = select v13, v7, v12  ; v7 = 0
       @0000                               v15 = bor v14, v9
                                           v16 -> v15
       @0000                               jump block2
       
                                       block2:
       @0000                               nop 
       @0001                               return v16
       }
       
       ; 1 verifier error detected (see above). Compilation aborted.

fn core::num::<impl u64>::funnel_shl(_1: u64, _2: u64, _3: u32) -> u64 {
    debug self => _1;
    debug rhs => _2;
    debug n => _3;
    let mut _0: u64;

    bb0: {
        _0 = std::intrinsics::funnel_shl::<u64>(move _1, move _2, move _3) -> [return: bb1, unwind unreachable];
    }

    bb1: {
        return;
    }
}

error: VerifierErrors([VerifierError { location: inst10, context: Some("v13 = icmp.i32 eq v11, v7  ; v7 = 0"), message: "arg 1 (v7) has type i16, expected i32" }])

error: cranelift verify error:
       function u0:3324(i16, i16, i32) -> i16 system_v {
                                       block0(v0: i16, v1: i16, v2: i32):
                                           v3 -> v0
                                           v4 -> v1
                                           v5 -> v2
                                           nop 
                                           jump block1
       
                                       block1:
                                           nop 
       @0000                               v6 = iconst.i32 16
       @0000                               v7 = iconst.i16 0
       @0000                               v8 = urem.i32 v5, v6  ; v6 = 16
       @0000                               v9 = ishl.i16 v3, v8
       @0000                               v10 = isub v6, v8  ; v6 = 16
       @0000                               v11 = urem v10, v6  ; v6 = 16
       @0000                               v12 = ushr.i16 v4, v11
       @0000                               v13 = icmp eq v11, v7  ; v7 = 0
       ;~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
       ; error: inst10 (v13 = icmp.i32 eq v11, v7  ; v7 = 0): arg 1 (v7) has type i16, expected i32
       
       @0000                               v14 = select v13, v7, v12  ; v7 = 0
       @0000                               v15 = bor v14, v9
                                           v16 -> v15
       @0000                               jump block2
       
                                       block2:
       @0000                               nop 
       @0001                               return v16
       }
       
       ; 1 verifier error detected (see above). Compilation aborted.

fn core::num::<impl u16>::funnel_shl(_1: u16, _2: u16, _3: u32) -> u16 {
    debug self => _1;
    debug rhs => _2;
    debug n => _3;
    let mut _0: u16;

    bb0: {
        _0 = std::intrinsics::funnel_shl::<u16>(move _1, move _2, move _3) -> [return: bb1, unwind unreachable];
    }

    bb1: {
        return;
    }
}

[RUSTC-TIMING] coretests test:true 63.225
error: could not compile `coretests` (test "coretests") due to 6 previous errors
env -u RUSTC_WRAPPER CARGO_ENCODED_RUSTDOCFLAGS="-Csymbol-mangling-version=v0\u{1f}-Zrandomize-layout\u{1f}-Zunstable-options\u{1f}--check-cfg=cfg(bootstrap)\u{1f}--check-cfg=cfg(llvm_enzyme)\u{1f}-Dwarnings\u{1f}-Wrustdoc::invalid_codeblock_attributes\u{1f}--crate-version\u{1f}1.91.0-nightly\t(06dd67077\t2025-08-24)" CARGO_ENCODED_RUSTFLAGS="-Csymbol-mangling-version=v0\u{1f}-Zrandomize-layout\u{1f}-Zunstable-options\u{1f}--check-cfg=cfg(bootstrap)\u{1f}--check-cfg=cfg(llvm_enzyme)\u{1f}-Zmacro-backtrace\u{1f}-Csplit-debuginfo=off\u{1f}-Clink-arg=-L/usr/lib/llvm-19/lib\u{1f}-Cllvm-args=-import-instr-limit=10\u{1f}-Clink-args=-Wl,-z,origin\u{1f}-Clink-args=-Wl,-rpath,$ORIGIN/../lib\u{1f}-Alinker-messages\u{1f}--cap-lints=allow\u{1f}--cfg\u{1f}randomized_layouts" RUSTC="/checkout/obj/build/aarch64-unknown-linux-gnu/stage2-tools/cg_clif/dist/rustc-clif" RUSTDOC="/checkout/obj/build/aarch64-unknown-linux-gnu/stage2-tools/cg_clif/dist/rustdoc-clif" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage0/bin/cargo" "test" "--manifest-path" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2-tools/cg_clif/build/sysroot_tests/Cargo.toml" "--target-dir" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2-tools/cg_clif/build/sysroot_tests_target" "--locked" "--target" "aarch64-unknown-linux-gnu" "-p" "coretests" "-p" "alloctests" "--tests" "--" "-q" exited with status ExitStatus(unix_wait_status(25856))
Command `/checkout/obj/build/aarch64-unknown-linux-gnu/stage0/bin/cargo run --target aarch64-unknown-linux-gnu -Zbinary-dep-depinfo -j 4 -Zroot-dir=/checkout --locked --color always --release --manifest-path /checkout/compiler/rustc_codegen_cranelift/build_system/Cargo.toml -- test --download-dir /checkout/obj/build/cg_clif_download --out-dir /checkout/obj/build/aarch64-unknown-linux-gnu/stage2-tools/cg_clif --no-unstable-features --use-backend cranelift --sysroot llvm --skip-test testsuite.extended_sysroot [workdir=/checkout/compiler/rustc_codegen_cranelift]` failed with exit code 1
Created at: src/bootstrap/src/core/build_steps/test.rs:3502:29
Executed at: src/bootstrap/src/core/build_steps/test.rs:3562:26

Command has failed. Rerun with -v to see more details.
Bootstrap failed while executing `--stage 2 test --skip tests --skip coverage-map --skip coverage-run --skip library --skip tidyselftest`
Build completed unsuccessfully in 0:13:05
  local time: Sun Aug 24 21:25:20 UTC 2025
  network time: Sun, 24 Aug 2025 21:25:21 GMT
##[error]Process completed with exit code 1.