LLVM submodule isn't checked out when building riscv64gc-unknown-linux-musl target

[catamorphism]

I cloned the Rust repo with the intention of building rustc for cross-compilation to the riscv64gc-unknown-linux-musl target. I did the following:

1. $x.py setup - selected the "codegen" option
2. Edited the config.toml file to append:

 [target.riscv64gc-unknown-linux-musl]
 ar = "riscv64-unknown-linux-musl-ar"
 linker = "riscv64-unknown-linux-musl-gcc"
 cc = "riscv64-unknown-linux-musl-gcc"
 cxx = "riscv64-unknown-linux-musl-g++"
 musl-root = "/home/tjc/riscv64-linux-musl-cross/riscv64-linux-musl"

3. ./x.py build --target x86_64-unknown-linux-gnu --target riscv64gc-unknown-linux-musl

This failed; the end of the output is as follows:

 Building stage1 library artifacts (x86_64-unknown-linux-gnu) 
[snip]
 Finished release [optimized] target(s) in 43.27s
Building crtbegin.o and crtend.o
running: "riscv64-unknown-linux-musl-gcc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-march=rv64gc" "-mabi=lp64d" "-mcmodel=medany" "-std=c11" "-DCRT_HAS_INITFINI_ARRAY" "-DEH_USE_FRAME_REGISTRY" "-o" "/home/tjc/rust2/build/riscv64gc-unknown-linux-musl/native/crt/crtbegin.o" "-c" "/home/tjc/rust2/src/llvm-project/compiler-rt/lib/crt/crtbegin.c"
cargo:warning=cc1: fatal error: /home/tjc/rust2/src/llvm-project/compiler-rt/lib/crt/crtbegin.c: No such file or directory
cargo:warning=compilation terminated.
exit status: 1


error occurred: Command "riscv64-unknown-linux-musl-gcc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-march=rv64gc" "-mabi=lp64d" "-mcmodel=medany" "-std=c11" "-DCRT_HAS_INITFINI_ARRAY" "-DEH_USE_FRAME_REGISTRY" "-o" "/home/tjc/rust2/build/riscv64gc-unknown-linux-musl/native/crt/crtbegin.o" "-c" "/home/tjc/rust2/src/llvm-project/compiler-rt/lib/crt/crtbegin.c" with args "riscv64-unknown-linux-musl-gcc" did not execute successfully (status code exit status: 1).


Build completed unsuccessfully in 0:02:32

After I executed git submodule update --init and retried the build command, the LLVM submodule was checked out and the build was able to make progress. It seems like this should be done automatically. I'm not sure if the issue is specific to this target or if it's happening with other cross-compilation targets.

[jyn514]

Hmm. This is some interaction between download-ci-llvm=true and cross-compiling, but I'm not sure exactly what — it should be a no-op unless you're building for the host.

As a workaround you can set download-ci-llvm=false and that should handle the submodule automatically.

[jyn514]

(Note that download-ci-llvm=true is implied by profile = codegen.)

[jyn514]

Ok, it looks like we only try to build crtbegin.o for musl targets:

rust/src/bootstrap/compile.rs

Lines 238 to 270 in 8d321f7

	/// Copies third party objects needed by various targets for self-contained linkage.
	fn copy_self_contained_objects(
	builder: &Builder<'_>,
	compiler: &Compiler,
	target: TargetSelection,
	) -> Vec<(PathBuf, DependencyType)> {
	let libdir_self_contained = builder.sysroot_libdir(*compiler, target).join("self-contained");
	t!(fs::create_dir_all(&libdir_self_contained));
	let mut target_deps = vec![];
	// Copies the libc and CRT objects.
	//
	// rustc historically provides a more self-contained installation for musl targets
	// not requiring the presence of a native musl toolchain. For example, it can fall back
	// to using gcc from a glibc-targeting toolchain for linking.
	// To do that we have to distribute musl startup objects as a part of Rust toolchain
	// and link with them manually in the self-contained mode.
	if target.contains("musl") {
	let srcdir = builder.musl_libdir(target).unwrap_or_else(|| {
	panic!("Target {:?} does not have a \"musl-libdir\" key", target.triple)
	});
	for &obj in &["libc.a", "crt1.o", "Scrt1.o", "rcrt1.o", "crti.o", "crtn.o"] {
	copy_and_stamp(
	builder,
	&libdir_self_contained,
	&srcdir,
	obj,
	&mut target_deps,
	DependencyType::TargetSelfContained,
	);
	}
	let crt_path = builder.ensure(llvm::CrtBeginEnd { target });
	for &obj in &["crtbegin.o", "crtbeginS.o", "crtend.o", "crtendS.o"] {

So I think we would have checked out the submodule if we were doing a full LLVM build, but because we're only building the CRT it got missed when download-ci-llvm was implemented. I think the simplest fix is to call builder.update_submodule at the start of CrtBeginEnd::run, like we do for the llvm::Llvm step:

rust/src/bootstrap/llvm.rs

Lines 1077 to 1097 in f98a271

	impl Step for CrtBeginEnd {
	type Output = PathBuf;
	fn should_run(run: ShouldRun<'_>) -> ShouldRun<'_> {
	run.path("src/llvm-project/compiler-rt/lib/crt")
	}
	fn make_run(run: RunConfig<'_>) {
	run.builder.ensure(CrtBeginEnd { target: run.target });
	}
	/// Build crtbegin.o/crtend.o for musl target.
	fn run(self, builder: &Builder<'_>) -> Self::Output {
	let out_dir = builder.native_dir(self.target).join("crt");
	if builder.config.dry_run() {
	return out_dir;
	}
	let crtbegin_src = builder.src.join("src/llvm-project/compiler-rt/lib/crt/crtbegin.c");
	let crtend_src = builder.src.join("src/llvm-project/compiler-rt/lib/crt/crtend.c");

[jyn514]

llvm::Libunwind looks like it also has the same bug:

rust/src/bootstrap/llvm.rs

Lines 1143 to 1167 in f98a271

	impl Step for Libunwind {
	type Output = PathBuf;
	fn should_run(run: ShouldRun<'_>) -> ShouldRun<'_> {
	run.path("src/llvm-project/libunwind")
	}
	fn make_run(run: RunConfig<'_>) {
	run.builder.ensure(Libunwind { target: run.target });
	}
	/// Build linunwind.a
	fn run(self, builder: &Builder<'_>) -> Self::Output {
	if builder.config.dry_run() {
	return PathBuf::new();
	}
	let out_dir = builder.native_dir(self.target).join("libunwind");
	let root = builder.src.join("src/llvm-project/libunwind");
	if up_to_date(&root, &out_dir.join("libunwind.a")) {
	return out_dir;
	}
	builder.info(&format!("Building libunwind.a for {}", self.target.triple));

as well as dist::Src:

rust/src/bootstrap/dist.rs

Lines 896 to 915 in f98a271

	/// Creates the `rust-src` installer component
	fn run(self, builder: &Builder<'_>) -> GeneratedTarball {
	let tarball = Tarball::new_targetless(builder, "rust-src");
	// A lot of tools expect the rust-src component to be entirely in this directory, so if you
	// change that (e.g. by adding another directory `lib/rustlib/src/foo` or
	// `lib/rustlib/src/rust/foo`), you will need to go around hunting for implicit assumptions
	// and fix them...
	//
	// NOTE: if you update the paths here, you also should update the "virtual" path
	// translation code in `imported_source_files` in `src/librustc_metadata/rmeta/decoder.rs`
	let dst_src = tarball.image_dir().join("lib/rustlib/src/rust");
	let src_files = ["Cargo.lock"];
	// This is the reduced set of paths which will become the rust-src component
	// (essentially libstd and all of its path dependencies).
	copy_src_dirs(
	builder,
	&builder.src,
	&["library", "src/llvm-project/libunwind"],

[KittyBorgX]

@rustbot claim

[KittyBorgX]

Does fixing llvm::Libunwind dist::Src also come under the scope of this issue? I'd be happy to fix that as well.

[jyn514]

@KittyBorgX sure thing, feel free to fix all three at once :)

[KittyBorgX]

@jyn514 Do I pause work on this till #110074 gets resolved? Since it's implementing the workaround you mentioned as a solution

[jyn514]

@KittyBorgX no, this fix still makes sense - not everyone is using the codegen profile without changes.