Continuous builds for DragonFlyBSD

[mneumann]

Would be nice to have continuous builds of the Rust compiler also for DragonFly, in a way that it also gets added to the src/snapshots.txt file. Are there enough resources to do this, and if yes, how can I help to get this going?

As a side note, go-lang has also continuous builds for dragonfly.

[alexcrichton]

Currently this involves having an AWS bot set up which is then also hooked up to our buildmaster. We currently don't have a way for community infrastructure to hook up to the buildmaster, but I believe that's something that we've wanted for awhile now!

[mneumann]

Will you discuss this in a weekly meeting or what'd be the next steps?

[mneumann]

@alexcrichton: One year has passed :). What's the current situation regarding setting up a buildbot for DragonFly? We have Rust in the ports tree, but it needs patching of snapshots.txt and my own snapshots for building.

[steveklabnik]

Cc @edunham

On Aug 30, 2015, at 17:27, Michael Neumann [email protected] wrote:

@alexcrichton: One year has passed :). What's the current situation regarding setting up a buildbot for DragonFly? We have Rust in the ports tree, but it needs patching of snapshots.txt and my own snapshots for building.

―
Reply to this email directly or view it on GitHub.

[alexcrichton]

If you have your own hardware you'd like builds to run on you can contact us for how to set it up, but we don't have a lot of precedent in the area unfortunately.

[mneumann]

Thanks. I am trying to organize some hardware then getting back to you.

Or else, is it possible to get the neccessary changes into src/snapshots.txt and optionally upload the snapshots to static.rust-lang.org? I am hosting them myself atm, so if someone wants to build DragonFly from git, he has to know where to get an initial snapshot from.

[edunham]

@mneumann, the FreeBSD and Bitrig snaps that @dhuseby contributes are a good example of how we've been handling community builds. The way we've handled the other community builds is to have the a community member set up a buildslave for the buildbot. After a slave is set up, our buildmaster handles uploading artefacts that it creates.

PRs such as #27821 to src/snapshots.txt can provide some context on how we add snapshots -- basically, updating src/snapshots.txt is the last step to adding a new buildslave.

[dhuseby]

@mneumann I actually have plans to spin up DragonflyBSD build bots. I haven't been able to build it successfully on Dragonfly yet tho. Maybe you can help me with that? If you've got a rustc and libs built for Dragonfly, let me know, I can use that to get a buildbot set up.

I already have the hardware and the infrastructure for setting up new buildbots. I am running the bitrig and freebsd 32/64 bots already. I was planning on standing up openbsd, netbsd, and dragonflybsd ones in the near future.

[mneumann]

@dhuseby : That would be perfect! If you try the latest version from git, it should actually compile out-of-the-box, as I have uploaded a snapshot for DragonFly to static.rust-lang.org. All versions prior of that required some minor patching.

[mneumann]

@dhuseby : Note that you can't build against the llvm from ports. I think FreeBSD has a similar problem. Also note that the build will fail if you already have Rust installed.

[dhuseby]

@mneumann that's fine, rust builds it's own llvm.

[alexcrichton]

The landscape today is that we just need nightlies for new platforms to gate on new platforms. Currently this is done via cross compilation as we've basically concluded that for now we're not maintaining BSD bots.

PRs are of course always welcome over at rust-lang/rust-buildbot especially to the linux-cross image! If there's a docker image to cross compile to dragonfly, we'll happily set up nightlies for it :)

[mneumann]

@alexcrichton can you point me at some information what needs to be done to setup cross-compilation for DragonFly? If it works for FreeBSD, it shouldn't be too hard for DragonFly and I could give it a try. This would probably also mean that we gain support in rustup, right?

[alexcrichton]

@mneumann sure!

From a high level, compilation of these platforms works not through bootstrapping but rather cross-compiling. The linux-cross container contains all the bits and pieces for this, and we use the new rustbuild build system to cross-compile via ./configure --host=new-platform. Those then make their ways to nightlies which enable Cargo builds and rustup builds (also using the same linux-cross container). Eventually we've got a full rust compiler package, and that can itself then later be used for a native bootstrap!

The linux-cross Dockerfile is a little sprawling, but it's basically just got a section for each platform it can cross-compile to. For example FreeBSD/NetBSD happen here, with the real meat being their corresponding scripts. I suspect that a DragonFly script would look pretty similar, right?

To get it work, the steps would look like:

1. Configure the linux-cross container for cross-compilation to DragonFly, checking in scripts and such.

2. Boot up the container and configure it to build a cross-compiler. You can copy a ./configure line similar to the other cross compiled hosts we have:

   ../configure --disable-valgrind --enable-optimize --enable-optimize-tests --disable-debug --enable-debug-assertions --enable-rustbuild --musl-root=/musl-i686 --build=x86_64-unknown-linux-gnu --host=x86_64-unknown-linux-gnu,arm-unknown-linux-gnueabihf
   

3. Run make and fix bugs until cross-compilation works and make dist produces a bunch of tarballs.

4. Send a PR to rust-lang/rust with any bug fixes

5. Send a PR to rust-lang/rust-buildbot with updates

From there I can take over deployment and adding an auto builder, should be pretty straightforward! Once that's all up an running it's a pretty similar set of steps to get Cargo/rustup building as well. The tricky part with these two is OpenSSL, but it may not be that hard in the long run. We can cross that bridge when we get there :)

[mneumann]

@alexcrichton thanks a lot. rust-lang-deprecated/rust-buildbot#105 contains part 1. I haven't been able to log into the container yet, because somehow the network is not correctly set up.

[alexcrichton]

Awesome! Odd that you're having problems logging into a container, I'm not sure I've ever had that kind of problem with docker ...