Build multi-arch images based on Debian #604
Conversation
cimnine
added
the
enhancement
cimnine
force-pushed
the
Buildx
branch
4 times, most recently
from
9b70e13 to
aef2e07
Compare
|
could be worth exploring https://github.com/linuxserver/docker-netbox arm build |
|
What is left to require a review? What exactly isn’t possible about building this image on alpine? |
The ARM bit needs testing. And currently we don't have ARM infra to test it besides emulators. The Debian bit must be communicated well as it's a breaking change. Wiki pages will need to be updated as well. Because it's such a big change, we will probably time it so that it's release falls together with a new minor of NetBox itself.
At least on my machine, the ARM build would not succeed if done with Alpine. That is AFAICT because some of the Python modules require native code. As Arch Linux uses musl-libc, most of that native code must be recompiled. On GNU libc Linuxes (which most are), many Python modules provide pre-compiled versions of their native code. This speeds up the build process significantly and also resolves some weird bugs with musl-libc and some Python modules' native code when cross-compiled for ARM. |
|
Understood. Is there a tag availble yet or should I build this myself to test? |
- Pinned unit version to prevent suprises with configuration changes - Added psql client for manage.py dbshell - Set LANG to "C.UTF8" to correctly read configuration files
Improved compatibility with other deployment methods.
|
The failing arm64 tests seem to have this problem: psycopg/psycopg2#1360 |
I believe your VM is arm64 as well. It takes long on GitHub because there it's an amd64 architecture that emulates arm64 using QEMU. |
|
If you need arm native server resources for build and test, you can get access to ARM native development resources here: https://osuosl.org/services/aarch64/request_hosting/ |
|
Hey guys; I've just opened a PR with a similar goal (ARM64 docker image) but different approach, feel free to leave some feedback #664 |
| postgresql-13 \ | ||
| python3-dev \ | ||
| python3-pip \ | ||
| python3-venv \ |
There was a problem hiding this comment.
| python3-venv \ | |
| python3-venv \ | |
| libpq-dev \ | |
| libssl-dev \ |
There was a problem hiding this comment.
I'll test as well shortly. @cimnine can you please make the changes to test via CI?
There was a problem hiding this comment.
Did you remove the psycopg2-binary==2.9.3 line from the Netbox requirements.txt or is psycopg2==2.9.3 simply added as an additional dependency?
There was a problem hiding this comment.
I didn't need to remove it from the netbox requirements.
|
Sorry, wrong button clicked 😔
If you have time to spend on it, sure. This PR here is now outdated. So decide first if you want to start your work off of this one or if you just take it for reference and start again from Also I believe this PR does not yet build truly multi-arch images. It builds an image for x86 and one for arm but no combined manifest. |
|
The current develop branch can be build on ARM64 and all tests pass. So we don't need to switch do Debian if we want an ARM64 image. |
|
@tobiasge - That's fair, I'm just firmly in the camp of Alpine pretty much never being worth the hassle, especially for python projects as manylinux wheels are still in short supply. I don't mind giving it a shot make the PR up-to-date, but if the goal is just ARM support, I don't mind just forking the project based on the debian parts of this PR to fit my needs. The only other maintained image seems to be linuxserver/netbox, which is also Alpine sadly. |
Isn't that just increasing the build time? And that shouldn't have a very big impact or do you build the image that often?
I'm not opposed to a Debian image (I did investigate it also), I just want too understand what the arguments against Alpine are. |
It's a small annoyance and a waste of resources.
Spotty wheel support and musl is just plain slower than glibc. cimine also listed his reasons in the PR text. It's probably just personal preference, but I value build time and performance more than a tiny bit of saved storage. Our other internal images use debian as a base also. I think cimine adequately explained the reasoning for the PR and again, I'm really just looking for clarification regarding the plan so I can decide if forking for my own use is necessary. |
|
Yes, I think these are valid arguments and we still have this #539 open. |
|
I have rebased my branch debian-based onto the current develop branch and added With the changes on this branch I could build and successfully test the image on the following systems: On my branch the CI workflow is not changed yet. I think we can do this in a second step. |
|
@tobiasge - Tested a bunch on x86_64 where it works great like expected. I'll test on my m1 macbook air when I get the chance. Thanks for taking the time! |
FWIW I built and tested this branch on a Mackbook M1 Pro and it works great! |
|
With tests looking good, what are our next steps? |
|
I have published images based on my PR #775 in this Docker organisation for further testing. |
|
Just an update. Everything works well for me on my m1 macbook. Thanks @tobiasge |
|
+1 currently running @tobiasge image in my cluster and it's working well :) |
|
Superseded by #797 |
8 participants
Related Issues: #468, #539
New Behavior
This PR implements multi-arch builds for NetBox Docker. It does this by leveraging the idea and the work of @tobiasge to switch to the Debian distribution for NetBox Docker.
To achieve this, this PR introduces several changes:
docker buildcompatibility shim to allow more advanced build options-ldapimage is no longer required.Contrast to Current Behavior
docker buildinfrastructure.amd64.Discussion: Benefits and Drawbacks
Currently, we use a build system that is compatible with the regular
docker buildcommand. This does local caching for each build-step of the Dockerfile. But it does not allow build caches to be shared between systems. And since every GitHub Action starts with a fresh system, the regular build cache can not be leveraged. The new buildx build system also allows us to build Docker images forlinux/arm64instead of 'just'linux/amd64.Switching to Debian will have more consequences for our users though. The negligible consequence, IMO, is the increased size of the Docker image by a few megabytes. The more impactful consequence is that people, which build upon our image (e.g. for plugins) will need to rewrite their Dockerfiles if they rely on Alpine Linux infrastructure themselves (i.e. if they need to install further dependencies via
apk).But the switch to Debian brings us two advantages: First, the build is much faster. This makes it easier to tweak our Dockerfile and test new ideas. It also uses less CPU resources on the shared GitHub Actions infrastructure.
The second advantage is that it was not possible to get to a working arm64 Docker image using the Alpine Linux as basis – at least not with reasonable effort.
The change to Debian is certainly a breaking change which would demand an increase of the major version number of our project.
Changes to the Wiki
Examples that rely on Alpine Linux need to be adjusted.
Proposed Release Note Entry
NetBox Docker's base image has changed from Alpine Linux to Debian
This change results in a better build performance and less incompatibilities because of musl libc. The one drawback is a slightly larger Docker image, which we believe to be a tolerable tradeoff.
NetBox Docker is available for arm64 CPUs
Now you can run your NetBox Docker on your Raspberry Pi, your NAS or your AWS Graviton instances.
Double Check
developbranch.