Docker image for building for aws lambda ARM architecture

[SpencerWhitehead7]

Perhaps this repo is the wrong place to ask for this, but I think this docker image is only for building for the AWS lambda x86 architectures. AWS also offers lambdas based on ARM architecture. Would a repo like this for a docker image for building for ARM make sense for rust-serverless?

[zamazan4ik]

As a base image, we use this one: https://github.com/rust-serverless/lambda-rust/blob/master/Dockerfile#L1

So, we need support from AWS side. Does AWS provide an ARM image?

I am interested in ARM support too.

[SpencerWhitehead7]

I'm quite new to this and unfortunately I don't know what AWS provides, but I found this repo https://github.com/aws/aws-lambda-base-images/tree/provided . Note that all the actual images are on branches. It looks like https://github.com/aws/aws-lambda-base-images/tree/provided.al2 might be what we need? It's the same .al2 suffix and there are subfolders for arm64 and x86_64.

It's a repo, not a set of actual images, but this section of the readme

What we're doing here
As soon as new AWS Lambda base images are available, an automated process snapshots the layers and configuration used to create these images and force-pushes them to this repository.

For examples, please see other branches in this repository.

Committed alongside the Dockerfiles in the branches are the tarballs, which balloon the repository size. Thus, we force-push branches that contain the tarballs.

Although we force-push the files away, the older versions of our images remain present on DockerHub and Amazon ECR.

makes it sound like the images are publicly available somewhere. Is that enough to tell if this is the right track? Would you know how to find the actual images from here if it is?

It sounds like you can build the images "from source" using that repo, but then you'd have to keep it up to date to track with them until the end of time vs just using a latest image published somewhere.

[zamazan4ik]

Thanks for the links! I think we can at least try to support simple image building. However, since our CI also does some testing - we need to run our tests on CI too (or just published untested ARM images). We need some time to investigate it.

[SpencerWhitehead7]

Sure, no problem, I don't have an urgent need for this or anything, I just wanted to make the suggestion.

[tysg]

Looks like AWS just released the arm64 base image on ECR: https://gallery.ecr.aws/lambda/provided . Maybe we can try a build with this image public.ecr.aws/lambda/provided:al2-arm64.

I tried building with the ARM64 image and seems like everything works just fine.

diff --git a/Dockerfile b/Dockerfile
index 78d4280..9b99d05 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,4 +1,4 @@
-FROM public.ecr.aws/lambda/provided:al2
+FROM public.ecr.aws/lambda/provided:al2-arm64
 
 ARG RUST_VERSION=1.58.0
 RUN yum install -y jq openssl-devel gcc zip

and running make build test

👌   15 tests passed.

[zamazan4ik]

@tysg Do you want to create a PR for adding ARM64-based Lambda docker images? If not, I will do it.

[tysg]

@tysg Do you want to create a PR for adding ARM64-based Lambda docker images? If not, I will do it.

I’m not sure how to base off a new image alongside the current one, so please go ahead!

[zamazan4ik]

Added it to master here: eb5620e

Beware - now we are not running CI for ARM64 since GitHub Actions doesn't provide ARM64 base images yet. Maybe we can try to fix it with a smth like qemu ...

[zamazan4ik]

By the way, it can be interesting for us: tokio-rs/tokio#4450

This is a ARM-based CI service. More info is here: https://cirrus-ci.org/faq/

[CumpsD]

Looking forward to see these -arm images pushed to Docker hub as well! :)

[0xfourzerofour]

Any ideas on when this image will be posted to dockerhub? Would be awesome for a project I am working on :)

[zamazan4ik]

@tysg when you will confirm that ARM Docker image works as expected, could you please post a comment here? And I will close the issue as resolved. Thanks in advance!

[tysg]

I modified the test/test.sh script to pull to arm64 image and managed to pass all the tests. So I do believe that the image can successfully compile ARM binaries, since I observe Docker on my Mac spins up qmeu-system-aarch64 processes during the test. And also the result:

👌   15 tests passed.

However, I don't think the test script fully does what it intend to: using either provided:al2 or provided:al2-arm64 can pass the test, which really baffles me.

@@ -91,14 +91,14 @@ verify_packaged_application() {
 
     if [ "$PROJECT" = "${HOOKS}" ]; then
         docker build -t mylambda:"${TSFRACTION}" -f- . <<EOF
-FROM public.ecr.aws/lambda/provided:al2
+FROM public.ecr.aws/lambda/provided:al2-arm64
 COPY bootstrap ${LAMBDA_RUNTIME_DIR}
 COPY output.log ${LAMBDA_TASK_DIR}
 CMD [ "function.handler" ]
 EOF
     else
         docker build -t mylambda:"${TSFRACTION}" -f- . <<EOF
-FROM public.ecr.aws/lambda/provided:al2
+FROM public.ecr.aws/lambda/provided:al2-arm64
 COPY bootstrap ${LAMBDA_RUNTIME_DIR}
 CMD [ "function.handler" ]
 EOF

I don't have enough capacity to debug this for now. Maybe @CumpsD @SpencerWhitehead7 @joshpauline can test it with your Lambda apps and comment here if it works?

[0xfourzerofour]

I tested this image on my M1 pro this morning and it seemed to compile fine however I was hit with this error.

fork/exec /var/task/bootstrap: no such file or directory"

The bootstrap artifact cannot be found for some reason. do any of yoy know a fix for this issue?

  "serverless": "^3.7.1"
  "serverless-rust": "^0.3.8"

  rust:
    dockerImage: "rustserverless/lambda-rust"
    dockerTag: "latest-arm64"

@tysg

[tysg]

@joshpauline sorry I haven't used the serverless CLI before.