No error returned when docker Image fails to build

[cmaerz]

Hey!

Currently there application doesn't stop with a error, when the Docker image fails to build. That doesn't really matter in local CLI, but my Gitlab Pipeline doesn't recognize that sth failed and just assumes that everything is ok.

[KollaAdithya]

Hi @cmaerz !

Just to re confirm , The docker build is failing in your local Cli, But in your Gitlab pipeline docker build is successful. Did I understand this correctly?

The screenshot that you pasted is it from the Gitlab logs?

[cmaerz]

Hey! Nope. The build fails everywhere (It should fail, cause there is an error in the Image), but the copilot svc depoly command doesn‘t Exit with an Error, so gitlab CI doesn‘t recognize it failed and assumes the deployment was ok. :) Adithya Kolla ***@***.***> schrieb am Mi. 21. Juni 2023 um 18:20:

Hi @cmaerz <https://github.com/cmaerz> ! Just to re confirm , The docker build is failing in your Cli, But in your Gitlab pipeline docker build is successful. Did I understand this correctly? The screenshot that you pasted is it from the Gitlab logs? — Reply to this email directly, view it on GitHub <#5018 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALJEERFSGDU4O5BITDUWFLXMMNORANCNFSM6AAAAAAZOVBB7M> . You are receiving this because you were mentioned.Message ID: ***@***.***>

--

…

______________________ *IT Solutions März* Christian März *Geschäftsführer* Waldstraße 31 93326 Abensberg Tel.: +49 (0) 160 96 65 12 70 Steuernummer: 126/247/10485 Ust.ID.: DE270029289 Hinweis: Dies ist eine vertrauliche Nachricht und nur für den Adressaten bestimmt. Es ist nicht erlaubt, diese Nachricht zu kopieren oder Dritten zugänglich zu machen. Sollten Sie irrtümlich diese Nachricht erhalten haben, bitte ich um Ihre Mitteilung per E-Mail oder der oben angegebenen Telefonnummer.

[KollaAdithya]

Hi @cmaerz !
Which Version of the Copilot are you using?

[cmaerz]

I use 1.28

[KollaAdithya]

Hey @cmaerz !
I tried to reproduce your issue! Unfortunately i could not.
I used AWS Code Pipeline created by Copilot with source as Github. Intentionally I provided a dockerfile with an error in it. Then the build stage of pipeline fails with a docker build error .

[kzkn]

what is the version your docker? Is buildx installed?
I had same issue, it was caused by I want to use buildkit for image building but buildx was not installed on the build environment.

[KollaAdithya]

Hello @kzkn

Just a couple of questions and also curious to know

1. How do you use buildx for image building
2. Can you also provide logs and also more info on what exactly is happening, if you do not have buildx installed.

[kzkn]

@KollaAdithya

How do you use buildx for image building

I use CodeBuild for building images. For buildx I've added the below code to buildspec.yml

env:
  variables:
    DOCKER_BUILDKIT: 1

Can you also provide logs and also more info on what exactly is happening, if you do not have buildx installed.

I got the problem when running copilot svc package on CodeBuild with image aws/codebuild/amazonlinux2-aarch64-standard:3.0 which has no docker buildx .

copilot svc package 's log:

Login Succeeded

·[?25lBuilding your container image "xxxxx": docker build -t xxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/xxxx/xxxx:latest -t xxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/xxxxx/xxxxx:xxxxx-xxxx-xxxx-staging --platform linux/arm64 --label com.aws.copilot.image.builder=copilot-cli --label com.aws.copilot.image.container.name=xxxx --label com.aws.copilot.image.version=v1.28.0 /codebuild/output/src739073975/src -f /codebuild/output/src739073975/src/Dockerfile

upload resources required for deployment for xxxx: build and push the image "xxxx": build Dockerfile at /codebuild/output/src739073975/src/Dockerfile: building image: exit status 1
Cloudformation stack and config files were not generated. Please check build logs to see if there was a manifest validation error.

I got the below log when I run docker build directly on the same environment:

ERROR: BuildKit is enabled but the buildx component is missing or broken.
       Install the buildx component to build images with BuildKit:
       https://docs.docker.com/go/buildx/

[Lou1415926]

@cmaerz Hey there!

but my Gitlab Pipeline doesn't recognize that sth failed and just assumes that everything is ok.

I think your GitLab CI is ignoring the exit code of 2; here is what I found out -

I ran copilot deploy with an erroneous Dockerfile, getting the same execute svc deploy: upload deploy resources for service web: build and push the image "web": build Dockerfile at <redacted>/Dockerfile: building image: exit status 1 as you did. Then I ran echo $?, which echos the exit code from copilot deploy, and got 1, which is an expected non-zero exit code.

I saw you had make deploy, so I created a Makefile and then ran make deploy && echo $?, the exit code this time is 2.

Your make process was wrapping the error returned from copilot deploy with an exit code of 2, which I guess is not recognized by GitLab CI as it exits on 1 only.

[Lou1415926]

@kzkn Hopefully aws/aws-codebuild-docker-images#642 gets merged soon so that BuildKit is integrated! Do you have workarounds right now? Anything I can help with 👀 ?

[kzkn]

@Lou1415926
Thanks. I use a workaround like this: aws/aws-codebuild-docker-images#653 (comment) . Now it's going well 😄

However, I think that what I have reported is only one example of this issue: No error returned when docker Image fails to build . I think copilot svc package should output the following error message that docker build outputs.

ERROR: BuildKit is enabled but the buildx component is missing or broken.
       Install the buildx component to build images with BuildKit:
       https://docs.docker.com/go/buildx/

If this had been output, it would have taken less time to investigate the problem.

[kzkn]

probably the same issue: #5124

[oiekjr]

I've been using aws/codebuild/amazonlinux2-aarch64-standard:3.0 as my build image and encountered the same issue.
However, I was able to circumvent the problem by modifying pipelines/*/buildspec.yml.

version: 0.2
phases:
  install:
    runtime-versions:
      docker: 23
      ruby: 3.2
      nodejs: 18
    commands:
      - echo "cd into $CODEBUILD_SRC_DIR"
      - cd $CODEBUILD_SRC_DIR
      # Download the copilot linux binary.
      - wget -q https://ecs-cli-v2-release.s3.amazonaws.com/copilot-linux-arm64-v1.30.1 -O copilot-linux
      - chmod +x ./copilot-linux
+     # Download Docker Buildx plugin.
+     - wget -q https://github.com/docker/buildx/releases/download/v0.11.2/buildx-v0.11.2.linux-arm64 -O docker-buildx
+     - chmod +x ./docker-buildx
+     - mkdir -p ~/.docker/cli-plugins
+     - mv docker-buildx ~/.docker/cli-plugins/docker-buildx

I hope this helps someone.