"lambda deploy-serverless" command tries to build image when ImageUri is set without any Metadata

[roncrush]

Describe the bug

According to the docs for "AWS::Serverless::Function" the "ImageUri" property can be set if the image has already been pushed to somewhere like ECR (https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-resource-function.html#sam-function-imageuri).

The behavior I'm seeing with the command "lambda deploy-serverless" is that it treats the "ImageUri" like a local path as if it's trying to build the image and push it.

Expected Behavior

The command should result in the function being successfully deployed by the CloudFormation stack.

Current Behavior

The following error appears:

Processing CloudFormation resource <Function Name>
Initiate packaging of <ImageUri> for resource
Directory that the field <Function Name>/CodeUri-Or-ImageUri is pointing doesn't exist

Reproduction Steps

1. Initialize the hello world sample:

sam init --package-type Image --base-image amazon/dotnet6-base

2. Manually build and deploy the Dockerfile to somewhere like ECR.
3. Update the template.yaml file to look like this (replace the <ImageUri> with the deployed image URI):

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Template to test bug with ImageUri

Globals:
  Function:
    Timeout: 10
    MemorySize: 128

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      PackageType: Image
      ImageUri: <ImageUri>

4, Run the "dotnet lambda deploy-serverless" command (will need a S3 bucket):

dotnet lambda deploy-serverless --s3-bucket "some-bucket-for-testing-bug" --s3-prefix "bug/" --stack-name "test-bug" --template "template.yaml"

Possible Solution

It seems like the packaging step should be skipped for Lambda Functions that have an ImageUri set without any metadata.

Additional Information/Context

No response

Targeted .NET platform

6.0.302

CLI extension version

Package Id	Version	Commands
amazon.lambda.tools	5.6.3	dotnet-lambda
dotnet-ef	6.0.8	dotnet-ef

Environment details (OS name and version, etc.)

macOS Monterey Version 12.6

[ashishdhingra]

@roncrush Thanks for opening the issue. I think you are affected feature request #196. Could you please check and confirm?

Thanks,
Ashish

[roncrush]

@ashishdhingra Thanks for the quick response. We actually recently updated the Amazon Lambda Tools package as part of our move from Windows to MacOS. The old version we were using was 4.3.0 and this case was working then, which is why I've reported it as a bug.

[ashishdhingra]

Hi @roncrush,

Good afternoon.

Issue appears to be reproducible. Below are the steps followed:

• Installed latest version of SAMS CLI.
• Executed command sam init --package-type Image --base-image amazon/dotnet6-base to create project named HelloWorld by choosing AWS Quick Start Templates.
• Navigated to HelloWorld/src/HelloWorld directory containing the .csproj and other files.
• In AWS Console, created a new ECR repository named helloworldtest.
  • Followed instructions in View push commands to build and push docker image.
  • From AWS console, took note of the Image Uri (which is something like <<account-id>>.dkr.ecr.us-east-2.amazonaws.com/helloworldtest:latest)
• Modified the ../../template.yaml to add the ImageUri property.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  Sample SAM Template for HelloWorld

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Timeout: 10
    MemorySize: 128

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      PackageType: Image
      ImageUri: <<account-id>>.dkr.ecr.us-east-2.amazonaws.com/helloworldtest:latest
      Events:
        HelloWorld:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /hello
            Method: get
    Metadata:
      DockerTag: dotnet6-v1
      DockerContext: ./src/HelloWorld
      Dockerfile: Dockerfile
      DockerBuildArgs:
        SAM_BUILD_MODE: run

Outputs:
  # ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
  # Find out more about other implicit resources you can reference within SAM
  # https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
  HelloWorldApi:
    Description: "API Gateway endpoint URL for Prod stage for Hello World function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
  HelloWorldFunction:
    Description: "Hello World Lambda Function ARN"
    Value: !GetAtt HelloWorldFunction.Arn
  HelloWorldFunctionIamRole:
    Description: "Implicit IAM Role created for Hello World function"
    Value: !GetAtt HelloWorldFunctionRole.Arn

• Executed the command dotnet lambda deploy-serverless --s3-bucket some-s3-bucket --s3-prefix "bug/" --stack-name "test-bug" --template ../../template.yaml

Amazon Lambda Tools for .NET Core applications (5.6.3)
Project Home: https://github.com/aws/aws-extensions-for-dotnet-cli, https://github.com/aws/aws-lambda-dotnet
        
Processing CloudFormation resource HelloWorldFunction
Initiate packaging of ./src/HelloWorld for resource HelloWorldFunction
Building Docker image for /Users/<<redacted>>/dev/repros/samnetcliextensions-imageuri-issue264/HelloWorld/src/HelloWorld/../.././src/HelloWorld
Inspecting Dockerfile to figure how to build project and docker image
Executing docker build
... invoking 'docker build', working folder '/Users/<<redacted>>/dev/repros/samnetcliextensions-imageuri-issue264/HelloWorld/src/HelloWorld/../.././src/HelloWorld, docker file /Users/<<redacted>>/dev/repros/samnetcliextensions-imageuri-issue264/HelloWorld/src/HelloWorld/../.././src/HelloWorld/Dockerfile, image name dotnet6-v1:latest'
... docker build  -f "/Users/<<redacted>>/dev/repros/samnetcliextensions-imageuri-issue264/HelloWorld/src/HelloWorld/../.././src/HelloWorld/Dockerfile" -t dotnet6-v1:latest --build-arg SAM_BUILD_MODE=run .
... docker build: #1 [internal] load build definition from Dockerfile
... docker build: #1 sha256:2ffc30f135a36a692885f3525ed911bf011a8ea1565aae75efc15224d23d49a1
... docker build: #1 transferring dockerfile: 37B done
... docker build: #1 DONE 0.0s
... docker build: #2 [internal] load .dockerignore
... docker build: #2 sha256:09532619e16bdbab5a4bbd930734381bd6248ed9e1d816ef46ccee2c46a90683
... docker build: #2 transferring context: 2B done
... docker build: #2 DONE 0.0s
... docker build: #4 [internal] load metadata for mcr.microsoft.com/dotnet/sdk:6.0
... docker build: #4 sha256:9eb4f6c3944cfcbfe18b9f1a753c769fc35341309a8d4a21f8937f47e94c712b
... docker build: #4 DONE 0.1s
... docker build: #3 [internal] load metadata for public.ecr.aws/lambda/dotnet:6
... docker build: #3 sha256:b0e709de1ce280b90bfd00976d19b961170e8b446b69d5ffe1f8022f8945df19
... docker build: #3 DONE 0.5s
... docker build: #5 [stage-1 1/2] FROM public.ecr.aws/lambda/dotnet:6@sha256:df3a25efae0d73cf367d07dfd14567adb61e2cb077177dc4dbb182cb91d2d9de
... docker build: #5 sha256:42584e7b18e9ca68e519c26d812ffb4aa20cd3a0ec5508208f92c9713a3e1fc8
... docker build: #5 DONE 0.0s
... docker build: #6 [build-image 1/9] FROM mcr.microsoft.com/dotnet/sdk:6.0@sha256:8189c53e1f0323b8beb5fa174853f413364a203f32b0b7c2488b534a3c1e76c6
... docker build: #6 sha256:87cb5ae0e817750b8601520c8fa2e443a0c6caeaf5c8fe8f7239ed7d5fae7993
... docker build: #6 DONE 0.0s
... docker build: #10 [internal] load build context
... docker build: #10 sha256:ed33d67c542c3e9f175f97682f689484c760c26e448b0209906a47c7d1d1292f
... docker build: #10 transferring context: 120B done
... docker build: #10 DONE 0.0s
... docker build: #8 [build-image 3/9] RUN mkdir /build
... docker build: #8 sha256:acdf1de403df574c2f6164764641fcb98d11d56c10185fe5668d7f8966d328ea
... docker build: #8 CACHED
... docker build: #11 [build-image 5/9] COPY Function.cs HelloWorld.csproj aws-lambda-tools-defaults.json /build/
... docker build: #11 sha256:a380f3f3bf4204e6498eafa8ed0ebc119cc5fb74f600c16856020e4fce965740
... docker build: #11 CACHED
... docker build: #9 [build-image 4/9] WORKDIR /build
... docker build: #9 sha256:635d0c7915105740c7f02872beef8b35cc5fc46faa1d4718f31c108bb49bcf34
... docker build: #9 CACHED
... docker build: #12 [build-image 6/9] RUN dotnet tool install -g Amazon.Lambda.Tools
... docker build: #12 sha256:9a143a9620bfd1d7049d00324ea642647cb5c2b7c1a91e93afee0b2d74c54236
... docker build: #12 CACHED
... docker build: #14 [build-image 8/9] RUN if [ "run" = "debug" ]; then dotnet lambda package --configuration Debug; else dotnet lambda package --configuration Release; fi
... docker build: #14 sha256:c3fe948bebdc5a87fe4a7cfc9f1d5c2bf12439f76c243f059a9d8c0be5e2a0cc
... docker build: #14 CACHED
... docker build: #15 [build-image 9/9] RUN if [ "run" = "debug" ]; then cp -r /build/bin/Debug/net6.0/publish/* /build/build_artifacts; else cp -r /build/bin/Release/net6.0/publish/* /build/build_artifacts; fi
... docker build: #15 sha256:8ddf33c6341811111b805c6883bd30175a76483f97165cb0f158df6464462083
... docker build: #15 CACHED
... docker build: #7 [build-image 2/9] RUN apt-get update && apt-get -y install zip
... docker build: #7 sha256:0e5275dcf76fc26841bd7c93fbaa32f3e86953af6086f4ff31b72e7c438f4176
... docker build: #7 CACHED
... docker build: #13 [build-image 7/9] RUN mkdir -p build_artifacts
... docker build: #13 sha256:3ca43a04992fa5a40d2306528e90ec99b3c50be2c58b7cccaa9d707327a034ec
... docker build: #13 CACHED
... docker build: #16 [stage-1 2/2] COPY --from=build-image /build/build_artifacts/ /var/task/
... docker build: #16 sha256:3ac39db0e8432e75f6a5e22e79b7bd2599a63fc83bf3b20463ce3606e7bdb93e
... docker build: #16 CACHED
... docker build: #17 exporting to image
... docker build: #17 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
... docker build: #17 exporting layers done
... docker build: #17 writing image sha256:0187e66b8f20e4127bc26ea55bfa7002272fe140c0a4080179a31aa1d1599ccc done
... docker build: #17 naming to docker.io/library/dotnet6-v1:latest done
... docker build: #17 DONE 0.0s
Fetching ECR authorization token to use to login with the docker CLI
Executing docker CLI login command
... invoking 'docker login'
... docker login: WARNING! Using --password via the CLI is insecure. Use --password-stdin.
... docker login: Login Succeeded
Creating ECR Repository dotnet6-v1
Taging image dotnet6-v1:latest with dotnet6-v1:helloworldfunction-0187e66b8f20-latest
... invoking 'docker tag'
Pushing image to ECR repository
... invoking 'docker push'
The push refers to repository [<<account-id>>.dkr.ecr.us-east-2.amazonaws.com/dotnet6-v1]
600dc3d98bcf: Pushed 
9f47e18cd643: Pushed 
6f049e498202: Pushed 
2d244e0816c6: Pushed 
c89552e947a7: Pushed 
c0d1687ac68f: Pushed 
0d91c86bcea1: Pushed 
helloworldfunction-0187e66b8f20-latest: digest: sha256:84c208d02d121807ee5c988355258813e732412ce22e30e1795e2197740d6696 size: 1788
Image <<account-id>>.dkr.ecr.us-east-2.amazonaws.com/dotnet6-v1:helloworldfunction-0187e66b8f20-latest Push Complete. 
Uploading to S3. (Bucket: some-s3-bucket Key: bug/test-bug-template-638144939575530990.yaml)
... Progress: 100%
Found existing stack: False
CloudFormation change set created
... Waiting for change set to be reviewed
Created CloudFormation stack test-bug
   
Timestamp            Logical Resource Id                      Status                                   
-------------------- ---------------------------------------- ---------------------------------------- 
3/15/2023 4:19 PM    test-bug                                 CREATE_IN_PROGRESS                      
3/15/2023 4:19 PM    HelloWorldFunctionRole                   CREATE_IN_PROGRESS                      
3/15/2023 4:19 PM    HelloWorldFunctionRole                   CREATE_IN_PROGRESS                      
3/15/2023 4:19 PM    HelloWorldFunctionRole                   CREATE_COMPLETE                         
3/15/2023 4:19 PM    HelloWorldFunction                       CREATE_IN_PROGRESS                      
3/15/2023 4:19 PM    HelloWorldFunction                       CREATE_IN_PROGRESS                      
3/15/2023 4:19 PM    HelloWorldFunction                       CREATE_COMPLETE                         
3/15/2023 4:20 PM    ServerlessRestApi                        CREATE_IN_PROGRESS                      
3/15/2023 4:20 PM    ServerlessRestApi                        CREATE_IN_PROGRESS                      
3/15/2023 4:20 PM    ServerlessRestApi                        CREATE_COMPLETE                         
3/15/2023 4:20 PM    HelloWorldFunctionHelloWorldPermissionProd CREATE_IN_PROGRESS                      
3/15/2023 4:20 PM    ServerlessRestApiDeployment47fc2d5f9d    CREATE_IN_PROGRESS                      
3/15/2023 4:20 PM    HelloWorldFunctionHelloWorldPermissionProd CREATE_IN_PROGRESS                      
3/15/2023 4:20 PM    ServerlessRestApiDeployment47fc2d5f9d    CREATE_IN_PROGRESS                      
3/15/2023 4:20 PM    ServerlessRestApiDeployment47fc2d5f9d    CREATE_COMPLETE                         
3/15/2023 4:20 PM    ServerlessRestApiProdStage               CREATE_IN_PROGRESS                      
3/15/2023 4:20 PM    ServerlessRestApiProdStage               CREATE_IN_PROGRESS                      
3/15/2023 4:20 PM    ServerlessRestApiProdStage               CREATE_COMPLETE                         
3/15/2023 4:20 PM    HelloWorldFunctionHelloWorldPermissionProd CREATE_COMPLETE                         
3/15/2023 4:20 PM    test-bug                                 CREATE_COMPLETE                         
Stack finished updating with status: CREATE_COMPLETE
   
Output Name                    Value                                             
------------------------------ --------------------------------------------------
HelloWorldFunctionIamRole      arn:aws:iam::<<account-id>>:role/test-bug-HelloWorldFunctionRole-ZJ7T05ZPLES1
HelloWorldApi                  https://uli61kpfk2.execute-api.us-east-2.amazonaws.com/Prod/hello/
HelloWorldFunction             arn:aws:lambda:us-east-2:<<account-id>>:function:test-bug-HelloWorldFunction-4oOLPbRsRYtK

As you could see from the output above, I didn't got the error reported by you. It ignored ImageUri property all together and pushed image to a new ECR repository. The reason is that per https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-resource-function.html#sam-function-imageuri, Building your application with necessary Metadata entries takes precedence over ImageUri, so if you specify both then ImageUri is ignored..

Thereafter, I removed Metadata entries and re-executed dotnet lambda deploy-serverless --s3-bucket some-s3-bucket --s3-prefix "bug/" --stack-name "test-bug" --template ../../template.yaml command. This gave the error reported by you:

Amazon Lambda Tools for .NET Core applications (5.6.3)
Project Home: https://github.com/aws/aws-extensions-for-dotnet-cli, https://github.com/aws/aws-lambda-dotnet
        
Processing CloudFormation resource HelloWorldFunction
Initiate packaging of <<account-id>>.dkr.ecr.us-east-2.amazonaws.com/helloworldtest:latest for resource HelloWorldFunction
Directory that the field HelloWorldFunction/CodeUri-Or-ImageUri is pointing doesn't exist

I think the related open feature request #196 does not document/validates the absence of Metadata properties (since it was reported as a feature request), hence is duplicate of this issue.

I would validate if this worked on Amazon.Lambda.Tools) version 4.3.0. If it does, then we would classify this as a bug instead of a feature request and take it from there.

Thanks,
Ashish

[roncrush]

Hey @ashishdhingra

As a temporary fix we ended up downgrading back to 4.3.0 and it started deploying properly. I didn't test with the template.yaml you've shown, but it did work with the template.yaml that's on the initial post. When I updated to the latest version of the tool it stopped working again.

[ashishdhingra]

I would validate if this worked on Amazon.Lambda.Tools) version 4.3.0. If it does, then we would classify this as a bug instead of a feature request and take it from there.

This appears to work in version 4.3.0 of Amazon.Lambda.Tools. Closing other open feature request #196 to track it as part of this issue.