Go modules builder fails when using local module with provisioned.AL2 runtime

[sliedig]

Description:
Using sam build in go projects with a dependency on a local module fails when the function runtime is Runtime: provided.al2

Steps to reproduce the issue:

1. create a new go application sam init --name sam-app --runtime go1.x --dependency-manager mod --app-template hello-world
2. Create a new module (here is the example I used). In a new folder called models, initialise a new module using go mod init hello-go/models

add a new go file called person.go add this code

package models

type Person struct {
	FirstName string `json:"first_name,omitempty"`
	LastName  string `json:"last_name,omitempty"`
}

func (p *Person) FullName() string {

	return p.FirstName + " " + p.LastName
}

3. Open the main.go file for the hello0world function. Import "hello-go/models" and replace the return statement with this

	person := models.Person{FirstName: "John", LastName: "Doe"}

	return events.APIGatewayProxyResponse{
		Body:       fmt.Sprintf("Hello %s, your ip address is %v", person.FullName(), string(ip)),
		StatusCode: 200,
	}, nil

4. Open the function go.mod file and add make sure the dependency is replaced to point to the local module

require (
	github.com/aws/aws-lambda-go v1.23.0
	hello-go/models v0.0.1
)

module hello-world

go 1.17

replace hello-go/models => ../models

5. Build the function sam build
6. If you run sam local invoke "HelloWorldFunction" you should see the output (with your IP address)

{"statusCode":200,"headers":null,"multiValueHeaders":null,"body":"Hello John Doe, your ip address is 127.0.0.1\n"}% 

7. Update the sam template to use Amazon Linux 2 instead of the managed runtime.

8. In the function definition replace handler and runtime properties, and the Metadata resource attribute

  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: hello-world/
      Handler: bootstrap 
      Runtime: provided.al2
      Tracing: Active # https://docs.aws.amazon.com/lambda/latest/dg/lambda-x-ray.html
      Events:
        CatchAll:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /hello
            Method: GET
      Environment: # More info about Env Vars: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#environment-object
        Variables:
          PARAM1: VALUE
    Metadata:
      BuildMethod: makefile

9. Create a new Makefile in the hello-world function directory and add

build-HelloWorldFunction:
	GOOS=linux go build -o bootstrap
	cp ./bootstrap $(ARTIFACTS_DIR)/.

10. Run sam build. The build now fails.

Observed result:

sam build
Building codeuri: /Users/sliedig/go/src/hello-go-al2/hello-world runtime: provided.al2 metadata: {'BuildMethod': 'makefile'} functions: ['HelloWorldFunction']
Running CustomMakeBuilder:CopySource
Running CustomMakeBuilder:MakeBuild
Current Artifacts Directory : /Users/sliedig/go/src/hello-go-al2/.aws-sam/build/HelloWorldFunction

Build Failed
Error: CustomMakeBuilder:MakeBuild - Make Failed: main.go:9:2: hello-go-al2/models@v0.0.1: replacement directory ../models does not exist
make[1]: *** [build-HelloWorldFunction] Error 1
make: *** [build] Error 1

Expected result:
I would expect the sam build system to correctly build my application irrespective of whether I am using go as a managed runtime or on AL2.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)
OS: Darwin x64 20.6.0
go version go1.17.1 darwin/amd64
SAM CLI, version 1.32.0

[sliedig]

Any update on this?

I have also noticed in Python projects, if you have local modules you want to share across functions that SAM build does not include those either. Seems the way to get around it is to do some layers voodoo but that seems like complete overkill for something that works natively.

[komapa]

@sliedig, the problem is with Running CustomMakeBuilder:CopySource which copies hello-world/ (your CodeUri) into a scratch directory (think a random directory under your OS temp folder) and then runs the go build commands. This means in that new scratch directory ../models does not make sense anymore as you are in a completely different directory.

I am about to open an issue myself asking why we even have the scratch path for the custom makefile builder and all the issues I see with it.