faas-cli build error with dep

[embano1]

faas-cli build ... fails to build the image when the function contains external libraries vendored with dep.

Steps to reproduce:

faas template pull https://github.com/openfaas-incubator/golang-http-template
faas-cli new --lang golang-http test-fn
cd test-fn
# now create a sample function (handler.go) using an external library, example below
# make sure when you save that the openfaas-incubator... import is not removed, e.g. due to go imports formating
dep init # assumption here is that dep should be run from within the handler folder (?)
tree -L 2 vendor
vendor
└── github.com
    ├── openfaas-incubator
    └── pkg
# dep correctly did its job
cd ..
faas-cli build -f test-fn.yml

Then it fails at Dockerfile step 7:

Step 7/17 : RUN CGO_ENABLED=0 GOOS=linux     go build --ldflags "-s -w" -a -installsuffix cgo -o handler . &&     go test $(go list ./... | grep -v /vendor/) -cover
 ---> Running in eb8face14c7b
# handler
./main.go:37:39: cannot use req (type "handler/vendor/github.com/openfaas-incubator/go-function-sdk".Request) as type "handler/function/vendor/github.com/openfaas-incubator/go-function-sdk".Request in argument to function.Handle
The command '/bin/sh -c CGO_ENABLED=0 GOOS=linux     go build --ldflags "-s -w" -a -installsuffix cgo -o handler . &&     go test $(go list ./... | grep -v /vendor/) -cover' returned a non-zero code: 2
2019/01/17 22:23:33 ERROR - Could not execute command: [docker build -t test-fn:latest .]

Took me a while to figure out how to work around that. Before faas-cli build remove the folder openfaas-incubator in the handler.go vendor folder. Then build works fine (as it will pull in the dependency from the template).

Example function:

package function

import (
	"fmt"
	"net/http"

	"github.com/openfaas-incubator/go-function-sdk"
	"github.com/pkg/errors"
)

// Handle a function invocation
func Handle(req handler.Request) (handler.Response, error) {
	var err error
	err = errors.Wrap(err, "hit a bug")

	message := fmt.Sprintf("Hello world, input was: %s", string(req.Body))

	return handler.Response{
		Body:       []byte(message),
		StatusCode: http.StatusOK,
	}, err

}

cc/ @alexellis

[alexellis]

Good find Michael.

This seems valid, although I didn't run into it in my usage of the template with @ewilde where we also vendored dependencies for the Monzo demos. I wonder what was different?

My suggestion as a workaround would be to try to add a constraint whereby you skip / omit the sdk dependency in the Gopkg.toml or delete it manually as you are doing now.

We need to figure out whether it's possible to have a package with its own vendor folder and then another sub-folder with more vendored folders under that. We get away with this in the "normal" Go template by not having an SDK at all.

[alexellis]

Derek add label: help wanted

[alexellis]

I got a similar error with your example:

# handler
./main.go:37:39: cannot use req 
(type "handler/vendor/github.com/openfaas-incubator/go-function-sdk".Request) 
as type 
"handler/function/vendor/github.com/openfaas-incubator/go-function-sdk".Request
 in argument to function.Handle

[alexellis]

I wonder if we can edit the Dockerfile so that the vendor folder gets moved one level up ready for the build?

[embano1]

Couple of thoughts on this:

• dep does not support nested vendor folders (support nested vendor/ dirs golang/dep#985)
• moving all dependencies (incl. openfaas-sdk) into the function subfolder won't compile (main.go can't resolve openfaas-sdk dependencies)
• moving all dependencies (incl. those from the function handler.go) into a vendor folder in the main.go build context folder (i.e. where Dockerfile resides) works

User story: As a template user I want to specify packages and versions in Gopkg.toml. faas-cli build should be able to merge this with template specific dependencies (e.g. openfaas-sdk).

faas-cli build only seems to copy function folder into the build folder (merged with the template logic). So the questions are:

• Where should user-specific dependency lock information be stored?
• How can we merge the individual Gopkg.toml files in the best way?

A workaround could be:

1. Run dep init in the root folder of the function (where the stack file, etc are located as well)
2. dep will create a vendor folder there (incl. openfaas-sdk deps)
3. Document the supported SDK pkg version to be used for the template (currently it's master anyways)
4. User is free to modify Gopkg.toml as per his/her needs (besides SDK version of course)
5. faas-cli would copy Gopkg.* to the function build context and run dep ensure -vendor-only
6. This will create a vendor folder in the root build (context) folder of the function

Not sure how much work that is for the faas-cli tool or whether there's a better/cleaner approach.

[SidGrundfos]

Hello.
I have an import as below -
import (
"log"

"github.com/nats-io/nats.go"
"github.com/nats-io/stan.go"

)
when I run go install publish-msg-go/handler.go it compiles ok
but when faas-cli build it gives me error

function/handler.go:6:2: cannot find package "github.com/nats-io/nats.go" in any of:
/usr/local/go/src/github.com/nats-io/nats.go (from $GOROOT)
/go/src/github.com/nats-io/nats.go (from $GOPATH)
function/handler.go:7:2: cannot find package "github.com/nats-io/stan.go" in any of:
/usr/local/go/src/github.com/nats-io/stan.go (from $GOROOT)
/go/src/github.com/nats-io/stan.go (from $GOPATH)

[alexellis]

I've been able to use nested folders with dep, Incan share an example if you want?

Ideally you should be looking at Go modules now. Try the example that I added to the readme last week.

[SidGrundfos]

Actually I am trying to test with NATS streaming and want to execute a small piece of code to send/publish messages. I have installed these modules in $GOPATH. when I build locally it builds OK but via faas-cli it fails. Any suggestion how I can make it work?

[LucasRoesler]

@SidGrundfos can you push your test function to a new git repo and share that? It will be easier to debug.

Also, for your specific case, have you seen the test functions in the nats-connector repo? https://github.com/openfaas-incubator/nats-connector/tree/master/contrib/test-functions
And the "try it out" instructions https://github.com/openfaas-incubator/nats-connector#try-it-out

Even if you aren't using the nats connector, perhaps you can checkout those test functions as a reference for your situation. It includes a publish function

[SidGrundfos]

Hello its is just a very small piece of code.

package function

import (
"log"
"fmt"
"net/http"

"github.com/nats-io/nats.go"
"github.com/nats-io/stan.go"

)

// Handle a function invocation
func Handle() {
nc, err := nats.Connect("nats://example-nats:4222")
if err != nil {
log.Fatal(err)
}
sc, err := stan.Connect("example-stan", "client-123", stan.NatsConn(nc))
if err != nil {
log.Fatal(err)
}
sc.Publish("hello", []byte("one"))
sc.Publish("hello", []byte("two"))
sc.Publish("hello", []byte("three"))

sc.Subscribe("hello", func(m *stan.Msg) {
	log.Printf("[Received] %+v", m)
}, stan.DeliverAllAvailable())

select {}

}

I have made 2 functions using c# to send and receive functions in openfaas which works fine.

[alexellis]

cc @LucasRoesler