cmd/go: #cgo pkg-config: add conditional --static flag to pkg-config

[kolyshkin]

I found this case when working on Docker. Docker can be linked either statically or dynamically. For static linking, one has to specify all the libraries explicitly, including those required indirectly (i.e. by libraries that you use). For dynamic linking, one needs to specify only the libraries one uses directly. For example, when linking against libxml2 on Linux, for the case of dynamic linking you have to specify only -lxml2, while for the static linking case you have to also specify all the libraries used by libxml2 (like -lz -lm -llzma).

Fortunately, pkg-config supports this:

$ pkg-config --libs libxml-2.0
-lxml2 
$ pkg-config --libs --static libxml-2.0
-lxml2 -lz -llzma -lm 

I was expecting Go to add --static flag when the result is going to be statically linked, unfortunately it never happens. My current workaround is to do this:

// #cgo pkg-config: --static libxml-2.0

It solves my issues, but the problem with this approach is this leads to "overlinking" in the dynamic case, as described e.g. here: https://wiki.openmandriva.org/en/Overlinking_issues_in_packaging

Is it possible to implement conditionally adding --static to pkg-config invocation?

[ianlancetaylor]

It's not immediately obvious to me how we can know whether we are going to be linking statically or linking dynamically. Are you suggesting that we look for the presence of --static in the -ldflags option?

[kolyshkin]

@ianlancetaylor That is the biggest obstacle to me as I'm not an expert, that's why I filed an issue rather than sent a pull request. I looked around for ways to tell go to produce a static binary with cgo or gccgo, and it looks like they all end up with passing -static flag.

[kolyshkin]

At the very least, it makes sense to add --static flag to pkg-config unconditionally, this will greatly improve the chances of making the build work.

[kolyshkin]

One example I just came across: gosqlite package which says
#cgo LDFLAGS: -lsqlite3
If you try to build a binary with it dynamically, everything works. If you try to build it statically, it complains about missing pthread* and dlopen/dlsym/etc symbols. Let's look at pkg-config:

$ pkg-config --libs sqlite3
-lsqlite3  
$ pkg-config --libs --static sqlite3
-lsqlite3 -ldl -lpthread 

So, it's all there they way it should be. One would thing that everything that is needed to fix it would be this:

-#cgo LDFLAGS: -lsqlite3
+#cgo pkg-config: sqlite3

But unfortunately it won't work either for static linking case. One either needs to add --static unconditionally (leading to overlinking in the dynamic case), or wish that cmd/go would do it for them, which is the subject of the issue.

If there is any other way to fix it, please let me know. I am only aware of an overlinking workaround, which is to pass -Wl,--as-needed, but with it parameters to linker needs to be in a proper order ($FLAGS $OBJECTS $LIBS), and it can also break some specific cases.

[ianlancetaylor]

@kolyshkin Sorry, I didn't mean to suggest that you should fix this. I'm trying to understand what you are suggesting. Let me try this: when you say "Docker can be linked either statically or dynamically" what do you mean? How does one decide to link Docker statically? How does one decide to link it dynamically?

[kolyshkin]

@ianlancetaylor sure! Looking into Docker build scripts, it's something like this (simplified for clarity):
go build -ldflags '-extldflags "-static"' ...
and when using gccgo this is:
go build -compiler gccgo --gccgoflags "-static" ...

[mattn]

s/pgk-config/pkg-config/g

[kolyshkin]

Sure it would be easier just to do something like
go build -static
and let cmd/go construct the proper flags depending on whether cgo or gccgo or something else is used. That would also make C static linking case detection easier. But I don't know enough to propose it.