misc/cgo/fortran, cmd/dist: Fortran Support Causing go/src/all.bash to Fail

[odysseus9672]

Please answer these questions before submitting your issue. Thanks!

1. What version of Go are you using (go version)?
   go version
   go version devel +7da4ced Sat Feb 27 21:12:19 2016 +0000 darwin/amd64
2. What operating system and processor architecture are you using (go env)?
   go env
   GOARCH="amd64"
   GOBIN=""
   GOEXE=""
   GOHOSTARCH="amd64"
   GOHOSTOS="darwin"
   GOOS="darwin"
   GOPATH=""
   GORACE=""
   GOROOT="/Users/sean/goLang/go"
   GOTOOLDIR="/Users/sean/goLang/go/pkg/tool/darwin_amd64"
   CC="clang"
   GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -gno-record-gcc-switches -fno-common"
   CXX="clang++"
   CGO_ENABLED="1"
3. What did you do?
   (Use play.golang.org to provide a runnable example, if possible.)
   Run $GOROOT/src/all.bash
4. What did you expect to see?
   A successful build and run of the test suite.
5. What did you see instead?

##### ../misc/cgo/fortran
# _/Users/sean/goLang/go/misc/cgo/fortran
ld: library not found for -lgfortran
clang: error: linker command failed with exit code 1 (use -v to see invocation)
FAIL    _/Users/sean/goLang/go/misc/cgo/fortran [build failed]
FAIL: go test
2016/02/27 14:47:14 Failed: exit status 1

The root of the problem is that the gfortran I use is supplied by the Fink software distribution (gcc5 package). Because of that, I need to find a way to get this into the LDFLAGS for the build process: -L/sw/lib/gcc5/lib

I've tried setting the environment variable before running all.bash, and that doesn't help. Ideally, there should three options that can be passed in to the build process to address this issue:

1: an explicit path to the fortran compiler to use,
2: the linker flag for things that need the standard fortran libs, and
3: a flag to disable the fortran feature if it isn't desired.

Maybe a set of environment variables (eg. GoFortran, GoFortranLibDir), where the fortran feature only kicks in if the first is set to the path of the fortran compiler and the second is a (list of?) directories to include in the linker's search space for fortran related builds.

[mikioh]

This occurs on the dragonfly buildbot too.

##### ../misc/cgo/fortran
# _/tmp/gobuilder/dragonfly-amd64-922ce58de018/go/misc/cgo/fortran
/usr/libexec/binutils225/elf/ld.gold: error: cannot find -lgfortran
FAIL    _/tmp/gobuilder/dragonfly-amd64-922ce58de018/go/misc/cgo/fortran [build failed]
FAIL: go test
2016/02/27 14:46:23 Failed: exit status 1

See http://build.golang.org/log/c10562643cfb4e429ca1f269dc0823bc76e830cf.

/cc @kortschak

[kortschak]

This is curious - if it's not building it should not run the test.

@odysseus9672 @mikioh before you do the operations below, there is an omission to my last CL that might help figure this out. Is there a main.exe in misc/cgo/fortran? If there is, what does it do when run?

Then, can you tell me the outcome of running the test.bash in misc/cgo/fortran with $FC (the name of your fortran compiler) as a parameter? Also for $FC helloworld/helloworld.f90 -o main.exe >& /dev/null; echo $? (then without the dev null redirect).

[odysseus9672]

Negative. This is the contents of misc/cgo/fortran/ .

ls -lh
total 32
-rw-r--r--  1 sean  staff   295B Feb 24 13:13 answer.f90
-rw-r--r--  1 sean  staff   264B Feb 24 13:13 fortran.go
-rw-r--r--  1 sean  staff   334B Feb 24 13:13 fortran_test.go
drwxr-xr-x  3 sean  staff   102B Feb 26 16:44 helloworld
-rwxr-xr-x  1 sean  staff   495B Feb 28 01:15 test.bash

Here are the results of the experiments you wanted run:

Sean-iBook4:fortran sean$ ./test.bash /sw/bin/gfortran
# _/Users/sean/goLang/go/misc/cgo/fortran
ld: library not found for -lgfortran
clang: error: linker command failed with exit code 1 (use -v to see invocation)
FAIL    _/Users/sean/goLang/go/misc/cgo/fortran [build failed]
FAIL: go test
Sean-iBook4:fortran sean$ $FC helloworld/helloworld.f90 -o main.exe >& /dev/null; echo $
$
Sean-iBook4:fortran sean$ $FC helloworld/helloworld.f90 -o main.exe
Sean-iBook4:fortran sean$ ./main.exe 
 Hello World!
Sean-iBook4:fortran sean$ echo $FC
/sw/bin/gfortran

I wonder if this is because some environment variables aren't correctly being set in the test.bash file?

[kortschak]

Not "echo $��, "echo $?��, and do this after both invocations of $FC - use the value you get from the following for $FC.

Can you echo $FC after it's set in the in misc/cgo/fortran/test.bash, and change "set -e" to "set -ex" and run all.bash paste the stanza regarding the fortran test here.

[odysseus9672]

I'll run your tests as soon as I describe the tests I've tried.

I've set out testing things in test.bash, and I can only suppose that there is something hinky with the version of bash used on Mac OS X and Dragonfly. I tried adding the LDFLAGS to the test.bash script explicitly, and even sourcing Fink's init script, and the result is the same. Stranger still, the output of adding the verbose flag to the test script and running a verbose build outside of the script has virtually the same output.

$FC helloworld/helloworld.f90 -o main.exe >& /dev/null; echo $?
0

##### ../misc/cgo/fortran
+ FC=/sw/bin/gfortran
+ echo /sw/bin/gfortran
/sw/bin/gfortran
+ /sw/bin/gfortran helloworld/helloworld.f90 -o main.exe
+ rm -f main.exe
+ go test
# _/Users/sean/goLang/go/misc/cgo/fortran
ld: library not found for -lgfortran
clang: error: linker command failed with exit code 1 (use -v to see invocation)
FAIL    _/Users/sean/goLang/go/misc/cgo/fortran [build failed]
+ echo 'FAIL: go test'
FAIL: go test
+ status=1
+ exit 1

[kortschak]

Can you remove the "rm -f main.exe" (this was just added today) from the test.bash file and run all.bash, then tell me if main.exe exists in misc/cgo/fortran at the end of the install?

[odysseus9672]

Modifying fortran.go to read as follows works:

// Copyright 2016 The Go Authors.  All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

package fortran

// #cgo LDFLAGS: -L/sw/lib/gcc5/lib
// int the_answer();
import "C"

func TheAnswer() int {
    return int(C.the_answer())
}

Of course, making that change is not a general solution, but there should be some way to get the appropriate information setup.

[odysseus9672]

The output you asked for: main.exe exists and runs correctly.

output from all.bash:

##### ../misc/cgo/fortran
+ FC=/sw/bin/gfortran
+ echo /sw/bin/gfortran
/sw/bin/gfortran
+ /sw/bin/gfortran helloworld/helloworld.f90 -o main.exe
+ go test
# _/Users/sean/goLang/go/misc/cgo/fortran
ld: library not found for -lgfortran
clang: error: linker command failed with exit code 1 (use -v to see invocation)
FAIL    _/Users/sean/goLang/go/misc/cgo/fortran [build failed]
+ echo 'FAIL: go test'
FAIL: go test
+ status=1
+ exit 1
2016/02/28 02:26:31 Failed: exit status 1

[odysseus9672]

Setting the environment variable CGO_LDFLAGS, as described here:

https://golang.org/cmd/cgo/

also makes all.bash run correctly. So, I imagine that Fortran will need a CGO_FORTRANFLAGS, and CGO_LDFLAGS will suffice.

[kortschak]

Thanks. So it seems your gfortran knows where the libraries are when invoked from bash, but not from Go. Do you have any env vars that might be helping it? One last thing you could do is to checkout the patch set 2 changes from https://golang.org/cl/19874 and see if all.bash passes. Don't give it any help with CGO_LDFLAGS.

[odysseus9672]

I might have some environment variables set that help it. I can't identify which one, though, because none of them point to /sw/lib in a way that gfortran should understand.

Edit: also, I'm up to the minute on the devel branch.
go version
go version devel +5fc4dec Sun Feb 28 10:36:01 2016 +0000 darwin/amd64

printenv
MANPATH=/sw/share/man:/usr/local/share/man:/usr/share/man:/opt/X11/share/man:/Library/TeX/texbin/man:/Applications/Xcode.app/Contents/Developer/usr/share/man:/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/share/man:/sw/lib/perl5/5.18.2/man
TERM_PROGRAM=Apple_Terminal
XDG_DATA_HOME=/Users/sean/.local/share
GNUPLOT_DEFAULT_GDFONT=Arial
SHELL=/bin/bash
TERM=xterm-256color
HGENCODING=UTF-8
TMPDIR=/var/folders/75/tsmllrjr8xl81b0059bfyyd00000gs/T/
PERL5LIB=/sw/lib/perl5:/sw/lib/perl5/darwin
Apple_PubSub_Socket_Render=/private/tmp/com.apple.launchd.dz7wyAWdKm/Render
TERM_PROGRAM_VERSION=343.7
TERM_SESSION_ID=D928C99E-652F-495C-BD11-B3E10E6B16BF
ac_cv_path_INTLTOOL_PERL=/usr/bin/perl
XAPPLRESDIR=/sw/etc/app-defaults/
GOROOT_BOOTSTRAP=/Users/sean/golang/go1.4.3
SGML_CATALOG_FILES=/sw/etc/sgml/catalog
USER=sean
GDK_USE_XFT=1
GDFONTPATH=/Library/Fonts
SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.aRfIhVGsJj/Listeners
__CF_USER_TEXT_ENCODING=0x1F9:0x0:0x0
GNOME_DISABLE_CRASH_DIALOG=1
XDG_CONFIG_DIRS=/sw/etc/xdg
PATH=/Users/sean/mybin:/sw/bin:/sw/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Library/TeX/texbin:/Users/sean/goLang/go/bin
HG=/sw/bin/hg
XML_CATALOG_FILES=/sw/etc/xml/catalog
PWD=/Users/sean
LANG=en_US.UTF-8
XPC_FLAGS=0x0
GOMAXPROCS=4
XPC_SERVICE_NAME=0
IRCNICK=BlackGriffin
SHLVL=1
HOME=/Users/sean
bc03=/Users/sean/AstroTools/bc03/src
GOROOT=/Users/sean/goLang/go
XDG_CONFIG_HOME=/Users/sean/.config
XDG_CACHE_HOME=/Users/sean/.cache
PYTHONPATH=/Users/sean/lib
LOGNAME=sean
XDG_DATA_DIRS=/sw/share
DBUS_SESSION_BUS_ADDRESS=launchd:env=DBUS_FINK_SESSION_BUS_SOCKET
INFOPATH=/sw/share/info:/sw/info:/usr/share/info
XDG_RUNTIME_DIR=/var/folders/75/tsmllrjr8xl81b0059bfyyd00000gs/C/
DISPLAY=/private/tmp/com.apple.launchd.5ZSuFRU7vW/org.macosforge.xquartz:0
LADSPA_PATH=/sw/lib/ladspa
SECURITYSESSIONID=186a5
_=/usr/bin/printenv

[kortschak]

@odysseus9672 would you please try

cd $GOROOT/src
git fetch https://go.googlesource.com/go refs/changes/74/19874/2
git checkout FETCH_HEAD
./all.bash

[odysseus9672]

The exit status changed, but nothing else I can see.

##### ../misc/cgo/fortran
# _/Users/sean/goLang/go/misc/cgo/fortran
ld: library not found for -lgfortran
clang: error: linker command failed with exit code 1 (use -v to see invocation)
FAIL    _/Users/sean/goLang/go/misc/cgo/fortran [build failed]
2016/02/28 15:41:58 Failed: exit status 2