cmd/go: 'go build' fails when current directory and output directory are symlinks on Windows

[ankeesler]

What version of Go are you using (go version)?

go version go1.11 windows/amd64

Does this issue reproduce with the latest release?

Yes. This is a regression from go1.10.3.

What operating system and processor architecture are you using (go env)?

Windows Server 1709 (OS Version 10.0.16299) and Windows Server 1803 (OS Version 10.0.17134).

set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Administrator\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\Users\Administrator\go
set GOPROXY=
set GORACE=
set GOROOT=C:\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g`<no output>` -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\Administrator\AppData\Local\Temp\go-build075376310=/tmp/go-build -gno-record-gcc-switches

What did you do?

On Windows, create this directory structure.

C:\some-directory\
  hello\
  src\
  src2 [-> symlinked to C:\some-directory\src]
    hello [-> symlinked to C:\some-directory\hello]
    hello.go

From the directory C:\some-directory\src2\, run go build -o c:\some-directory\src2\hello\hello.exe .\hello.go.

What did you expect to see?

The command succeeds and builds hello.exe in the directory C:\some-directory\hello\.

What did you see instead?

go build command-line-arguments: mkdir C:\some-directory\src2\hello\: Cannot create a file when that file already exists.

[andybons]

@bcmills @rsc

[alexbrainman]

@ankeesler I can reproduce this here, thank you very much.. It is broken on the tip too.

Alex

[ankeesler]

Thanks @andybons + @alexbrainman. Let me know if you need anything else from me, happy to help.

[rsc]

I'm sorry this broke but we just fundamentally do not support magic tricks with symlinks. The answer is almost certainly to stop trying to trick the go command using symlinks.

[ankeesler]

@rsc thanks for taking a look here. We were hoping that since this works on Linux and it has worked in previous versions on Windows, then it would continue to function the same way going forward. Windows symlinks can certainly by annoying at times (most times). All this being said, is there any chance that this will be fixed in a future go version?

We are working with an interface that expects the directory hierarchy outlined in my initial comment. Since this works on Linux, it will likely be difficult, if not impossible, for us to change the hierarchy. This interface is used in our automated deployment tooling for Kubernetes.

[alexbrainman]

I have used https://play.golang.org/p/xYnM-XHKDve to find the commit that broke the test:

e83601b4356f92f2f4d05f302d5654754ff05a6d is the first bad commit
commit e83601b4356f92f2f4d05f302d5654754ff05a6d
Author: Alex Brainman <[email protected]>
Date:   Sun Jan 7 12:12:25 2018 +1100

    os: use WIN32_FIND_DATA.Reserved0 to identify symlinks

    os.Stat implementation uses instructions described at
    https://blogs.msdn.microsoft.com/oldnewthing/20100212-00/?p=14963/
    to distinguish symlinks. In particular, it calls
    GetFileAttributesEx or FindFirstFile and checks
    either WIN32_FILE_ATTRIBUTE_DATA.dwFileAttributes
    or WIN32_FIND_DATA.dwFileAttributes to see if
    FILE_ATTRIBUTES_REPARSE_POINT flag is set.
    And that seems to worked fine so far.

    But now we discovered that OneDrive root folder
    is determined as directory:

    c:\>dir C:\Users\Alex | grep OneDrive
    30/11/2017  07:25 PM    <DIR>          OneDrive
    c:\>

    while Go identified it as symlink.

    But we did not follow Microsoft's advice to the letter - we never
    checked WIN32_FIND_DATA.Reserved0. And adding that extra check
    makes Go treat OneDrive as symlink. So use FindFirstFile and
    WIN32_FIND_DATA.Reserved0 to determine symlinks.

    Fixes #22579

    Change-Id: I0cb88929eb8b47b1d24efaf1907ad5a0e20de83f
    Reviewed-on: https://go-review.googlesource.com/86556
    Reviewed-by: Brad Fitzpatrick <[email protected]>
    Run-TryBot: Brad Fitzpatrick <[email protected]>
    TryBot-Result: Gobot Gobot <[email protected]>

:040000 040000 6e4ea25b8e56d55d3119d258431eb49b66ac19fc a6eac27eee057861c2fa8254
dc627fa181fd6ae5 M      src

It looks like cmd/go fails when it calls os.MkdirAll, which calls os.Stat of C:\Users\Alex\AppData\Local\Temp\TestAlex496017355\src2\hello\ - note the backslash at the end of the path, and C:\Users\Alex\AppData\Local\Temp\TestAlex496017355\src2 is the symlink. os.Stat of such directory succeeds on linux, and returns directory, but fails on windows.

So this looks like a dup of #27225 or very similar.

If someone wants to have these 2 issues fixed, please, review https://golang.org/cl/134195 .

Alex

[gopherbot]

Change https://golang.org/cl/143578 mentions this issue: os: use CreateFile for Stat of symlinks

[alexbrainman]

@ankeesler please try https://go-review.googlesource.com/c/go/+/143578

Does it fixes your problem?

Thank you.

Alex

[ankeesler]

@alexbrainman thank you for your work on this. We will try this out and let you know.

[ankeesler]

@alexbrainman we verified that this change does indeed fix the issue. Thank you.

[alexbrainman]

@alexbrainman we verified that this change does indeed fix the issue

Thank you for checking. I will have it reviewed and submitted.

Alex