path/filepath: WebAssembly uses *nix conventions on Windows

[evanw]

What version of Go are you using (go version)?

$ go version
go version go1.15.6 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\User\AppData\Local\go-build
set GOENV=C:\Users\User\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\User\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\User\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=c:\go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=c:\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\User\AppData\Local\Temp\go-build012101774=/tmp/go-build -gno-record-gcc-switches

What did you do?

Here's a quick demonstration of the problem. Make a file called main.go that looks like this:

package main

import "fmt"
import "path/filepath"

func main() {
  fmt.Printf("IsAbs: %v\n", filepath.IsAbs("c:\\test"))
}

Run this program natively on Windows:

> go build -o .\main.exe .\main.go
> .\main.exe
IsAbs: true

Run this program via WebAssembly on Windows:

> set GOOS=js
> set GOARCH=wasm
> go build -o .\main.wasm .\main.go
> copy c:\go\misc\wasm\wasm_exec.js .\wasm_exec.js
> node .\wasm_exec.js .\main.wasm
IsAbs: false

What did you expect to see?

I expected to see IsAbs: true in both cases.

What did you see instead?

I see IsAbs: true when running a native executable on Windows but I see IsAbs: false when running via WebAssembly on Windows.

I'm the developer behind esbuild, a JavaScript bundler written in Go. I publish native executables for all major operating system and architecture combinations, and I also separately publish a WebAssembly executable. The WebAssembly executable is intended to enable people to use esbuild in the browser and also in scenarios where they want to use esbuild outside of the browser in an os-independent context, which is the promise of WebAssembly.

Here is the context for this issue: evanw/esbuild#687. The Yarn JavaScript package manager would like to use the WebAssembly version of esbuild instead of the native version. They try to avoid install scripts with platform-specific behavior since it messes with their preferred package installation strategy. Thus the same directory tree must be able to work on all platforms.

I have (perhaps incorrectly) assumed that Go's WebAssembly build target produces a cross-platform executable. This doesn't appear to be the case. It also doesn't appear to be something that can be fixed in a backwards-compatible manner, so I can understand if you either won't or can't fix it. It does seem like a bug to me because it violated my expectations of what the behavior should be, so I have reported it as a bug. But it's also possible that the WebAssembly build target only works in Unix-style environments by design and I was incorrect in assuming it would be cross-platform.

Regardless, I could use some advice for what to do here. Am I on my own for how to fix this? I imagine I could essentially fork the Go standard library and try to include both copies in the WebAssembly build. But I would prefer not to do that if possible.

[arcanis]

I don't know how applicable it is here, but in Yarn we also happen to deal with filesystem path portability, and our solution to this problem is to always only work with Posix paths (ie even on Windows our path representation is /c:/foo/bar), only translating them back into their native version (c:\foo\bar) right when we have to pass them to the Node APIs.

[evanw]

Since the title now implies the problem is just with IsAbs: The problem is with the path/filepath package, not just with filepath.IsAbs. For example, filepath.Dir fails to split on backslashes and filepath.Join inserts forward slashes instead of backward slashes.

[mihaiav]

I think is more a WASI issue

[historyliao]

When run sample code above on linux it prints false too! It's not just about WebAssembly. Python solved this problem with different module for unix and windows. That is posixpath and ntpath according to docs https://docs.python.org/3/library/os.path.html

[evanw]

It looks like this also affects other modules outside of path/filepath too. The os.MkdirAll and os.IsPathSeparator functions also doesn't work on Windows when using WebAssembly.

[bjorn3]

WASI emulates an UNIX environment and thus uses UNIX path separators. WASI contains various things that make it very clear that it is a UNIX environment and not a Windows environment, like device and inode numbers which don't have a direct Windows counterpart AFAIK. It also doesn't have junctions, but hardlinks and symlinks. Further more it uses a single namespace file descriptors which are plain integers just like UNIX, while Windows uses HANDLE's which are pointers and a separate namespace for sockets.

Edit: I thought Go uses WASI. Looks like it has it's own wasm syscall interface instead. My bad.