x/sys/unix: missing some AIX syscalls, notably `Flock`

[mohammed90]

Go version

1.21.5

What operating system and processor architecture are you using (go env)?

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/runner/.cache/go-build'
GOENV='/home/runner/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/runner/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/runner/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/opt/hostedtoolcache/go/1.21.5/x64'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/opt/hostedtoolcache/go/1.21.5/x64/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.21.5'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/home/runner/work/caddy/caddy/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build498393660=/tmp/go-build -gno-record-gcc-switches'

What did you do?

Trying to (cross-)compile Caddy for AIX with the following command

GOOS=AIX GOARCH=ppc64 go build -trimpath -o caddy

What did you expect to see?

Successful compilation and build

What did you see instead?

The compilation fails with errors about unix.Flock not existing:

$ GOOS=aix GOARCH=ppc64 go build
# github.com/dgraph-io/badger
../../../../go/pkg/mod/github.com/dgraph-io/badger@v1.6.2/dir_unix.go:63:13: undefined: unix.Flock
# github.com/dgraph-io/badger/v2
../../../../go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.4/dir_unix.go:62:13: undefined: unix.Flock

The error points to the absence of Flock syscall for AIX in the golang.org/x/sys/unix package. I searched the respective directory within the package, and indeed the function is missing. AIX documentation documents the syscall Flock exists (see: flock.h File, and lockfx, lockf, flock, or lockf64 Subroutine). I understand the syscalls definitions in the package are generated, so whatever source or script used to generate them is resulting in the gap.

As of now, we're only aware of the Flock function missing because it directly impacts us (see caddyserver/caddy#5970), but others may be missing and subject to verification.

[prattmic]

cc @golang/aix

[laboger]

I see these CLs related to the removal of the Flock syscall on AIX:
https://go-review.googlesource.com/152397
https://go-review.googlesource.com/153938
Apparently AIX does not support the Flock syscall, and before these CLs went in it was emulated using FcntlFlock.

[bcmills]

AIX documentation documents the syscall Flock exists (see: flock.h File, and lockfx, lockf, flock, or lockf64 Subroutine).

Note that per https://www.ibm.com/docs/en/aix/7.3?topic=l-lockfx-lockf-flock-lockf64-subroutine (emphasis added):

The flock subroutine locks and unlocks entire files. This is a limited interface maintained for BSD compatibility, although its behavior differs from BSD in a few subtle ways. To apply a shared lock, the file must be opened for reading. To apply an exclusive lock, the file must be opened for writing.

Unfortunately, there isn't enough detail in that documentation for me to tell whether the “subtle differences” are just the restrictions on the file mode, or other behaviors as well. (For example, there may be other differences regarding the behavior of locks across exec, reentrant locking by other threads or file descriptors in the same process, and so on.)

With more documentation it might be possible to provide a usable AIX flock wrapper. In the meantime, I suggest using FcntlFlock instead. That is the approach taken by cmd/go/internal/lockedfile, and it has been fairly robust in practice once we worked out some non-trivial kinks like #32817.

There is an open proposal to export the lockedfile package as a publicly-importable package:

• proposal: os: make the internal lockedfile package public #33974

See previously:

• syscall: remove AIX implementation of Flock #29084
• proposal: syscall: implement Flock on Solaris in terms of FcntlFlock #24684
• cmd/go/internal/lockedfile: spurious EDEADLK failures on AIX and Solaris #32817

[laboger]

@ayappanec Can you provide some input on this issue?

[ayappanec]

Let me check with AIX core team if we can get more details about this flock subroutine.

[bmarwell]

@ayappanec have you heard back from the AIX team?

[ayappanec]

I haven't heard anything from them yet, as most of them are on vacation.
I myself had a look at the flock implementation. It is almost same as other bsd implementations, like the one here --> https://github.com/NetBSD/src/blob/trunk/tools/compat/flock.c
The one difference in AIX is EWOULDBLOCK can also happen for the errno EACCES.