net: listen tcp: errno -9

[alllexx88]

What version of Go are you using (go version)?

go1.10beta2

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="mipsle"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOEXE=""
GOHOSTARCH="mipsle"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/root/go"
GORACE=""
GOROOT="/opt/lib/golang"
GOTMPDIR=""
GOTOOLDIR="/opt/lib/golang/pkg/tool/linux_mipsle"
GCCGO="/opt/bin/gccgo"
GOMIPS="softfloat"
CC="/opt/bin/gcc"
CXX="/opt/bin/g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -mabi=32 -march=mips32 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build074360272=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Ran this code

package main

import (
	"net"
	"fmt"
)

func main() {
	_, err := net.Listen("tcp", ":8080")
	if err != nil {
		fmt.Println(err)
	}
}

What did you expect to see?

No output

What did you see instead?

listen tcp :8080: errno -9

I'm trying to get Go programs to run on a softfloat mips32r2 router. The first issue I encountered was pipe2 syscall not being supported by the kernel, so I wrote a patch to use pipe syscall plus fcntl instead (see mips_no_pipe2.patch). Now I'm facing this issue with net.Listen always failing. Any help or hints on how to debug this are appreciated.

Thanks,
Alex

[davecheney]

Which kernel is your system running?

[alllexx88]

@davecheney The router is running tomatousb firmware. The kernel is a custom 2.6.22.19 kernel:

[root@unknown root]$ uname -a
Linux unknown 2.6.22.19 #33 Sun Jun 26 17:47:11 CEST 2016 mips GNU/Linux

If I'm not mistaken, kernel sources can be found here, and config is here

[davecheney]

I don’t know the exact minimum kernel version MIPS needs but I suspect the problem you have is your kernel doesnt support accept4. Again the details are sketchy, but it’s the version of accept that supports passing the close on accept flag. Look in the net package for calls to syscall.Accept and then patch as appropriate. There may be some fallback code from the days we didn’t support rhel5, but still tried anyway.

…

On 16 Jan 2018, at 07:06, alllexx88 ***@***.***> wrote: @davecheney The router is running tomatousb firmware. The kernel is a custom 2.6.22.19 kernel: ***@***.*** root]$ uname -a Linux unknown 2.6.22.19 #33 Sun Jun 26 17:47:11 CEST 2016 mips GNU/Linux If I'm not mistaken, kernel sources can be found here, and config is here — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[alllexx88]

@davecheney Looks like my kernel indeed doesn't support accept4 syscall. I'll try to workaround it using accept and fcntl tomorrow. Thanks!

[mvdan]

I take it that there is nothing to fix in Go about this, then? Or should the minimum requirements for this case be better specified somewhere?

In any case, https://github.com/golang/go/wiki/MinimumRequirements does mention 2.6.23, and this post refers to 2.6.22.

[davecheney]

The minimum kernel version of non intel platforms is higher than 2.6.23. Maybe we should document the sys calls we need, accept4(1), etc, rather than the kernel version.

…

On 16 January 2018 at 08:20, Daniel Martí ***@***.***> wrote: I take it that there is nothing to fix in Go about this, then? Or should the minimum requirements for this case be better specified somewhere? In any case, https://github.com/golang/go/wiki/MinimumRequirements does mention 2.6.23, and this post refers to 2.6.22. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#23446 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA09GU6v6-3j5BlQqMvwz8MMRupDeks5tK8ElgaJpZM4Redmg> .

[bradfitz]

Maybe we should document the sys calls we need, accept4(1), etc, rather
than the kernel version.

Please no. That will help a very certain type of user, but not most users. Most users know their kernel version but don't know what system calls their kernel supports.

If anything, any system calls assumed to be present by Go should perhaps throw on ENOSYS with "your kernel is too old".

[alllexx88]

@davecheney Looks like it's not accept4 at fault in my case. I wrote this patch(hack) to avoid using accept4 syscall, but net.Listen() gives me the same error:

--- a/src/syscall/zsyscall_linux_mips.go
+++ b/src/syscall/zsyscall_linux_mips.go
@@ -1416,11 +1416,17 @@ func accept(s int, rsa *RawSockaddrAny, addrlen *_Socklen) (fd int, err error) {
 // THIS FILE IS GENERATED BY THE COMMAND AT THE TOP; DO NOT EDIT
 
 func accept4(s int, rsa *RawSockaddrAny, addrlen *_Socklen, flags int) (fd int, err error) {
-	r0, _, e1 := Syscall6(SYS_ACCEPT4, uintptr(s), uintptr(unsafe.Pointer(rsa)), uintptr(unsafe.Pointer(addrlen)), uintptr(flags), 0, 0)
+	r0, _, e1 := Syscall(SYS_ACCEPT, uintptr(s), uintptr(unsafe.Pointer(rsa)), uintptr(unsafe.Pointer(addrlen)))
 	fd = int(r0)
 	if e1 != 0 {
 		err = errnoErr(e1)
+		return
+	}
+	flags2, err := fcntl(fd, F_GETFL, 0)
+	if err != nil {
+		return
 	}
+	_, err = fcntl(fd, F_SETFL, flags2 | flags)
 	return
 }
 
--- a/src/syscall/zsyscall_linux_mipsle.go
+++ b/src/syscall/zsyscall_linux_mipsle.go
@@ -1416,11 +1416,17 @@ func accept(s int, rsa *RawSockaddrAny, addrlen *_Socklen) (fd int, err error) {
 // THIS FILE IS GENERATED BY THE COMMAND AT THE TOP; DO NOT EDIT
 
 func accept4(s int, rsa *RawSockaddrAny, addrlen *_Socklen, flags int) (fd int, err error) {
-	r0, _, e1 := Syscall6(SYS_ACCEPT4, uintptr(s), uintptr(unsafe.Pointer(rsa)), uintptr(unsafe.Pointer(addrlen)), uintptr(flags), 0, 0)
+	r0, _, e1 := Syscall(SYS_ACCEPT, uintptr(s), uintptr(unsafe.Pointer(rsa)), uintptr(unsafe.Pointer(addrlen)))
 	fd = int(r0)
 	if e1 != 0 {
 		err = errnoErr(e1)
+		return
+	}
+	flags2, err := fcntl(fd, F_GETFL, 0)
+	if err != nil {
+		return
 	}
+	_, err = fcntl(fd, F_SETFL, flags2 | flags)
 	return
 }
 

I grepped for ENOSYS in strace output and here's what I got (not sure if it matters):

$ strace ./test 2>&1 | grep ENOSYS
futex(0x7fee8938, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, NULL, 0) = -1 ENOSYS (Function not implemented)

@mvdan

I take it that there is nothing to fix in Go about this, then?

Probably you are correct, though I still don't know why net.Listen fails in my case (not due to missing accept4, as it seems), but Go works fine on my other systems with newer kernels (arm, i686). Is it OK for this discussion-attempt to get Go to work on 2.6.22.19 kernel to stay here, or should I better post on some mailing list?

Thanks

[ianlancetaylor]

According to https://github.com/golang/go/wiki/MinimumRequirements for little-endian mips64 you need a kernel newer than 4.1. No specific requirements are listed for little-endian 32-bit mips, but it seems very likely that 2.6.22 is too old.

Errno 9 is EBADF, but errno -9 is a bug in how the port interacts with the kernel. I have no idea whether this is due to using an old kernel or due to some other problem.

[crvv]

The errno -9 is similar to #14297. It is 9, EBADF.
The error EBADF is returned by epollctl.

go/src/runtime/netpoll_epoll.go

Line 47 in 99e6e48

return -epollctl(epfd, _EPOLL_CTL_ADD, int32(fd), &ev)

I can reproduce the issue on MIPS Linux 2.6.18.
It works fine on MIPS Linux 2.6.32 and 4.4.89.

[gopherbot]

Change https://golang.org/cl/89235 mentions this issue: runtime: fix errno sign for epollctl on mips, mips64 and ppc64

[alllexx88]

@crvv Thank you for clarifying things! 👍 So it's a EBADF after all. Now it would be great if there was a workaround for this error for older MIPS Linux.

[ianlancetaylor]

@alllexx88 There's really nothing we can do if the epoll_ctl system call doesn't work. On GNU/Linux systems the Go runtime requires epoll.

[alllexx88]

@ianlancetaylor epoll_ctl should work in Linux 2.6.22.19, if I understand it correctly. It just returns EBADF on an fd that is expected to be valid. I just don't understand where (and why) it is happening, since I'm not really familiar with Go internals. Patching for missing pipe2 and accept4 syscalls was relatively simple, but I can't figure out where it goes wrong with epoll_ctl. On a side note, this is not a Go issue, modern GNU/Linux MIPS platforms that're powerful enough to run heavy Go applications usually have newer kernels (at least 3.2+ I think).

I'm still interested in finding a workaround, but this issue can be closed, I think. Thank you all.

[gopherbot]

Change https://golang.org/cl/125895 mentions this issue: runtime: fix syscall error returns on mips/mips64/ppc64

[dwimmer]

This error was caused by epollcreate1() returning ENOSYS instead of -ENOSYS. This was interpreted as success and the value of ENOSYS was later passed as a file descriptor to epollctl() leading to the (valid) EBADF.

[dwimmer]

@daenney, I saw your post about net.Listen() on the RT-AC66U and I thought you might be interested to know that it should work fine now that this is fixed (should be in Go 1.11 or try it out now on master).