linux/mips 1.14: illegal instruction

[nexus166]

hi,

Running go (latest) or any compiled (with latest go) prog on a MIPS 24Kc V7.4 running OpenWRT (linux) results in the following

root@wgw:~# go env
SIGILL: illegal instruction
PC=0x47a6cc m=0 sigcode=128

goroutine 1 [running, locked to thread]:
runtime.asyncPreempt()
	/usr/lib/go-1.14/src/runtime/preempt_mipsx.s:40 +0x7c fp=0x142876c sp=0x1428678 pc=0x47a6cc
context.init()
	/usr/lib/go-1.14/src/context/context.go:157 fp=0x1428770 sp=0x1428770 pc=0x535854
runtime.doInit(0xe43bc0)
	/usr/lib/go-1.14/src/runtime/proc.go:5414 +0xcc fp=0x1428784 sp=0x1428770 pc=0x4515ec
runtime.doInit(0xe46140)
	/usr/lib/go-1.14/src/runtime/proc.go:5409 +0x80 fp=0x1428798 sp=0x1428784 pc=0x4515a0
runtime.doInit(0xe46e00)
	/usr/lib/go-1.14/src/runtime/proc.go:5409 +0x80 fp=0x14287ac sp=0x1428798 pc=0x4515a0
runtime.doInit(0xe48100)
	/usr/lib/go-1.14/src/runtime/proc.go:5409 +0x80 fp=0x14287c0 sp=0x14287ac pc=0x4515a0
runtime.main()
	/usr/lib/go-1.14/src/runtime/proc.go:190 +0x22c fp=0x14287ec sp=0x14287c0 pc=0x441dc0
runtime.goexit()
	/usr/lib/go-1.14/src/runtime/asm_mipsx.s:651 +0x4 fp=0x14287ec sp=0x14287ec pc=0x479f38

r0   0x0	r1   0x4
r2   0x0	r3   0x535854
r4   0x4	r5   0x1
r6   0xe0	r7   0x1464540
r8   0x5	r9   0x1
r10  0x4	r11  0x8
r12  0x1	r13  0xd
r14  0xf	r15  0xb328d9
r16  0xb328db	r17  0x3
r18  0xf8	r19  0x0
r20  0x1464570	r21  0xa38f84
r22  0xe43bdc	r23  0xfffffffb
r24  0x1000002	r25  0x77fae67c
r26  0x0	r27  0x0
r28  0x77fe32a0	r29  0x1428678
r30  0x14000e0	r31  0x535854
pc   0x47a6cc	link 0x535854
lo   0x0	hi   0x0

other example

root@wgw:~# cat test.go 
package main

import (
	"fmt"
	"runtime"
)

func main() {
	fmt.Println(runtime.GOOS)
}
root@wgw:~# go run test.go 
SIGILL: illegal instruction
PC=0x47a6cc m=0 sigcode=128

goroutine 1 [running, locked to thread]:
runtime.asyncPreempt()
	/usr/lib/go-1.14/src/runtime/preempt_mipsx.s:40 +0x7c fp=0x102a758 sp=0x102a664 pc=0x47a6cc
go/parser.init()
	/usr/lib/go-1.14/src/go/parser/parser.go:171 fp=0x102a75c sp=0x102a75c pc=0x5b7154
runtime.doInit(0xe46440)
	/usr/lib/go-1.14/src/runtime/proc.go:5414 +0xcc fp=0x102a770 sp=0x102a75c pc=0x4515ec
runtime.doInit(0xe47660)
	/usr/lib/go-1.14/src/runtime/proc.go:5409 +0x80 fp=0x102a784 sp=0x102a770 pc=0x4515a0
runtime.doInit(0xe45bc0)
	/usr/lib/go-1.14/src/runtime/proc.go:5409 +0x80 fp=0x102a798 sp=0x102a784 pc=0x4515a0
runtime.doInit(0xe46e00)
	/usr/lib/go-1.14/src/runtime/proc.go:5409 +0x80 fp=0x102a7ac sp=0x102a798 pc=0x4515a0
runtime.doInit(0xe48100)
	/usr/lib/go-1.14/src/runtime/proc.go:5409 +0x80 fp=0x102a7c0 sp=0x102a7ac pc=0x4515a0
runtime.main()
	/usr/lib/go-1.14/src/runtime/proc.go:190 +0x22c fp=0x102a7ec sp=0x102a7c0 pc=0x441dc0
runtime.goexit()
	/usr/lib/go-1.14/src/runtime/asm_mipsx.s:651 +0x4 fp=0x102a7ec sp=0x102a7ec pc=0x479f38

r0   0x0	r1   0x55
r2   0x0	r3   0x5b7154
r4   0x4	r5   0x1
r6   0x69	r7   0x1080000
r8   0x3	r9   0x1
r10  0x1	r11  0x8
r12  0x1	r13  0xd
r14  0xf	r15  0xb327f9
r16  0x0	r17  0x0
r18  0xf1	r19  0x18
r20  0x1080020	r21  0xa395e3
r22  0xe46480	r23  0xe80000
r24  0x1000002	r25  0x77f6667c
r26  0x0	r27  0x0
r28  0x77f9b2a0	r29  0x102a664
r30  0x10000e0	r31  0x5b7154
pc   0x47a6cc	link 0x5b7154
lo   0x0	hi   0x0
root@wgw:~# Connection to 192.168.8.1 closed.

me@box:/opt/go/src/test $  GOARCH=mips GOOS=linux CGO_ENABLED=0 go build -a -o test -gcflags="all=-d softfloat"

me@box:/opt/go/src/test $  scp test root@192.168.8.1:
test                                                                                                                                                                                                        100% 1572KB   1.7MB/s   00:00    

me@box:/opt/go/src/test $  ssh root@192.168.8.1

BusyBox v1.31.1 () built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt SNAPSHOT, r12565-32726846c8
 -----------------------------------------------------
root@wgw:~# ./test
linux
root@wgw:~# 

[nexus166]

the minimal fmt.Println(runtime.GOOS) program works but everything else, including the go tools, do not.

[cherrymui]

Dup of #37653. It is fixed on tip, and will be in 1.14.1 release. Thanks.

[nexus166]

ohhh sorry! And thanks a lot 😃

[cherrymui]

No worries :)

[nexus166]

still seeing this with program compiled using 1.14.1

SIGILL: illegal instruction
PC=0x86138 m=0 sigcode=128

goroutine 1 [running, locked to thread]:
runtime.asyncPreempt()
	/usr/lib/go/src/runtime/preempt_mipsx.s:41 +0x7c fp=0xc84e54 sp=0xc84d60 pc=0x86138
math/big.nat.scan(0x0, 0x14, 0x15, 0x357ea0, 0xc4b8a0, 0x10, 0x4, 0x14, 0xc4b8a0, 0x2c93e0, ...)
	/usr/lib/go/src/math/big/natconv.go:220 +0x130 fp=0xc84ecc sp=0xc84e58 pc=0x14ea00
math/big.(*Int).scan(0xc0a660, 0x357ea0, 0xc4b8a0, 0x10, 0x20, 0x2d2660, 0x1357e00, 0xc4b8a0)
	/usr/lib/go/src/math/big/intconv.go:190 +0x9c fp=0xc84f0c sp=0xc84ecc pc=0x144ab0
math/big.(*Int).setFromScanner(0xc0a660, 0x357ea0, 0xc4b8a0, 0x10, 0x1000007, 0xc0a660)
	/usr/lib/go/src/math/big/int.go:431 +0x54 fp=0xc84f30 sp=0xc84f0c pc=0x141274
math/big.(*Int).SetString(0xc0a660, 0x306fed, 0x100, 0x10, 0xc4b800, 0xc4b7c0)
	/usr/lib/go/src/math/big/int.go:425 +0x9c fp=0xc84f4c sp=0xc84f30 pc=0x1411ec
golang.org/x/crypto/ssh.init.0()
	/opt/go/pkg/mod/golang.org/x/crypto@v0.0.0-20200214034016-1d94cc7ab1c6/ssh/kex.go:392 +0x6c fp=0xc84f84 sp=0xc84f4c pc=0x26e614
runtime.doInit(0x4b5a20)
	/usr/lib/go/src/runtime/proc.go:5414 +0xcc fp=0xc84f98 sp=0xc84f84 pc=0x5e9e0
runtime.doInit(0x4b3ea0)
	/usr/lib/go/src/runtime/proc.go:5409 +0x80 fp=0xc84fac sp=0xc84f98 pc=0x5e994
runtime.doInit(0x4b41a0)
	/usr/lib/go/src/runtime/proc.go:5409 +0x80 fp=0xc84fc0 sp=0xc84fac pc=0x5e994
runtime.main()
	/usr/lib/go/src/runtime/proc.go:190 +0x22c fp=0xc84fec sp=0xc84fc0 pc=0x4f508
runtime.goexit()
	/usr/lib/go/src/runtime/asm_mipsx.s:651 +0x4 fp=0xc84fec sp=0xc84fec pc=0x859b8

r0   0x0	r1   0xf0000000
r2   0x7	r3   0xc4b8a0
r4   0x357ea0	r5   0x10
r6   0xc9e0c0	r7   0x15
r8   0x9a	r9   0x0
r10  0x0	r11  0x30
r12  0xffffffff	r13  0x0
r14  0x0	r15  0x0
r16  0x14	r17  0x0
r18  0xffffffff	r19  0x10
r20  0x1	r21  0x0
r22  0x4b5ae0	r23  0x4b0000
r24  0x0	r25  0x0
r26  0x0	r27  0x0
r28  0x0	r29  0xc84d60
r30  0xc000e0	r31  0x14ea00
pc   0x86138	link 0x14ea00
lo   0x0	hi   0x0

SIGILL: illegal instruction
PC=0x86138 m=0 sigcode=128

goroutine 1 [running, locked to thread]:
runtime.asyncPreempt()
	/usr/lib/go/src/runtime/preempt_mipsx.s:41 +0x7c fp=0xc84f80 sp=0xc84e8c pc=0x86138
crypto/tls.init()
	/usr/lib/go/src/crypto/tls/alert.go:47 fp=0xc84f84 sp=0xc84f84 pc=0x253d4c
runtime.doInit(0x4b50c0)
	/usr/lib/go/src/runtime/proc.go:5414 +0xcc fp=0xc84f98 sp=0xc84f84 pc=0x5e9e0
runtime.doInit(0x4b46e0)
	/usr/lib/go/src/runtime/proc.go:5409 +0x80 fp=0xc84fac sp=0xc84f98 pc=0x5e994
runtime.doInit(0x4b41a0)
	/usr/lib/go/src/runtime/proc.go:5409 +0x80 fp=0xc84fc0 sp=0xc84fac pc=0x5e994
runtime.main()
	/usr/lib/go/src/runtime/proc.go:190 +0x22c fp=0xc84fec sp=0xc84fc0 pc=0x4f508
runtime.goexit()
	/usr/lib/go/src/runtime/asm_mipsx.s:651 +0x4 fp=0xc84fec sp=0xc84fec pc=0x859b8

r0   0x0	r1   0x2
r2   0x0	r3   0x253d4c
r4   0x8db7a4	r5   0x1
r6   0x8	r7   0x3
r8   0x2c26e0	r9   0x350c12
r10  0x8	r11  0x0
r12  0x214	r13  0x80000000
r14  0x1	r15  0x215
r16  0x0	r17  0x1
r18  0xf2	r19  0x38
r20  0xc0d0a0	r21  0xc0d0a0
r22  0x4b5154	r23  0x4d0000
r24  0x0	r25  0x0
r26  0x3	r27  0x0
r28  0x0	r29  0xc84e8c
r30  0xc000e0	r31  0x253d4c
pc   0x86138	link 0x253d4c
lo   0x0	hi   0x0

[ianlancetaylor]

Reopening.

[cherrymui]

@nexus166 is your machine lack of an FPU? Did you set GOMIPS=softfloat? Thanks.

[nexus166]

Is GOMIPS=softfloat the current way to do it? As shown in one of my previous comments, I have always been building the bins with -gcflags="all=-d softfloat", and it got the job done. I can try this out. I have a mini fleet of various MIPS variants (mips/mips64/mips64le), all of them without (hw)FPU and in all cases the kernel does not have an FPU emulator built in either. I am building OpenWRT with the kernel FPU to see if it changes anything.

[cherrymui]

Yeah, GOMIPS=softfloat is the right way to do (and has already been for a long time). -d softfloat is really the compiler's internal flag, not a user facing flag. And more importantly, it only applies to Go code, not assembly code. In this particular case, runtime.asyncPreempt is an assembly function. GOMIPS=softfloat covers both.

For 64-bit MIPS, it should be GOMIPS64=softfloat.

[nexus166]

It looks like its all nice and well with the right env variables. I haven't tested running the full 1.14.1 toolchain on MIPS yet, but in the meantime: thank you all for the attention and clarifications and apologies for having you re-opening the issue for nothing!