Skip to content

Commit 0a5fae2

Browse files
prattmicprattmic
committed
runtime, syscall: reimplement AllThreadsSyscall using only signals.
In issue 50113, we see that a thread blocked in a system call can result in a hang of AllThreadsSyscall. To resolve this, we must send a signal to these threads to knock them out of the system call long enough to run the per-thread syscall. Stepping back, if we need to send signals anyway, it should be possible to implement this entire mechanism on top of signals. This CL does so, vastly simplifying the mechanism, both as a direct result of newly-unnecessary code as well as some ancillary simplifications to make things simpler to follow. Major changes: * The rest of the mechanism is moved to os_linux.go, with fields in mOS instead of m itself. * 'Fixup' fields and functions are renamed to 'perThreadSyscall' so they are more precise about their purpose. * Rather than getting passed a closure, doAllThreadsSyscall takes the syscall number and arguments. This avoids a lot of hairy behavior: * The closure may potentially only be live in fields in the M, hidden from the GC. Not necessary with no closure. * The need to loan out the race context. A direct RawSyscall6 call does not require any race context. * The closure previously conditionally panicked in strange locations, like a signal handler. Now we simply throw. * All manual fixup synchronization with mPark, sysmon, templateThread, sigqueue, etc is gone. The core approach is much simpler: doAllThreadsSyscall sends a signal to every thread in allm, which executes the system call from the signal handler. We use (SIGRTMIN + 1), aka SIGSETXID, the same signal used by glibc for this purpose. As such, we are careful to only handle this signal on non-cgo binaries. Synchronization with thread creation is a key part of this CL. The comment near the top of doAllThreadsSyscall describes the required synchronization semantics and how they are achieved. Note that current use of allocmLock protects the state mutations of allm that are also protected by sched.lock. allocmLock is used instead of sched.lock simply to avoid holding sched.lock for so long. Fixes #50113 Change-Id: Ic7ea856dc66cf711731540a54996e08fc986ce84 Reviewed-on: https://go-review.googlesource.com/c/go/+/383434 Reviewed-by: Austin Clements <[email protected]> Trust: Michael Pratt <[email protected]> Run-TryBot: Michael Pratt <[email protected]> TryBot-Result: Gopher Robot <[email protected]>
1 parent 0b321c9 commit 0a5fae2

35 files changed

+429
-428
lines changed

‎src/runtime/defs_linux.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,8 @@ const (
9191
SIGPWR = C.SIGPWR
9292
SIGSYS = C.SIGSYS
9393

94+
SIGRTMIN = C.SIGRTMIN
95+
9496
FPE_INTDIV = C.FPE_INTDIV
9597
FPE_INTOVF = C.FPE_INTOVF
9698
FPE_FLTDIV = C.FPE_FLTDIV

‎src/runtime/defs_linux_386.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,8 @@ const (
6464
_SIGPWR = 0x1e
6565
_SIGSYS = 0x1f
6666

67+
_SIGRTMIN = 0x20
68+
6769
_FPE_INTDIV = 0x1
6870
_FPE_INTOVF = 0x2
6971
_FPE_FLTDIV = 0x3

‎src/runtime/defs_linux_amd64.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,8 @@ const (
6464
_SIGPWR = 0x1e
6565
_SIGSYS = 0x1f
6666

67+
_SIGRTMIN = 0x20
68+
6769
_FPE_INTDIV = 0x1
6870
_FPE_INTOVF = 0x2
6971
_FPE_FLTDIV = 0x3

‎src/runtime/defs_linux_arm.go

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,7 @@ const (
6363
_SIGIO = 0x1d
6464
_SIGPWR = 0x1e
6565
_SIGSYS = 0x1f
66+
_SIGRTMIN = 0x20
6667
_FPE_INTDIV = 0x1
6768
_FPE_INTOVF = 0x2
6869
_FPE_FLTDIV = 0x3

‎src/runtime/defs_linux_arm64.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,8 @@ const (
6464
_SIGPWR = 0x1e
6565
_SIGSYS = 0x1f
6666

67+
_SIGRTMIN = 0x20
68+
6769
_FPE_INTDIV = 0x1
6870
_FPE_INTOVF = 0x2
6971
_FPE_FLTDIV = 0x3

‎src/runtime/defs_linux_mips64x.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,8 @@ const (
6666
_SIGXCPU = 0x1e
6767
_SIGXFSZ = 0x1f
6868

69+
_SIGRTMIN = 0x20
70+
6971
_FPE_INTDIV = 0x1
7072
_FPE_INTOVF = 0x2
7173
_FPE_FLTDIV = 0x3

‎src/runtime/defs_linux_mipsx.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,8 @@ const (
6666
_SIGXCPU = 0x1e
6767
_SIGXFSZ = 0x1f
6868

69+
_SIGRTMIN = 0x20
70+
6971
_FPE_INTDIV = 0x1
7072
_FPE_INTOVF = 0x2
7173
_FPE_FLTDIV = 0x3

‎src/runtime/defs_linux_ppc64.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,8 @@ const (
6363
_SIGPWR = 0x1e
6464
_SIGSYS = 0x1f
6565

66+
_SIGRTMIN = 0x20
67+
6668
_FPE_INTDIV = 0x1
6769
_FPE_INTOVF = 0x2
6870
_FPE_FLTDIV = 0x3

‎src/runtime/defs_linux_ppc64le.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,8 @@ const (
6363
_SIGPWR = 0x1e
6464
_SIGSYS = 0x1f
6565

66+
_SIGRTMIN = 0x20
67+
6668
_FPE_INTDIV = 0x1
6769
_FPE_INTOVF = 0x2
6870
_FPE_FLTDIV = 0x3

‎src/runtime/defs_linux_riscv64.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,8 @@ const (
6565
_SIGPWR = 0x1e
6666
_SIGSYS = 0x1f
6767

68+
_SIGRTMIN = 0x20
69+
6870
_FPE_INTDIV = 0x1
6971
_FPE_INTOVF = 0x2
7072
_FPE_FLTDIV = 0x3

‎src/runtime/defs_linux_s390x.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,8 @@ const (
6464
_SIGPWR = 0x1e
6565
_SIGSYS = 0x1f
6666

67+
_SIGRTMIN = 0x20
68+
6769
_FPE_INTDIV = 0x1
6870
_FPE_INTOVF = 0x2
6971
_FPE_FLTDIV = 0x3

‎src/runtime/os3_solaris.go

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -634,3 +634,12 @@ func sysauxv(auxv []uintptr) {
634634
}
635635
}
636636
}
637+
638+
// sigPerThreadSyscall is only used on linux, so we assign a bogus signal
639+
// number.
640+
const sigPerThreadSyscall = 1 << 31
641+
642+
//go:nosplit
643+
func runPerThreadSyscall() {
644+
throw("runPerThreadSyscall only valid on linux")
645+
}

‎src/runtime/os_aix.go

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -373,3 +373,12 @@ func setNonblock(fd int32) {
373373
flags := fcntl(fd, _F_GETFL, 0)
374374
fcntl(fd, _F_SETFL, flags|_O_NONBLOCK)
375375
}
376+
377+
// sigPerThreadSyscall is only used on linux, so we assign a bogus signal
378+
// number.
379+
const sigPerThreadSyscall = 1 << 31
380+
381+
//go:nosplit
382+
func runPerThreadSyscall() {
383+
throw("runPerThreadSyscall only valid on linux")
384+
}

‎src/runtime/os_darwin.go

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -459,3 +459,12 @@ func sysargs(argc int32, argv **byte) {
459459
func signalM(mp *m, sig int) {
460460
pthread_kill(pthread(mp.procid), uint32(sig))
461461
}
462+
463+
// sigPerThreadSyscall is only used on linux, so we assign a bogus signal
464+
// number.
465+
const sigPerThreadSyscall = 1 << 31
466+
467+
//go:nosplit
468+
func runPerThreadSyscall() {
469+
throw("runPerThreadSyscall only valid on linux")
470+
}

‎src/runtime/os_dragonfly.go

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -324,3 +324,12 @@ func raise(sig uint32) {
324324
func signalM(mp *m, sig int) {
325325
lwp_kill(-1, int32(mp.procid), sig)
326326
}
327+
328+
// sigPerThreadSyscall is only used on linux, so we assign a bogus signal
329+
// number.
330+
const sigPerThreadSyscall = 1 << 31
331+
332+
//go:nosplit
333+
func runPerThreadSyscall() {
334+
throw("runPerThreadSyscall only valid on linux")
335+
}

‎src/runtime/os_freebsd.go

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -460,3 +460,12 @@ func raise(sig uint32) {
460460
func signalM(mp *m, sig int) {
461461
thr_kill(thread(mp.procid), sig)
462462
}
463+
464+
// sigPerThreadSyscall is only used on linux, so we assign a bogus signal
465+
// number.
466+
const sigPerThreadSyscall = 1 << 31
467+
468+
//go:nosplit
469+
func runPerThreadSyscall() {
470+
throw("runPerThreadSyscall only valid on linux")
471+
}

0 commit comments

Comments
 (0)