runtime: fractional workers can cause 20ms mutator latency #21698
Labels
Milestone
Comments
|
Change https://golang.org/cl/68573 mentions this issue: |
|
Change https://golang.org/cl/68574 mentions this issue: |
|
Change https://golang.org/cl/68571 mentions this issue: |
gopherbot
pushed a commit
that referenced
this issue
Oct 13, 2017
When GOMAXPROCS is not small, fractional workers don't add much to throughput, but they do add to the latency of individual goroutines. In this case, it makes sense to just use dedicated workers, even if we can't exactly hit the 25% CPU goal with dedicated workers. This implements this logic by computing the number of dedicated mark workers that will us closest to the 25% target. We only fall back to fractional workers if that would be more than 30% off of the target (less than 17.5% or more than 32.5%, which in practice happens for GOMAXPROCS <= 3 and GOMAXPROCS == 6). Updates #21698. Change-Id: I484063adeeaa1190200e4ef210193a20e635d552 Reviewed-on: https://go-review.googlesource.com/68571 Reviewed-by: Rick Hudson <[email protected]>
gopherbot
pushed a commit
that referenced
this issue
Oct 13, 2017
Currently only a single P can run a fractional mark worker at a time. This doesn't let us spread out the load, so it gets concentrated on whatever unlucky P picks up the token to run a fractional worker. This can significantly delay goroutines on that P. This commit changes this scheduling rule so each P separately schedules fractional workers. This can significantly reduce the load on any individual P and allows workers to self-preempt earlier. It does have the downside that it's possible for all Ps to be in fractional workers simultaneously (an effect STW). Updates #21698. Change-Id: Ia1e300c422043fa62bb4e3dd23c6232d81e4419c Reviewed-on: https://go-review.googlesource.com/68574 Run-TryBot: Austin Clements <[email protected]> TryBot-Result: Gobot Gobot <[email protected]> Reviewed-by: Rick Hudson <[email protected]>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
What version of Go are you using (
go version)?1.9 (or tip b039215)
What did you expect to see?
Fractional mark workers should have minimal impact on mutator latency.
What did you see instead?
The fractional mark worker often runs for 20ms at a time, which means other goroutines on the same P can be significantly delayed. (In contrast, dedicated mark workers kick all other goroutines on the P to the global run queue, where they're more likely to get picked up by other Ps.)
We should consider imposing a tight preemption bound of fractional workers (they'd have to voluntarily preempt without sysmon's help). At larger GOMAXPROCS, we may want to simply not use fractional workers.
This problem, and others, was observed in #18534 (see, in particular, #18534 (comment)).
/cc @RLH @tarm
The text was updated successfully, but these errors were encountered: