runtime: break out new minimum heap size into a goexperiment

The new minimum heap of 512 KiB has been the cause of some build
slowdown (~1%) and microbenchmark slowdown (usually ~0%, up to ~50%)
because of two reasons:
1. Applications with lots of small short-lived processes execute many
   more GC cycles.
2. Applications with heaps <4 MiB GC up to 8x more often.

In many ways these consequences are inevitable given how GOGC works,
however we need to investigate more as to whether the apparent slowdowns
are indeed unavoidable or if the GC has issues scaling down, which it's
too late for for this release.

Given that this release is already huge, it's OK to push this back.
We'll take a closer look at it next cycle, so place block it behind a
new goexperiment to allow users and ourselves to easily experiment with
it.

Fixes #49744.
Updates #44167.

Change-Id: Ibad51f7873de7517490c89802f3c593834e77ff0
Reviewed-on: https://go-review.googlesource.com/c/go/+/368137
Trust: Michael Knyszek <[email protected]>
Run-TryBot: Michael Knyszek <[email protected]>
TryBot-Result: Gopher Robot <[email protected]>
Reviewed-by: Austin Clements <[email protected]>
Reviewed-by: David Chase <[email protected]>

Author: mknyszek

src/internal/goexperiment/exp_heapminimum512kib_off.go

@@ -89,4 +89,11 @@ type Flags struct {
 	// Details regarding the new pacer may be found at
 	// https://golang.org/design/44167-gc-pacer-redesign
 	PacerRedesign bool
+
+	// HeapMinimum512KiB reduces the minimum heap size to 512 KiB.
+	//
+	// This was originally reduced as part of PacerRedesign, but
+	// has been broken out to its own experiment that is disabled
+	// by default.
+	HeapMinimum512KiB bool
 }

src/internal/goexperiment/exp_heapminimum512kib_on.go

@@ -53,8 +53,8 @@ const (
 	gcOverAssistWork = 64 << 10
 
 	// defaultHeapMinimum is the value of heapMinimum for GOGC==100.
-	defaultHeapMinimum = goexperiment.PacerRedesignInt*(512<<10) +
-		(1-goexperiment.PacerRedesignInt)*(4<<20)
+	defaultHeapMinimum = (goexperiment.HeapMinimum512KiBInt)*(512<<10) +
+		(1-goexperiment.HeapMinimum512KiBInt)*(4<<20)
 
 	// scannableStackSizeSlack is the bytes of stack space allocated or freed
 	// that can accumulate on a P before updating gcController.stackSize.