proposal: runtime garbage profile

[benburkert]

For latency sensitive applications, excess garbage¹ can have a major impact on tail latencies. The alloc reporting feature of benchmarks is a excellent tool for identifying garbage, but cannot be used during runtime on live applications. And heap profiling can show live objects and allocs but it is often difficult to discern garbage from regular heap allocated objects. A profile that records short lived objects and their allocation sites over a given duration could be a useful tool for identifying sources of garbage.

The memory statistics provided by the runtime package do not provide exact numbers for garbage allocations, but likely provide enough information for accurate estimation. One way to estimate garbage is by comparing memory statistics before and after each GC pass. Taking the minimum value between a) alloced objects/bytes before GC and b) freed objects/bytes after GC gives an estimate of garbage per GC. Repeating this over a long enough duration may produce a statistically accurate estimate of garbage.

Please see the github.com/benburkert/pprof-garbage package proof of concept and demo.

1: garbage being defined as extremely short lived objects/bytes: marked as live for less than one period between GC passes.

[quentinmit]

/cc @aclements who was thinking about something very similar (putting dead objects in the normal heap profile)

[adg]

@aclements ping!

[aclements]

And heap profiling can show live objects and allocs but it is often difficult to discern garbage from regular heap allocated objects.

In the default "-inuse_space" mode, it shows only reachable objects, so there isn't any garbage to discern.

A profile that records short lived objects and their allocation sites over a given duration could be a useful tool for identifying sources of garbage.

Have you tried taking two heap profiles and using go tool pprof -alloc_space -base <before> <after>?

If this doesn't solve the problem, could you take a look at my proposal at #13463 (comment)? The aim of that proposal is to more clearly surface sources of short-lived garbage.

[adg]

Any news here? Leaning toward declining, based on lack of interest.

[rsc]

It sounds like the right place for further discussion is #13463, which is already about more detail in memory profiles.