Cleanup go.sum

[grosser]

there are so many duplicated dependencies in there, which

• makes installation take forever
• could add bugs if multiple versions of the same library are used
• increases the binary size

for example:

k8s.io/apimachinery v0.0.0-20180126010752-19e3f5aa3adc h1:+YMPfU/2f9LMz0Sy68+JodreTTgo9PNSpkLw3PakNbs=
k8s.io/apimachinery v0.0.0-20180126010752-19e3f5aa3adc/go.mod h1:ccL7Eh7zubPUSh9A3USN90/OzHNSVN6zxzde07TDCL0=
k8s.io/apimachinery v0.0.0-20181015213631-60666be32c5d/go.mod h1:ccL7Eh7zubPUSh9A3USN90/OzHNSVN6zxzde07TDCL0=
k8s.io/apimachinery v0.0.0-20190404173353-6a84e37a896d/go.mod h1:ccL7Eh7zubPUSh9A3USN90/OzHNSVN6zxzde07TDCL0=
k8s.io/apimachinery v0.0.0-20190816221834-a9f1d8a9c101 h1:QtHYUjIdgXTtJVdYQhWIQZZoXa32aF3O9BNX2up2plE=
k8s.io/apimachinery v0.0.0-20190816221834-a9f1d8a9c101/go.mod h1:ccL7Eh7zubPUSh9A3USN90/OzHNSVN6zxzde07TDCL0=

I'd propose some CI step that makes sure it is only using 1 version per dependency ... does that make sense ?

[xueweiz]

Hi @grosser ,
From my understanding, unused entries in go.sum does not affect anything:

1. The only thing that's really affecting installation or compilation is what we stored in the /vendor directory (since we are using the "-mod vendor" option).
2. The golden rule determining the code in the /vendor directory is stored at go.mod.

I don't think it would slow down installation or increases binary size. And I don't think multiple versions in go.sum hurts anything (other than a few Bytes of storage on the repository :) ).

The closest tool I can think of to trim them, is to run GO111MODULE=on go mod tidy.
According to the documentation, it should remove unnecessary entries in go.mod and go.sum. I ran it once against the current state, and filed #391 .

---

One thing to note out is that even go mod tidy does not clean up the unused versions in go.sum, it only cleans up unused modules. See some related golang upstream discussions: golang/go#28456 golang/go#26381

The current opinion of the upstream folks is that the historical version checksums in go.sum is intended. We will see how it goes. But I think the bottom line is, we can follow upstream's decision, and just use the go mod tidy tool.

---

I also thought about adding a presubmit test or automation to run go mod tidy on every PR. However that's actually a little bit complicated. This is because go mod tidy trims the files using the currently specified build tags (e.g. linux, windows, journald...). NPD is built by different people using different build tags. It's very likely that the automation will accidentally trim modules used by some people in special environments.

So I think the best way is to NOT automate this, unless we can have a good understanding of the build tags that people are using. For now, I think we should just do a manual trim once a while (since it really doesn't hurt anything major).

[xueweiz]

/assign @xueweiz

[grosser]

👍 yeah not worth the effort I guess