cmd/go: allow 'mod verify' to use multiple CPUs

[mvdan]

For example, on a moderately sized module of mine:

$ /usr/bin/time go mod verify
all modules verified
13.33user 2.39system 0:15.78elapsed 99%CPU (0avgtext+0avgdata 24472maxresident)k
1558152inputs+0outputs (0major+4086minor)pagefaults 0swaps

Given that I have four physical CPU cores, and that the bottleneck seems to be hashing moderately sized zip files on disk, I think there's plenty of room for improvement.

For example, if we verified up to GOMAXPROCS concurrently, I expect the process could easily get two or three times faster on my laptop.

I realise that this command doesn't need to be super fast, but I make use of it for "module sanity checks" in CI along with others like go mod tidy, so it's unfortunate that it doesn't make proper use of the resources available.

/cc @bcmills @jayconrod @matloob

[mvdan]

I also wonder if there are other commands which are similarly bottlenecked on a single CPU and could be improved. I haven't really noticed any others; for example, go mod tidy regularly uses at least three of my cores.

[jayconrod]

I also wonder if there are other commands which are similarly bottlenecked on a single CPU and could be improved.

Definitely. @matloob is looking at adding tracing that would help identify bottlenecks.

[matloob]

Yeah I'm hoping tracing will help us find more of those commands.

[mvdan]

This was marked as "help wanted", and I think it's a safe and easy change to make a week before the freeze, so I'll mail a CL shortly.

[gopherbot]

Change https://golang.org/cl/229817 mentions this issue: cmd/go: make 'mod verify' use multiple CPUs