protoc-gen-go: disable recording the versions in generated file

[bmhatfield]

In the v2 version of protoc-gen-go, genGeneratedHeader now outputs a versions section.

We check in our generated protobuf files. This versions output is very problematic for us, as any time there's a minor version bump in protoc (which upgrades automatically via homebrew) all of our generated protobuf files are now different. In order to keep un-necessary changes down, and to avoid diff "flapping", everyone must be on the same minor release of protoc-gen-go and protoc, even if it has no effect on functionality or the actual generated code.

Can we remove that feature or at least offer an option to disable version output?

[dsnet]

This use-case for wanting to disable the version markers is suspect as it's bad practices to have a number of developers working on the same module, but using different versions.

[bmhatfield]

I see where you're coming from, but so far, my experience with protoc minor/patch versions specifically is that they do not affect the output of generated code in any way.

So IMO it's hard to argue that version differences matter there? Perhaps this issue could be reframed as something more like "don't thrash git diffs of generated code when a version change doesn't otherwise affect the code output"?

Given that reframing, it's possible that just hiding sub-versions of protoc might resolve the base problem I'm having.

[dsnet]

Given that reframing, it's possible that just hiding sub-versions of protoc might resolve the base problem I'm having.

By sub-version, do you mean the "PATCH" version per semver terminology? That's something we could consider, but PATCH versions themselves can theoretically introduce changes to the generated .pb.go output and will require more thought.

In general, I recommend that the project enforce some means of ensuring everyone is using the same version of tools for checked in code. This applies to protoc and protoc-gen-go and even go fmt.

[ydnar]

This has bitten us a few times as protoc on macOS developer machines is typically installed via Homebrew, which lags by a few days.

We have dirty tree checks in CI to ensure generated code matches what’s checked into the repo, and tests that verify the behavior of the generated code. If the code changes, CI will catch it. Because of this, the explicit version of the tools used to generate the code is less relevant.

Would you consider adding a flag to disable this?

[scudette]

IMHO it is fine to have a version output in a generated file but it should not update the file if this is the only difference. The code should check if regenerating the file updates it (i.e. updates any of the real lines in the code) and only then change the file.

As a workaround we need to write a hacky python script to clean up the generated files and remove these

[puellanivis]

🤔 We would need to read in the existing file (if it exists) and then do a semantically aware comparison of the file.

Unfortunately, if this is something one truly wants, I think it is a problem that is best solved with a pre-commit check if the only difference is the version like and if so, revert it.

[scudette]

Or it could be an option to not emit those in the first place, which is the most convenient option for some users.

[scudette]

Just for reference to anyone coming to this thread we ended up using sed to clean up the generated files

https://github.com/Velocidex/velociraptor/blob/5e5509145244ebffcef433d957e4500cb4b94a97/make_proto.sh#L64

sed -i -e '1h;2,$H;$!d;g' -re 's|// versions.+// source:|// source:|' $i/*.pb.go

[puellanivis]

I’m not sure the value of “letting protobufs generated by different versions output the same code by omitting version output” fully offsets the value of knowing what version the protobuf was built with.

Adding knobs and dials has effort and cost. Especially, when protoc-gen-go is not being called by us, but by protoc itself.

It is not that I see zero value to this option. But no option or control knob is perfectly free…

[scudette]

When checking the generated files in we reduce the dependency on our users, and we can see when new structs or members in the files are generated due to the git diff anyway. So it becomes very obvious when the file is regenerated using a completely different version of the compiler anyway (because the whole thing would be different).

If protoc would use semantic versioning and would output the major version that would solve both concerns because we know the general layout of the generated code is not changing in incompatible ways without incrementing major versions while minor version differences wont change the file comments.

[puellanivis]

¿I’m confused as to why you would say “if protoc would use semantic versioning”, Because it does? Nothing in the files that is semantically relevant is changing between any of the minor or patch versions.

The comment about minor version differences not changing file comments is additionally weird, because even if we were to flip the case of every single character of nearly all of the comments (all except the autogenerated message), there would be no breaking changes that happen.

[scudette]

My point was that I can see why it would be nice to add a comment to the generated file to document the major version number to ensure compatibility. But indeed if there was a guarantee that compatibility is not affected when major numbers are not changed, then it makes sense to only include the major version number of the generator in the comment.

Doing this will ensure the file does not change unnecessarily while at the same time ensuring that when it does change there is a reason to pay attention to the change (i.e major version number has changed)

Also as you point out the git diff will actually also reveal changes in the actual code generated in case of incompatible changes so it's not really that important to record the version after all.