protoc-gen-go: consider applying M option to prefixes #992
Comments
|
When one For example, consider: // a.proto
syntax = "proto3";
import "b.proto";
message A {
B field_b = 1;
}// b.proto
syntax = "proto3";
message B {}When we generate type A {
FieldB B
}But if they are not in the same package, import "example.com/package/b"
type A {
FieldB b.B
}But how do we tell what the Go package associated with a The protobuf language has a concept of a package (e.g., Relying on the filename doesn't work, because there's no guarantee that the filesystem layout of the Currently, The best way to tell the code generator the Go package associated with a |
|
Yes for sure - I know it's a difficult problem. I was actually on the original issue that added However, I think while the original intention was good, it's had some really bad unintended consequences that effectively have led to many (including myself) not using long-term The Protobuf schema shouldn't be used to dictate this so strongly, there should be flexibility in generation. However, I recognize that I'm saying this is a problem without proposing a solution. I think we should come up with something better in the long term. Just as a simple idea, one option that I think covers many use cases is a variation on the This doesn't handle the |
|
When it comes to deriving the Go package paths for generated packages, there's primarily two ways to sensibly do it: 1) encode it in the The intention of the warning is to push the world towards either of those two. Inside Google, we primarily use the Also, the |
|
I'll double check if the warning is present with with Note that it would be worth having that discussion - the |
|
I'd want to think about it, but I definitely feel the pain of trying to use the |
|
That would be great. I'm happy to lend a hand if you're open to PRs. Another related note, but probably worthy of a new issue: I would love to be able to generate assets produced by Having this ability to specify the location of the core Golang types from |
neild
changed the title
Add a generator option that strips a prefix from the generated filenames. Consider this case: We have google/protobuf/empty.proto, with a go_package option of "google.golang.org/protobuf/types/known/emptypb". We want to generate the code for this file, placing it into the appropriate directory of our repository. In the default mode used by the code generator (paths=import), the generator outputs the file: google.golang.org/protobuf/types/known/emptypb/empty.pb.go This is close to what we want, but has an unnecessary "google.golang.org/protobuf/" prefix. In the GOPATH world, we could pass --go_out=$GOPATH to protoc and get a generated file in the desired location, but this path is not useful in the modules world. The 'module' option allows us to strip off the module prefix, generating the desired filename (types/known/emptypb/empty.pb.go): protoc --go_out=. --go_opt=module=google.golang.org/protobuf google/protobuf/empty.proto The module name must be an exact, character-for-character match. This matches protoc's file handling in general. Default to and require the paths=import option when module= is specified, since it only makes sense when combined with it. Updates golang/protobuf#992. Change-Id: Idbfe4826b6c0ece30d64dbc577131a4f16391936 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/219298 Reviewed-by: Joe Tsai <[email protected]>
|
A couple thoughts:
|
|
Just to revitalize this discussion: where did we land on this? Requiring |
I think I'm missing something on prefix of "", do you mean I feel like this only comes into play if you had
Might be nice, I still think |
|
I mean The proto compiler doesn't provide local filesystem paths to If |
|
Yea I wasn't thinking clearly, definitely true re: local fs paths (and the bane of my existence). Some other options:
|
|
I whipped up https://go-review.googlesource.com/c/protobuf/+/240238 when sketching out a fix for #1158 a few months ago. @dsnet pointed me over here to this issue. It might not be the full solution being discussed here, but I'm interested if people think it's headed in the right direction, or if we need extensive rework to make sense of things. |
|
An expansion of environment variables would solve the problem: This mechanism could be used for providing values for other options as well. By the way, for the time being I solve this problem using an old-school technique: rm -rf $PROTO_BUILD/*
cp -r $PROTO_ROOT/* $PROTO_BUILD/
find $PROTO_BUILD -name '*.proto' -exec sed -r -i "s@option go_package = \"(.*)\";@option go_package = \"$GITHUB_PKG/\1\";@g" {} \;Hope that might help someone. |
|
Is there a solution now that is something like the one @bufdev proposed |
I was testing out the v2 plugin and got this:
Can this be discussed? I would strongly argue against requiring long-form
go_packagevalues in the Protobuf schema definition. This ties a Protobuf schema to a specific generated code location, effectively, which is undesirable in many situations. For example, a lot of existing customers generate code on the fly in individual repositories with different base Golang modules, and this will effectively outlaw that (not that doing so is a good decision, but it's a widely used pattern).This seems very restrictive, and will many a bunch of existing build tools around Protobuf not work. Personally, I've always strongly recommended against using long-form
go_packagevalues because of the multitude of ways that users consume Protobuf definitions.The text was updated successfully, but these errors were encountered: