proto: panic: duplicate enum registered: google.protobuf.FieldDescriptorProto_Type

[Globegitter]

I am trying to use custom options in our codebase, as mentioned here: https://developers.google.com/protocol-buffers/docs/proto#customoptions

Our codebase actively supports python, go and java. So protos are built against these three environments. It seems while custom options work fine for java and python there seems to be an issue with go leading to following error message when running code coverage on one of our tests:

2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.FieldMask
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.Timestamp
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.Any
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.FileDescriptorSet
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.FileDescriptorProto
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.DescriptorProto
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.DescriptorProto.ExtensionRange
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.FieldDescriptorProto
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.EnumDescriptorProto
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.EnumValueDescriptorProto
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.ServiceDescriptorProto
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.MethodDescriptorProto
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.FileOptions
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.MessageOptions
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.FieldOptions
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.EnumOptions
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.EnumValueOptions
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.ServiceOptions
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.MethodOptions
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.UninterpretedOption
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.UninterpretedOption.NamePart
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.SourceCodeInfo
2016/04/27 15:33:58 proto: duplicate proto type registered: google.protobuf.SourceCodeInfo.Location
panic: proto: duplicate enum registered: google.protobuf.FieldDescriptorProto_Type

goroutine 1 [running]:
panic(0x8fa960, 0xc8201552d0)
    /usr/lib/go/src/runtime/panic.go:464 +0x3e6
github.com/golang/protobuf/proto.RegisterEnum(0xbbca60, 0x29, 0xc820199980, 0xc8201999b0)
    /path/to/third_party/go/ptypes/src/github.com/golang/protobuf/proto/properties.go:811 +0xe2
third_party/proto/google/protobuf.init.4()
    third_party/proto/google/protobuf/descriptor.pb.go:1913 +0x706
third_party/proto/google/protobuf.init()
    third_party/proto/google/protobuf/descriptor.pb.go:2798 +0x5cd
main.init()
    /path/to/lobby_test_main.go:149 +0x7e

The issue being here we are importing the module third_party/proto/google/protobuf multiple times like so:

import ( 
...
    _cover9 "third_party/proto/google/protobuf"

    _cover10 "third_party/proto/google/protobuf"

    _cover11 "third_party/proto/google/protobuf"

    _cover12 "third_party/proto/google/protobuf"
)

these are being used to set up the coverage before running the actual tests. If I remove these imports and the coverage instrumentation for them all works fine.

So it seems to me that the init function of the descriptor.pb.go is being called multiple times, which in turn registers the enum multiple times and then fails. I don't quite understand how importing the module multiple times can have any effect. If I import our own protos multiple times (that also register enums) everything seems fine.

Is that an issue with the generated code? And is there any way to fix this? (possibly via #131 ?)

[dsymonds]

Importing the same package multiple times shouldn't cause multiple runs of its init functions. Is it possible you have the same package being imported under more than one import path? That would cause the observed behaviour.

[Globegitter]

@dsymonds Thanks for the response; that should not be happening but I will look into that.

Also just tested #131 and it would indeed fix it given that we would add the right go_package statement upstream. Is there any possibility of this happening?

[Globegitter]

There is nowhere that I can find it being imported under more than one import path right now, but given that a solution more or less already exists in two other existing PRs it would be much preferable to wait for that.

Then we don't have to do our custom imports and this would also be on par with how the python library do it and simplify things quite a bit for cross-langauge support.

[erikreppel]

Did this ever get any kind of resolution? I'm hitting this issue when trying to use the gcp datastore library and protobufs in the same project.

[douglas-reid]

I, too, am seeing this issue when trying to use any of the google cloud client libraries and protobufs in the same project.

[zombiezen]

/cc @jba

I would echo dsymond's question: are you sure you haven't put the same generated proto under multiple import paths?

[jba]

@okdave is working on the go-package option in some files.

We did recently (a month ago?) put a lot of protos in a common place. For example, FieldMask is now in https://github.com/google/go-genproto/blob/master/protobuf/field_mask.pb.go, which has the import path "google.golang.org/genproto/protobuf". Does this also live under another import path?

[okdave]

If you're seeing this, it would be good to understand your project's layout. It's possible that you'll see this issue with vendoring – for example, having the same proto .pb.go file vendored into two different places in your source tree.

[yohcop]

I have the same problem, and I'll try to describe what is happening to me. I've tried many different things/layouts, and had 3 problems - and 0 solutions so far. The enum being registered multiple times was the starting point.

Problem 1

• I use this package, and my code uses descriptor.pb.go from this repo (because I implement a protoc plugin)
• since this repo doesn't include descriptor.proto, I needed to get it from the original repo to use it in my own .proto files
• my.proto file uses the descriptor.proto of course, to add field and message extensions
• I also use Bazel... so my go_proto_library rule for my.proto, depends on the go_proto_library rule I hand-created for descriptor.proto.
• that last rule for descriptor.proto, creates a package, google/protobuf (later I'll explain what happens if that package is set to github.com/golang/....)
• I now have 2 libraries, google/protobuf and github.com/golang/.../descriptor defining those enums. They are both linked because I used this package, as well as my proto which is linked against descriptor.proto.

Problem 2

My code was importing github.com/golang/.../descriptor, but the generated proto from the go_proto_library was importing google/protobuf. So I decided to change my code's imports from github.com/golang/.../descriptor to google/protobuf instead.

The problem next was that the package github.com/golang/../plugin has a plugin.CodeGeneratorRequest. That proto contains a ProtoFile field, which are of type github.com/golang/.../descriptor.FileDescriptorProto. And this type is then incompatible with google/protobuf.FileDescriptorProto.

Problem 3

I then decided to move the google/protobuf package to github.com/golang/.../descriptor. But because I use bazel and that the github.com repository is in an external/ directory I'm not supposed to modify it. So I tried to recreate the directory structure github.com/golang/..../descriptor, and move descriptor.proto and the BUILD file there.

But now, I have two different packages both producing a github.com/golang/..../descriptor.a object file. And Bazel complains about it, of course.

Solution?

I think adding descriptor.proto to this repo would actually solve this. I would then have 2 options:

• create a go_proto_library that uses both the go_library for descriptor.pb.go and a filegroup for descriptor.proto.
• forget about the pre-built descriptor.pb.go and only add a go_proto_library for descriptor.proto.

Any other ideas?

This may be a little contrived, and I hope my explanations are clear enough. It's relatively unlikely @Globegitter's problem is the same exact thing, but who knows...

Thanks!

[yohcop]

So to summarize, yes this was due in my case to having descriptor.pb.go in two packages, linked together, but the problem was created by this repo not having both descriptor.pb.go and descriptor.proto.

https://github.com/google/go-genproto may be a solution, but then again, for this to work this repo should not have a descriptor.pb.go either, or the output of two libraries will clash (problem 3).

I also found #131 like @Globegitter mentioned. I hope this could be resolved soon :)

[junghoahnsc]

I have that same issue. Two other third-party packages are using Descriptor in different package:

org_golang_google_genproto/protobuf/descriptor.pb.go
com_github_golang_protobuf/protoc-gen-go/descriptor/descriptor.pb.go

I think Descriptor should exist in one package. What is the best workaround for now?

[songtianyi]

Any solution here?

[jba]

We are working on de-duplicating the descriptor proto.