Support for recursive definitions in proto files

[yfeng72]

When using the grpc-web protoc plugin, the files generated need to be packed using Closure Compiler. Unfortunately it doesn't really support recursive definitions of protobuf messages, i.e.

Message Node
{
  oneof Contents {
        List list = 1;
        Value val = 2;
    }
}

Message List {
      repeated Node = 1;
}

.....

To represent data structures that are similar to JSON, the structure above is needed. The C++ plugin of protoc supports it, but Closure Compiler does not.

The feature request is to enable such a structure by either generating JS code that uses something other than Closure Compiler, or expand Closure Compiler's functionality to support it.

[stanley-cheung]

Sorry for the late reply. Is this more of an issue for protobuf?

[stanley-cheung]

I looked into this more closely and think that nested message (and other protobuf features) seems to work here.

See PR #226 for a more detailed demo.

I added a more elaborate proto structure to our Echo demo:

message Bar {
  string name = 1;
}
message Baz {
  int32 id = 1;
}
message Foo {
  string val = 5;
  oneof id {
    Bar bar = 6;
    Baz baz = 7;
  }
}
message EchoRequest {
  string message = 1;
  Foo standalone_foo = 2;
  repeated Foo foos = 4;
  oneof identity {
    Foo username = 5;
    int32 userid = 6;
  }
}
message EchoResponse {
  string message = 1;
  int32 message_count = 2;
  Foo standalone_foo = 3;
  repeated Foo foos = 4;
  oneof identity {
    Foo username = 6;
    int32 userid = 7;
  }
}

Here there are multiple-layer nested messages, oneof's, repeated field, and a mixture of those features. I am able to set these fields from JS (generated code from the standard javascript protoc plugin), echo them back from the backend c++ server, and reads them back from JS.

This also works for the both Closure style and CommonJS style import.

So this appears to be working fine as is. If there are a specific use case that doesn't work, please re-open this issue and I will be happy to investigate.

[stanley-cheung]

OP clarified that this is about recursive definitions in proto messages, not just nested. Will investigate more.

[stanley-cheung]

Yes indeed. I made another demo (#227) which features some recursive proto definitions:

message Node {
  oneof contents {
    List list = 1;
    string val = 2;
  }
}
message List {
  repeated Node nodes = 1;
}

This does work with the CommonJS style import. This pieces of JS code output node1.val = node1.

var node1 = new Node();
node1.setVal("node1");

var list = new List();
list.setNodesList([node1]);

var node2 = new Node();
node2.setList(list);

console.log("node1.val = "+node2.getList().getNodesList()[0].getVal());

And I am indeed able to reproduce the error with the Closure compiler:

../../../../../net/grpc/gateway/examples/echo/out/node.js:15: ERROR - required "proto.grpc.gateway.testing.List" namespace not provided yet
goog.require('proto.grpc.gateway.testing.List');
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

A search shows that it has been an open issue on the Closure Compiler for 2 years.

Since we do have a workaround now here for gRPC-Web, I will close this issue.