How to encode/decode nested messages using protobuf?

I have a protobuf structure that represents nested layers/envelopes of structures. When encoding this data, the entire hierarchy can be processed in one set.

However, when decoding only the first outer envelop should be decoded and the nested envelope needs to remain encoded to be passed as is downstream for decoding by the next architectural layer, etc. Each layer is only aware of the structure of the envelop intended for it.

Right now I have a separate proto file for each envelope and nested envelopes are represented in their parents as bytes. I don't like this approach as it requires maintenance of separate files and non-trivial code for encoding.

Is there a better way to handled this? And a bonus question, is there a better way when using nanopb?

1 Answer
1

If you're using proto2 (rather than proto3), this would seem to be a good fit for extensions. Each "outer" message would define an extension range, for example (to quote from the doc):

message Foo
// ...
extensions 100 to 199;



Then in your "inner" schema, you can declare additional fields for the parent message - either trivial fields, or entire objects:

message Bar /*...*/
extend Foo
optional Bar bar = 126;



The "inner" code then uses whatever the "extensions" API is for your chosen framework; if you're using the Google implementation, that is a range of methods like HasExtension(), ClearExtension(), GetExtension(), MutableExtension(), and AddExtension() - but different implementations may have different APIs.

HasExtension()

ClearExtension()

GetExtension()

MutableExtension()

AddExtension()

With this approach, the "outer" message never has (or needs) any knowledge of the inner data; the library deals with all of it.

The great thing about this approach is that it is byte-wise idential to using bytes - so your existing data should be fully compatible to if you had used:

bytes

message Foo
optional bytes bar = 126;



Note that proto3 does not implement extensions. There is a new Any concept that offers broadly similar outcomes, but works very differently. It is not byte-wise compatible.

Any

Thanks for contributing an answer to Stack Overflow!

But avoid …

To learn more, see our tips on writing great answers.

Required, but never shown

Required, but never shown

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.