0

There are several Rust crates to support protobufs, but they all require running code generation during the build.rs stage.

Is it possible to generate protobuf-serialization code without the extra build step? I see two ways:

  • using Serde-like approach with the declared structs and possibly some derive or other macros, where the user actually writes out -- this might not work well for arbitrary protobuf serialization because the .proto file might not match well with the user-written structs.
  • using some magical "macro-based inline compiler", where user writes some magical
    protobuf!(include_str!("my_protobuf.proto"))
    and that generates all the needed code inline, without involving a separate build.rs magic. Is this something not possible at all? Or some negatives to this approach? Or just hasn't been done yet?
Yuri Astrakhan
  • 8,808
  • 6
  • 63
  • 97
  • I can verify that both approaches could work (except that the syntax for the second approach will probably be `protobuf_from_file!("my_protobuf.proto")` and not using `include_str!()` since it cannot invoke this macro). I don't know whether it was done, but I see no reason for it to be done. The existing approaches work well. – Chayim Friedman Jun 19 '23 at 18:43
  • 2
    Isn't there a big advantage to `build.rs` in terms of caching? The macro will probably have to run (almost?) every time you build your crate, but the build script will only run when the `.proto` files change, right? – isaactfa Jun 19 '23 at 18:52

1 Answers1

2

The reason most of the crates don't use the Serde-like approach is that serializing protobufs requires additional data that isn't part of the struct, namely the ID of the field. Because Serde doesn't provide a way to expose this functionality and most people don't want to write general-purpose parsing functionality like Serde already provides, in general this approach isn't effective.

It is in theory possible to use a macro to generate the protobuf code. However, as mentioned in the comments, this is much less efficient because it must be done on every compilation rather than cached. In large projects, Rust can already have slow compile times, so it's preferable to avoid recompiling things when possible.

bk2204
  • 64,793
  • 6
  • 84
  • 100
  • wouldn't the build.rs put a lot of complexities for non-cargo users like Bazel/Buck? Also, the build.rs approach may require extras like building the build.rs itself and running it, vs possibly quicker macro execution (already pre-compile-cached)? I wonder if any perf eval comparison was ever done on this... – Yuri Astrakhan Jun 21 '23 at 18:29
  • If Bazel does not invoke `build.rs`, then your crate will be broken, sure, but so will tons of other crates. You can do that, but if it breaks, you keep both pieces. Macro execution failure also produces worse error messages in my experience than regular code, so that may also be a factor. – bk2204 Jun 21 '23 at 21:13