1

I have a project which uses Google Protobuf2. Now I want to use the prebuilt tensorflow C-API which I integrate via CMake. But now I am getting the following error:

libprotobuf FATAL google/protobuf/stubs/common.cc:61] This program requires version 3.9.0 of the Protocol Buffer runtime library, but the installed version is 2.6.1. Please update your library. If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library. (Version verification failed in "bazel-out/k8-opt/bin/tensorflow/core/framework/tensor_shape.pb.cc".) terminate called after throwing an instance of 'google::protobuf::FatalException' what(): This program requires version 3.9.0 of the Protocol Buffer runtime library, but the installed version is 2.6.1. Please update your library. If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library. (Version verification failed in "bazel-out/k8-opt/bin/tensorflow/core/framework/tensor_shape.pb.cc".)

Is there an easy way to fix it? Tensorflow is not exchanging any messages with the rest of the project, so can I just keep both versions? Downgrading the tensorflow proto is not possible since it is a prebuilt library. Upgrading the rest of the project to proto3 would also take a lot of effort. Can I just use proto 3.9.0 and add syntay="proto2" to all old proto files?

Emilio7773
  • 187
  • 1
  • 9
  • Worst-case, you might be able to use `dlopen` with `RTLD_LOCAL` to "hide" libraries from each other. This is a pain, though, since you have to manually access bind every single symbol you use; it may be simplest if you actually `dlopen` another part of your own app so you can simplify the border. – o11c Aug 28 '21 at 21:40
  • @o11c I think your solution is right! Can you tell me where and how to use dlopen() ? – Emilio7773 Aug 29 '21 at 19:32
  • Have you read the [`dlopen` man page](https://man7.org/linux/man-pages/man3/dlopen.3.html)? (note that `dlmopen` doesn't really help you; it is useful when you need multiple libraries to be aware of each other, but not aware of other sets of libraries). It may be useful to use [x macros](https://en.wikipedia.org/wiki/X_Macro) (the separate file version is saner) to maintain the set of symbols to import across the library boundary, which is much easier if it's your own thing, but it's not strictly necessary. – o11c Aug 29 '21 at 19:51
  • @o11c Yes there was just a little bit of vocabulary that I did not understand. So for my case: I have the tensorflor-c-api consiting of a few header and a few binaries (.so). No I have a c++ wrapper to use the tensorflow-C-Api in my package. And now where do I have to load the dynamic shared object? In my cpp-wrapper header? And than do I really need to put the handlerto before every function that uses a tensorflow fnction? – Emilio7773 Aug 29 '21 at 21:08

1 Answers1

1

The schema language and the runtime are separate. More: proto2 remains the default if you don't add an explicit syntax marker, so strictly speaking you don't need to change the proto files at all, although it will emit warnings recommending you to add a syntax marker. In which case, yes: just add syntax="proto2";

Note, however, that there may be other API changes in the emitted code but that this should be relatively minor.

Marc Gravell
  • 1,026,079
  • 266
  • 2,566
  • 2,900
  • Thank you! But now I get the following error: exit code 139 (interrupted by signal 11: SIGSEGV) – Emilio7773 Aug 29 '21 at 02:29
  • @Emilio7773 a segfault? There's no way I can diagnose that just through a comment (plus: while I know a lot about protobuf, C/C++ isn't my primary language) – Marc Gravell Aug 29 '21 at 09:20
  • I just found this: https://github.com/tensorflow/tensorflow/issues/41080 I think thats exactly the issue. And my only solution right now would be to to load libtensorflow in my application with dlopen with RTLD_DEEPBIND flag. But I do not know what that means – Emilio7773 Aug 29 '21 at 19:25
  • @Emilio7773 `RTLD_DEEPBIND` is in the realm of "scary and might break things in hard-to-understand ways". You really do not want uncontrolled symbol collisions in the first place; my suggestion of `RTLD_LOCAL` entirely *avoids* collisions (and when it fails, it fails in a sane way). – o11c Aug 29 '21 at 19:46