protobuf incompatibility when installing tensorflow model garden

Question

I try to test my installation of tensorflow model garden and execute the following command in my PowerShell: python object_detection/builders/model_builder_tf2_test.py Thereby, I get the following error message:

Traceback (most recent call last):
  File "...\Documents\TensorFlow\models\research\object_detection\builders\model_builder_tf2_test.py", line 21, in <module>
    import tensorflow.compat.v1 as tf
  File "...\Anaconda3\lib\site-packages\tensorflow\__init__.py", line 37, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "...\Anaconda3\lib\site-packages\tensorflow\python\__init__.py", line 37, in <module>
    from tensorflow.python.eager import context
  File "...\Anaconda3\lib\site-packages\tensorflow\python\eager\context.py", line 29, in <module>
    from tensorflow.core.framework import function_pb2
  File "...\Anaconda3\lib\site-packages\tensorflow\core\framework\function_pb2.py", line 16, in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
  File "...\Anaconda3\lib\site-packages\tensorflow\core\framework\attr_value_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
  File "...\Anaconda3\lib\site-packages\tensorflow\core\framework\tensor_pb2.py", line 16, in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
  File "...\Anaconda3\lib\site-packages\tensorflow\core\framework\resource_handle_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
  File "...\Anaconda3\lib\site-packages\tensorflow\core\framework\tensor_shape_pb2.py", line 36, in <module>
    _descriptor.FieldDescriptor(
  File "...\Anaconda3\lib\site-packages\google\protobuf\descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

In my opinion, I need to install a later version than 3.19.0. of protoc, because the cal from a _pb2.py - file. As a result I looked for the version of protoc with pip show protobuf and get the following result:

Name: protobuf
Version: 4.21.1
Summary:
Home-page: https://developers.google.com/protocol-buffers/
Author: protobuf@googlegroups.com
Author-email: protobuf@googlegroups.com
License: 3-Clause BSD License
Location: ...\anaconda3\lib\site-packages
Requires:
Required-by: tensorflow, tensorflow-metadata, tensorflow-hub, tensorflow-datasets, tensorboard, proto-plus, googleapis-common-protos, google-api-core, apache-beam

So, there is installed a later version than 3.19.0 but it doesn't work.

Why does it not work and what can I do that it works?

For your information: I used the tutorial of tensorflow on this site.

Stephen Ostermiller · Answer 1 · 2022-10-05T19:54:25.573

Google introduced a new breaking change in protobuf-4.21.0. Anything later than that will not work. You need to install protobuf 3.20.x or earlier (like version 3.20.3.)

Alternately, you could set the environment variable that it recommends. I have found that it is convenient to put it into your __init__.py file like:

import os

# Google introduced an incompatibility into protobuf-4.21.0
# that is not backwards compatible with many libraries.  
# Once those libraries have updated to rebuild their _pb2.py files,
# this can be removed.
os.environ["PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION"] = "python"

The other solution would be to upgrade Tensorflow if they have a new version out that rebuilds with the latest protobuf.

protobuf incompatibility when installing tensorflow model garden

1 Answers1