0

I have been debugging this issue for a while now. I have developed an LSTM encoder decoder model which I plan to deploy in C++.

Having saved the model in the .pb file format, I am able to import the model and data and deploy it within python. However, when I try to import the graph using the C++ API, the function TF_GraphImportGraphDef() returns an error code of TF_NOT_FOUND. I suspect, this is because I am using the beam search approach and the symbols for the op GatherTree is not contained with the tensorflow.dll/lib that I generated using Bazel on windows.

I would like to know if anyone has come across this issue before or have any solutions for this issue.

Would linking to the _beam_search_ops.dll be a possible solution? I tried that too using the TF_LoadLibrary() function available in the c_api.h. However, I was unable to load the library.

Any inputs will be appreciated. Also, I am working with tensorflow version 1.14.

vikash singh
  • 1,479
  • 1
  • 19
  • 29
Narayan Narvekar
  • 1
  • 1
  • 1
    Have you checked the message in the `TF_Status` struct? (you can get that with `TF_Message()`). About loading an additional DLL, `TF_LoadLibrary` should work, why is it not working for you? – jdehesa Nov 01 '19 at 11:32
  • Hi, i just found a solution wherein i had to link the '_pywrap_tensorflow_internal.pyd' library as well for some reason. (Source: https://stackoverflow.com/questions/50115117/using-ops-from-tensorflow-contrb-on-windows-via-java-api). Now the dll can be loaded successfuly. However, when i try to import the graph, it still fails, but with a different error code this time, which is, TF_INVALID_ARGUMENT. What do these error codes indicate? Is there any documentation available on these? – Narayan Narvekar Nov 01 '19 at 11:40
  • @NarayanNarvekar Check the docs on those errors here: https://www.tensorflow.org/api_docs/python/tf/errors/InvalidArgumentError ```This may occur, for example, if an operation is receives an input tensor that has an invalid value or shape. For example, the tf.matmul op will raise this error if it receives an input that is not a matrix, and the tf.reshape op will raise this error if the new shape does not match the number of elements in the input tensor.``` Feels odd that you would be seeing it on graph import -- normally I would expect that sort of error to occur during inference. – HanClinto Nov 01 '19 at 17:47
  • @HanClinto I came across this too and realized that it did not make much sense w.r.t this issue. And it is true. The errors are actually identical to the error values in 'error_codes.proto'. After reading the error message through the TF_Message() method, i realised that it was a version mismatch issue between the dlls that were loaded. I will post the solution that worked for me next. – Narayan Narvekar Nov 01 '19 at 20:36

1 Answers1

0

So i dug deeper and managed to find a solution for the following issues which enabled me to load the LSTM encoder decoder saved model successfully with the C++ api:

TF_NOT_FOUND error: The error message read as follows: message: Op type not registered 'GatherTree' in binary running on LAPTOP-5R9P6BHL. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) tf.contrib.resampler should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed. At this stage, i had only linked the tensorflow.dll and tensorflow.lib to my C++ project. A possible solution was to link the library which contains the definition for Gather Tree op, which was the _beam_search_ops.dll.

_beam_search_ops.dll not found error: This error was observed while trying to load the library using TF_LoadLibrary() from the c_api.h. After looking into some posts on stack overflow, it seemed as if the library is dependant on the python37.dll and _pywrap_tensorflow_internal.pyd libraries. And after linking these two libraries, i was successfully able to load the _beam_search_ops.dll.

TF_INVALID_ARGUMENT error: Even after loading the _beam_search_ops.dll library, the graph import failed with the invalid argument error. On reading the error message, i realized that this was due to the _beam_search_ops.dll and _pywrap_tensorflow_internal.pyd being generated with tensorflow V1.13 and the tensorflow.dll/lib and .pb file being generated with tensorflow V1.14. When i linked the _beam_search_ops.dll and _pywrap_tensorflow_internal.pyd from the tensorflow V1.14 version, the loading of the graph was successful and i was able to read the contents of the graph.

If anyone has got a better solution for this issue, please do post it here. Thank you.

Narayan Narvekar
  • 1
  • 1