2

For CPP integration we use TensorFlow lib from here https://www.tensorflow.org/install/lang_c

As we saw the first TensorFlow session run is time-consuming for our specific model it took 20s and next time it took 200ms. We strongly limited in time for inference and also we should serve multiple requests so we decided to do next trick:

Use forked processes to serve our TensorFlow for parallel requests. Init TensorFlow session and run blank inference for the first time in the main process and thus next time run session in the forked process. But it stuck without any error message.

Is there a way to deal with such a problem.

Any suggestion is acceptable.

user10333
  • 331
  • 1
  • 9
  • Have you seen [tensorflow serving](https://github.com/tensorflow/serving) for model inference? – o-90 Oct 30 '19 at 16:00
  • we strongly limited to c++ offline solution. We should provide single lib that does all without any external interchange – user10333 Oct 30 '19 at 16:08

0 Answers0