How to run TensorRT based deep learning model as real time?

Question

I have optimized my deep learning model with TensorRT. A C++ interface is inferencing images by optimized model on Jetson TX2. This interface is providing average 60 FPS (But it is not stable. Inferences are in range 50 and 160 FPS). I need to run this system as real time on real time patched Jetson.

So what is your thoughts on real time inference with TensorRT? Is it possible to develop real time inferencing system with TensorRT and how?

I have tried set high priorities to process and threads to provide preemption. I expect appoximatly same FPS value on every inference. So I need deterministic inference time. But system could not output deterministicaly.

score 0 · Answer 1 · answered Mar 29 '19 at 04:59

0

Have you tried to set the clock on Jetson: sudo nvpmodel -m 0

Here is some links for more information:

answered Mar 29 '19 at 04:59

Pooya Davoodi

147
9

Yes. I am using both "sudo ~/jetson_clocks.sh" and "sudo nvpmodel -m 0" commands. They're improving performance but what i need to do is deterministic inferencing. – Klushka Mar 30 '19 at 06:57
By deterministic inference, do you mean getting the same results for the same inputs to inference? – Pooya Davoodi Apr 10 '19 at 04:33
He wants stable fps. – mibrahimy Apr 17 '19 at 06:41
Yes, I exactly want "stable fps". – Klushka May 06 '19 at 10:27

How to run TensorRT based deep learning model as real time?

1 Answers1