C-API Asynchronous Batch Inference with OpenVino

Question

I'm trying to find an example of using the C-API (not C++) asynchronous batch interface for OpenVino image inference.

I'm able to do inference on a single image at a time no problem, but its not clear to me how to expand this to batch inference with the C-API.

Does anyone have an example or reference for doing so? The OpenVino documentation is limited on this front, nor do they provide any C-based examples for doing so that I've been able to find.

EDIT: Per comment below, clarifying that the challenge is in understanding how to load up the input blobs with multiple images. The existing examples either assume the C++ interface and use vectors to move things around, or are unclear in what is idiomatic to the example (e.g. the object detection sample using the C-API)

score 0 · Answer 1 · answered Aug 13 '20 at 18:34

You can use the following functions: ie_infer_request_infer_async() and ie_infer_request_set_batch(). Relevant information is available at the following link: https://docs.openvinotoolkit.org/2020.4/ie_c_api/group__InferRequest.html
I suggest you refer to the following two C API based samples:

Hello Classification C Sample https://docs.openvinotoolkit.org/2020.4/openvino_inference_engine_ie_bridges_c_samples_hello_classification_README.html

Hello NV12 Input Classification C Sample https://docs.openvinotoolkit.org/2020.4/openvino_inference_engine_ie_bridges_c_samples_hello_nv12_input_classification_README.html
High-level description of the process of integrating the Inference Engine into your application is available at the following page: https://docs.openvinotoolkit.org/2020.4/openvino_docs_IE_DG_Integrate_with_customer_application_new_API.html

Hello! Appreciate the prompt response. The challenge I'm having is in packing the input blobs with multiple images using the C approach. Other examples provided either use a single image, or do so using vectored approaches or have what may be idiomatic methods, e.g. object detection. — I.F. Adams, Aug 14 '20 at 19:07

C-API Asynchronous Batch Inference with OpenVino

1 Answers1