0

I saw some docs on Intel openvino website. And there is some docs about how to use only one NCS2, the performance is great. Now I have two NCS2, I want to test both of them on a platform, but there is no reference that how to use multi ncs2 to work on one task.

user12701639
  • 21
  • 2

1 Answers1

2

The OpenVINO™ toolkit (>=2019 R2) introduced a Multi-Device Plugin that automatically assigns inference requests to available devices in order to execute the requests in parallel. What this does is allow you to use the Multi-Device Plugin with multiple Intel® Neural Compute Stick 2 devices.

The benchmark_app in C++/Python is a good starting point to check how such plugin works. If you'd like to test drive this Multi-Device plugin, my recommendation would be to follow the article here as it contains a comprehensive and detailed walk-through of testing this feature on both Windows and Linux environments.

The typical "setup" of multi-device can be described in three major steps:

  1. Configuration of each device as usual (e.g. via conventional SetConfig method)
  2. Loading of a network to the Multi-Device plugin created on top of (prioritized) list of the configured devices.
  3. Just like with any other ExecutableNetwork (resulted from LoadNetwork) you just create as many requests as needed to saturate the devices.

For this and more detailed information check Multi-Device Plugin documentation.

avitial
  • 176
  • 6