0

OpenVINO API 2.0 changed how images are preprocessed.

Documentation describes an overload used to construct a region of interest tensor from another tensor.

Tensor(const Tensor& other, const Coordinate& begin, const Coordinate& end)

I found a brief example but did not find it clear enough to translate my use case.

/** input_tensor points to input of a previous network and
    cropROI contains coordinates of output bounding box **/
ov::Tensor input_tensor(ov::element::f32, ov::Shape({1, 3, 20, 20}));
ov::Coordinate begin({0, 0, 0, 0});
ov::Coordinate end({1, 2, 3, 3});
//...

My goal is to take an image (cv::Mat) and bounding box (cv::Rect) and execute inference on the ROI without having to copy the ROI memory before calling set_input_tensor (replacement for SetBlob). I have not been able to find sufficient examples or documentation for how to achieve this. I am currently unsure how to translate a cv::Rect into ov::Coordinates.

Previously I was able to do this with API 1.0 using the following example:

const InferenceEngine::ROI roi(0, bounding_box.x, bounding_box.y, bounding_box.width, bounding_box.height);
const InferenceEngine::TensorDesc tensor_desc(InferenceEngine::Precision::U8, { 1, channels, image_height, image_width }, InferenceEngine::Layout::NHWC);
const InferenceEngine::TensorDesc roi_tensor_desc = InferenceEngine::make_roi_desc(tensor_desc, roi, true);
const InferenceEngine::Blob::Ptr image_blob = InferenceEngine::make_shared_blob<uint8_t>(roi_tensor_desc, image.data);
request.SetBlob(input_name, image_blob);
Jared
  • 15
  • 1
  • 4
  • Hi, could you share the details/files of your setup for the API 1.0 and 2.0 for us to replicate? (eg: your custom code/model/etc) – Iffa_Intel Jul 20 '23 at 04:04
  • No, I cannot provide the model files and I do not believe that to be necessary. Additionally, the code being discussed is already in the post. API 1.0 uses "make_roi_desc". This has been replaced in API 2.0 with the functionality listed in the documentation and the provided example, however it is unclear how to translate that to a real use case (take an image and an ROI and pass them to openvino). The example from Intel provides no insights on what the values for the coordinates represent or how to transform a cv::Rect (or any x, y, width, height coordinates) into them. – Jared Jul 20 '23 at 06:29

2 Answers2

0

OpenVINO API 2.0 changed layout work for models with NHWC. If your model has input { 1, channels, image_height, image_width } with NHWC layout, after conversion to IR with cmd '--layout=input_1(NHWC)' '--input_shape=[1, image_height, image_width, channels]', you have the input shape [1, image_height, image_width, channels]. Thus, you need to make ROI relative to this size.

For example, to make ROI tensor, you need:

const auto cv::Rect bounding_box = <>;
const auto shared_tensor = ov::Tensor(<data>); // it's tensor with shared input data
// N, H, W, C
const auto begin = ov::Coordinate({0, bounding_box.y, bounding_box.x, 0}); // the coordinates first bounding box corner
const auto end = ov::Coordinate({1, bounding_box.y + bounding_box.height, bounding_box.x + bounding_box.width, 3}); // the coordinates second bounding box corner
const auto roi_tensor = ov::Tensor(shared_tensor, begin, end);

ov::Coordinate is class to represent coordinates on shapes. In this case these coordinates are of the bounding box.

alexa
  • 16
  • 2
  • This seems to be close. I had to switch bounding_box `x` and `y` and `width` and `height` to match the tensor layout. Using `{0, y, x, 0}` and `{1, y+h, x+w, 3}` works but appears to miss a step that the original API 1.0 version performed. I am now getting an error message: `Can't set input blob, because model input (shape=[1,128,128,3]) and blob (shape=(1.667.1278.3)) are incompatible`. The API 1.0 version consumed the ROI as coordinates from the source image and resized the ROI into the input size. – Jared Jul 24 '23 at 21:42
  • If you fix the y, x, height, width ordering in the comment I'll mark it as solution even though I cannot use it. I'm beginning to think that my use case for ROI does not match Intel's new intended use cases and I will handle resize and then pass the data in through the tensor. Thank you. – Jared Jul 24 '23 at 21:52
  • @Jared yeah, you're right about coordinate order. Thank you! I always mix up them. As for your exception, seems like shared tensor has the big shape. Probably you can resize input shape of model from `[1, 128,128,3]` to `[1, 667, 1278, 3]` (or to any needed shape) if possible. – alexa Jul 25 '23 at 05:36
0

Previously in legacy API, you need to specify h and w dimensions.

Meanwhile, in OV 2.0 API you need to specify the starting and ending of coordinates for the ROI tensor.

You may refer to here for more info.

Iffa_Intel
  • 106
  • 3