Optimise Tensorflow MaskRCNN from zoo with OpenVINO

Question

This subject is related to this question on Intel Forum ==> Here <==

I want to run tensorflow MaskRCNN to segment some defect but i have some contraints :

• No hardware changes (i5-64bit, no GPU)
• Inference time should be shorter than acquisition time (3sec / images)

To do this, i used mask_rcnn_inception_v2_coco from Tensorflow object detection zoo.

1. Retrain model with my own features => OK
2. Generate a frozen_inference_graph =>OK
3. Build a Watchdog to process images => OK

The problem i face is that the operation to process image is 12sec. In order to solve this problem, i tryed to use OpenVINO Model optimiser solution :

After installing the library, i used this command :

python .\mo_tf.py --input_model "<path>\frozen_inference_graph.pb"
--tensorflow_use_custom_operations_config 
extensions/front/tf/mask_rcnn_support_api_v1.11.json 
--tensorflow_object_detection_api_pipeline_config 
'<path>\mask_rcnn_inception_v2_coco.config'

[ SUCCESS ] Generated IR model.
[ SUCCESS ] XML file: <path>\frozen_inference_graph.xml
[ SUCCESS ] BIN file: <path>\frozen_inference_graph.bin
[ SUCCESS ] Total execution time: 24.00 seconds.

Next, i wanted to build my own inference engine in Python. I did it this way :

# Loading Network :

network=IENetwork(model=MODEL_XML,weights=MODEL_BIN)
network.add_outputs("detection_output")

input_wrapper = next(iter(network.inputs))
n, c, h, w = network.inputs[input_wrapper].shape
out_wrapper = next(iter(network.outputs))

plugin=IEPlugin(device="CPU")
log.info("Loading CPU Extension...")
plugin.add_cpu_extension(CPU_EXTPATH)

supported_layers = plugin.get_supported_layers(network)
not_supported_layers = [l for l in network.layers.keys() if l not in supported_layers]
if len(not_supported_layers) != 0:
  log.error("Not Supported Layers : "+str(not_supported_layers))

Execution_Network=plugin.load(network=network)
del network
log.info("Network Loaded")

# Inference :
image_np = cv2.imread(imagepath)
im = cv2.cvtColor(image_np,cv2.COLOR_BGR2RGB)
image_org=image_np.copy()
#used to resize image with good dimensions
i0=image_to_tensor(im,c,h,w)
res=Execution_Network.infer(inputs={input_wrapper: i0})

To generate the tensor, i use this function :

def image_to_tensor(image,channels,h,w,info=""):
  print(image[0])
  image_tensor=np.zeros(shape=(1,channels,h,w),dtype=np.float32)
  if image.shape[:-1]!=(h,w):
    log.warning("Image {} is resized from {} to {}".format(info, image.shape[:-1],(h,w)))
  image=cv2.resize(image,(w,h))
  image = image.transpose((2, 0, 1))
  image_tensor[0]=image

  return image_tensor

After this, i've build my own custom functions in order to process Boundingbox and masks (like a wrapper with Tensorflow ObjectDetectionAPI methods).

All of this seems to work like a charm. The problem, is that comparing to Tensorflow, probability are lower and classes are wrong.

Openvino : Tensorflow :

When i use OpenVINO maskrcnn_demo with the same network, it seems to work :

Average running time of one iteration: 6774.36 ms
[ INFO ] Processing output blobs
[ INFO ] Detected class 16 with probability 0.98652: [2043.3, 1104.9], [2412.87, 1436.52]
[ INFO ] Image out.png created!
[ INFO ] Execution successful

Then the problem is with python engine and not model export.enter code here

Do somebody already faced such behavior ? Is it normal and due to OpenVINO's optimisations or there is issues in my way to do this ?

Thanks !