How to convert segmentation model model to openvino int8 model?

Question

I am trying convert my tensorflow segmentation model to openvino with quantization. I convert my .pb model to intermediate representation with openvino model optimizer. But how quantize model. In official documentation write that to do it with DL workbench. But in workbench i have only detection and classification dataset.

Can i convert my model to int8 without dataset or can i create dataset to segmentation?

Answer 1

The overall flow for converting a model from FP32 to INT8 is:

• Select an FP32 model

• Select an appropriate dataset

• Run a baseline inference

• Configure INT8 calibration settings

• Configure inference settings for a calibrated model

• View INT8 calibration

• View inference results

• Compare the calibrated model with the original FP32 model

Only some convolution models in the FP32 format can be quantized to INT8. If your model is incompatible, you will receive an error message.

The second stage of creating a configuration is adding a sample dataset. You can import a dataset, automatically generate a test dataset consisting of Gaussian distributed noise, or select a previously uploaded dataset.

You can find more details in the below link: http://docs.openvinotoolkit.org/latest/_docs_Workbench_DG_Select_Datasets.html

Answer 2

You can find additional information about low precision inference in OpenVINO here:

1. General approach: https://docs.openvino.ai/latest/openvino_docs_IE_DG_Int8Inference.html
2. Post-Training Optimisation Tool (POT) with default algorithm: https://docs.openvino.ai/latest/pot_docs_LowPrecisionOptimizationGuide.html#doxid-pot-docs-low-precision-optimization-guide

Let me know if you still have questions.