2

I am fine-tuning SSD Mobilenet v2 for a custom dataset. I am fine-tuning the model for 50k steps and quantization aware training kicks in at 48k step count.

graph_rewriter {
  quantization {
    delay: 48000
    weight_bits: 8
    activation_bits: 8
  }
}

I am observing a 95%+ training, validation and testing mAP post training.

After quantization using the commands

python object_detection/export_tflite_ssd_graph.py 
--pipeline_config_path=${CONFIG_FILE} 
--trained_checkpoint_prefix=${CHECKPOINT_PATH} 
--output_directory=${OUTPUT_DIR} --add_postprocessing_op=true

./bazel-bin/tensorflow/contrib/lite/toco/toco 
 --input_file=${OUTPUT_DIR}/tflite_graph.pb \
 --output_file=${OUTPUT_DIR}/detect.tflite \
 --input_format=TENSORFLOW_GRAPHDEF \
 --output_format=TFLITE \
 --inference_type=QUANTIZED_UINT8 \
 --input_shapes="1,300,300,3" \
 --input_arrays=normalized_input_image_tensor \
 --output_arrays="TFLite_Detection_PostProcess","TFLite_Detection_PostProcess:1","TFLite_Detection_PostProcess:2","TFLite_Detection_PostProcess:3" \
 --std_values=128.0 --mean_values=128.0 --allow_custom_ops --default_ranges_min=0 --default_ranges_max=6

I tested the generated detect.tflite model using same test set. I see a drop in mAP to about 85%.

Is this mAP number drop to be expected? How can I improve the post quantization mAP?

Anil Maddala
  • 898
  • 16
  • 34
  • Are you able to ensure that the input images are pre-processed in the same way? We have seen mAP is affected by that. We have not seen such a big drop in accuracy in quantized models for SSD MobileNet V2. – achowdhery Feb 08 '19 at 00:09
  • @achowdhery I am using out of the box Tensorflow Lite Android Object Detection application for the metrics with new classes, re-trained model. I remember the pre-processing step to be the same. Please point me to anything specific you want me to verify. – Anil Maddala Apr 26 '19 at 17:10

0 Answers0