0

I am running the below github code for inference on my Raspberry Pi .I have managed to succesfully run my models on my Pi , even though one of them predicts really bad compared to the non quantized version . I have studied the code and libraries but I have difficilty understanding 2 small parts that I believe affect my model's performance .

Tensorflow's official code for running inference on tflite models . https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/examples/python/label_image.py

The two parts are :

parser.add_argument(
      '--input_mean',
      default=127.5, type=float,
      help='input_mean')
  parser.add_argument(
      '--input_std',
      default=127.5, type=float,
      help='input standard deviation')

I can't understand how those 2 arguments affect the Inference and why they are used in this command. How are the args values supposed to alter the input data .

if floating_model:
    input_data = (np.float32(input_data) - args.input_mean) / args.input_std

Second , when is a model np.float32 type ? Aren't quantized models int type ?

floating_model = input_details[0]['dtype'] == np.float32
Atheros
  • 61
  • 7

1 Answers1

0

how those 2 arguments affect the Inference

Application of mean and standard deviation called "Normalization". This operation needed to change range of input data. In your case input data has range 0...255, while floating point model requires range -1...1. This only depends on model a model architecture and data used for training. See details with example about normalization parameters

Aren't quantized models int type ?

Actually there are different kinds of quantization (for example Float16) but most commonly associated types with quantization are int8/uint8. Model provided in your example is FP. Check your model inputs-outputs with app like https://netron.app/

In the end: looks like input_mean/input_std changes are pretty old and to me they do not match model they offer to use. According to model you do not need normalization, model takes as input float values in range 0...255.

Alex K.
  • 842
  • 7
  • 17
  • Thanks for the answer . So in case my model accepts values in ranged 0-255 , couldn't I achieve normalization just by dividing input with 255 in the range [0,1]? How can I know if my input should be in range [-1,1] or [0,1] . With trial and error they both seem to be working isn't this strange – Atheros Dec 10 '20 at 09:20
  • If input model requires range 0...255 you actually need to be sure your input data in that range: if you have FP with range 0..1 you should normalize it into range (multiply by 255); if you have integers 0...255, you good, just cast to type; if you already in FP in range 0..255 no normalization required. Input information can be found in model metadata(like our case) or in supplement docs with model, normalization is usually part of preprocessing. – Alex K. Dec 10 '20 at 09:37
  • Regarding [-1, 1] and [0, 1]: it all depends on what model requires and how its deal with over ranged values. Example: model input [-1, 1], feed image [0, 1] - see as reduction of dynamical range, your images will be "brighter" to net. It can be insensitive to it. If model [0, 1] and feed [-1, 1] values can be clamped but still 50% information available. It can work but with dramatic drop in accuracy. – Alex K. Dec 10 '20 at 09:41