Stack Overflow
Stack Overflow

Questions tagged [quantization]

Use this tag for questions related to quantization of any kind, such as vector quantization.

Quantization, in mathematics and digital signal processing, is the process of mapping a large set of input values to a (countable) smaller set.

For more, please read the Wikipedia article for more.

444 questions
0
votes
1 answer

Tensorflow Quantization - Failed to parse the model: pybind11::init(): factory function returned nullptr

I'm working on a TensorFlow model to be deployed on an embedded system. For this purpose, I need to quantize the model to int8. The model is composed of three distinct models: CNN as a feature extractor TCN for temporal prediction FC/Dense as last…
Yaxit
  • 167
  • 2
  • 11
0
votes
0 answers

Why there are some 1s, 0s, and some NAN in the metrics of evaluating the quantized model?

I am doing CNN quantization, I use the following code to calculate some metrics. FP = confusion_matrix.sum(axis=0) - np.diag(confusion_matrix) FN = confusion_matrix.sum(axis=1) - np.diag(confusion_matrix) TP =…
Xiaolin Li
  • 13
  • 1
  • 4
0
votes
1 answer

ValueError: Unknown layer: AnchorBoxes quantization tensorflow

I am applying quantization to a SSD model. The gist is attached. There is a custom object called "AnchorBoxes" which is added while loading the model. This works fine when I don't do quantization. But when I apply quantization, this custom object is…
Sachin Mohan
  • 883
  • 8
  • 15
0
votes
1 answer

Tensorflow Quantization Aware Training

I want to quantize a DenseNet model. I am using Tensorflow 2.4. import tensorflow_model_optimization as tfmot model = tf.keras.applications.DenseNet121(include_top=True,weights=None,input_tensor=None,input_shape=None,pooling=None,classes=1000)…
Clemens Huber
  • 33
  • 4
0
votes
1 answer

How to perform fixed-point quantization in Python

I wish to quantize the weights and biases of an existing Neural Network model. As per my understanding, the fixed-point representation ensures a fixed bit-width of the weights, biases and activations, with pre-determined fixed number of integer and…
DarthCavader
  • 35
  • 6
0
votes
1 answer

Explanation of TensorflowLite inference official tutorial

I am running the below github code for inference on my Raspberry Pi .I have managed to succesfully run my models on my Pi , even though one of them predicts really bad compared to the non quantized version . I have studied the code and libraries but…
Atheros
  • 61
  • 7
0
votes
1 answer

When using generator for representative dataset in quantization it "Failed to convert value into readable tensor"

I am quantizing a model. The model takes 224x224 input. I preprocess the data with a build-in function preprocess_input() which subtracts some center pixels. Now when using a simple image with this preprocessing function in the…
Florida Man
  • 2,021
  • 3
  • 25
  • 43
0
votes
0 answers

Tensorflow (TF2) quantization to full integer error with TFLiteConverter RuntimeError: Quantization not yet supported for op: 'CUSTOM'

At the End is a benefit analysis with Hi Guys, at the moment I'm stuck with the conversion of an .pb Modell to a fully quantized integer TFLite model in TF2. I used a pre-trained model (SSD MobileNet v2 320x320) from the TensorFlow 2 Detection Model…
DemolationPeter
  • 1
0
votes
1 answer

Copy Frozen Values From A Frozen Graph to Another Frozen Graph

I have 2 frozen_graphs which are trained and stored as different pb files. They all share some same nodes. How can I transfer the node value from 1 graph to the other one? For example, how can I copy the FakeQuantWithMinMaxVars nodes to replace the…
dtlam26
  • 1,410
  • 11
  • 19
0
votes
1 answer

Very high error after full integer quantization of a regression network

I have trained a fully connected neural network with one hidden layer of 64 nodes. I am testing with the Medical Cost dataset. With the original precision model, the mean absolute error is 0.22063259780406952. With a model quantized to float16 or…
Samvid Mistry
  • 783
  • 1
  • 8
  • 14
0
votes
0 answers

Different results between quantized TFlite model to its implementation using Numpy

I am working with Tensorflow/Keras and want to quantize model parameters and then implement the model with Numpy. I've build 1D CNN model ,train it, then quantize its parameters , to UINT8 ,using Tensorflow post training quantization , then i've…
ItamarE
  • 41
  • 4
0
votes
1 answer

QAT output nodes for Quantized Model got the same min max range

Recently, I have worked on quantization aware training on tf1.x to push the model to Coral Dev Board. However, when I finished training the model, why is my min max of my 2 outputs fake quantization is the same? Should it be different when one's…
dtlam26
  • 1,410
  • 11
  • 19
0
votes
2 answers

Speed Up Multiple Model Inference on EDGE TPU

I have retrained a RESNET50 model for reidentification on EDGE TPU. However, it seems to be no way to fetch a batch of image to EDGE_TPU. I have come up with a solution of running multiple same model for images. However, is there anyway to speed up…
dtlam26
  • 1,410
  • 11
  • 19
0
votes
1 answer

full-quatization does not except int8 data to change model input layer to int8

I am quantizing a keras h5 model to uint8. To get full uint8 quatization, user dtlam26 told me in this post that the representative dataset should already be in uint8, otherwise the input layer is still in float32. The problem is, that if I feed…
Florida Man
  • 2,021
  • 3
  • 25
  • 43
0
votes
1 answer

Quantizing object detection model

[2] frozen_graph_file = # path to frozen graph (.pb file) [3] input_arrays = ["normalized_input_image_tensor"] [4] output_arrays = ['TFLite_Detection_PostProcess', [5] 'TFLite_Detection_PostProcess:1', [6] …
Knilakshan20
  • 59
  • 6
1 2 3
…
29 30