Stack Overflow
Stack Overflow

Questions tagged [quantization]

Use this tag for questions related to quantization of any kind, such as vector quantization.

Quantization, in mathematics and digital signal processing, is the process of mapping a large set of input values to a (countable) smaller set.

For more, please read the Wikipedia article for more.

444 questions
1
vote
2 answers

Any idea of to solve the version problem of PyTorch model on android device? The model version must be between 3 and 5But the model version is 7

I am getting the following error while running a PyTorch model on android model? Lite Interpreter verson number does not match. The model version must be between 3 and 5But the model version is 7 () Any suggestion?
Celik
  • 2,311
  • 2
  • 32
  • 54
1
vote
0 answers

Post-training quantization in non-tflite format

I have a TensorFlow model on which I would like to perform post-training quantization. I am using C# for software and the device will be in c++ (where I will use tflite). It would be nice to have the same quantized model in both the software and…
Josh
  • 159
  • 2
  • 10
1
vote
0 answers

RuntimeError: faster_rcnn_fbnetv3a_C4.yaml not available in Model Zoo! in d2go

I am trying to run D2GO_Introduction.ipynb Notebook link file but I am facing some issue with that I am getting that error during getting the model from model zoo ,, "RuntimeError: faster_rcnn_fbnetv3a_C4.yaml not available in Model Zoo" Here is the…
Bappy Ahmed
  • 25
  • 3
1
vote
2 answers

ValueError: Unsupported ONNX opset version: 13

Goal: successfully run Notebook as is on Jupyter Labs. Section 2.1 throws a ValueError, I believe because of the version of PyTorch I'm using. PyTorch 1.7.1 Kernel conda_pytorch_latest_p36 Very similar SO post; the solution was to use the latest…
DanielBell99
  • 896
  • 5
  • 25
  • 57
1
vote
1 answer

Use Quantization on HuggingFace Transformers models

I'm learning Quantization, and am experimenting with Section 1 of this notebook. I want to use this code on my own models. Hypothetically, I only need to assign to model variable in Section 1.2 # load model model =…
DanielBell99
  • 896
  • 5
  • 25
  • 57
1
vote
0 answers

How to access SkiaSharp color palette for quantization purposes?

I'm trying to make a quantizer that works with SkiaSharp right now. For that purpose I need a way to get a ColorPalette or a way to work with indexed color format in SkiaSharp. I see that Index8 is no longer a supported ColorType, and a lot of the…
Pangamma
  • 731
  • 12
  • 28
1
vote
0 answers

How to freeze BN stats in PyTorch Quantization Aware Training when using FX graph mode

Freezing BN stats when doing Quantization Aware Training is a common training technique as introduced in Google Quantization Whitepaper. And PyTorch official tutorial's code snippet also shows that how to do it in PyTorch: num_train_batches = 20 #…
user8510613
  • 1,242
  • 9
  • 27
1
vote
0 answers

how can I do calibration for 8-bit quantization using KL Divergence

I have FP32 tensor inputs which shape are [1, 4, 1024, 256] I need to quantize the tensor to INT8, but naive quantization has triggered a problem in my NLP model that actually get rid of EOS. So I have to do calibration rather than go with 'absolute…
esse non videri
  • 29
  • 2
1
vote
2 answers

Int8 quantization of a LSTM model. No matter which version, I run into issues

I want to use a generator to quantize a LSTM model. Questions I start with the question as this is quite a long post. I actually want to know if you have manged to quantize (int8) a LSTM model with post training quantization. I tried it different TF…
Florida Man
  • 2,021
  • 3
  • 25
  • 43
1
vote
2 answers

Accurate color quantization of image to minimize color palette

I'm trying to quantize an image keeping all primary colors in place and removing all minor colors such as "anti-aliasing" borders. E.g. the image below ultimately should be quantized to 3 colors whereas the number of actual colors in the original…
Greg
  • 137
  • 2
  • 11
1
vote
0 answers

' ValueError: to_quantize can only either be a tf.keras Sequential or Functional model

I've tried QAT implementation in my model training script. I'm using functional API for the model creation. Steps that I followed to implement QAT API, Build the model acrhitecture Inserted the appropriate quantize_model function Train the…
Raisa Alphonse M
  • 35
  • 6
1
vote
1 answer

TFLite Interpreter: defining optimal number of threads

I am running a quantized TFLite model (in Linux PC) for inference using XNNPack backend. I am aware that TFLite models may suffer high latency for prediction and i'm trying to optimize its performance defining number of threads to…
heresthebuzz
  • 678
  • 7
  • 21
1
vote
0 answers

ValueError: Failed to parse the model: pybind11::init(): factory function returned nullptr

After a long search for my problem, any solution fund won’t work for me. I hope that you can help me to overcome this problem so I can continue my project. The problem is while doing post-training integer quantization of a GRU model, it gives me the…
yah_ma
  • 11
  • 2
1
vote
0 answers

Is Elu int8 quantisation working on Tensorflow Lite?

Context : I would like to run inferencing of a DL-model on an Arduino and, since I don't have much memory available, I need to post-training int8-quantize my model. But the quantization of my model doesn't seem to be working, and it seems to be…
EléonoreB
  • 11
  • 2
1
vote
0 answers

How to get around `'Proxy' object does not support item assignment`

Here's the line that triggers this error during torch.quantization.quantize_fx.prepare_fx patch_embed[:, 1:] = patch_embed[:, 1:] + self.proj(self.norm1_proj(pixel_embed).reshape(B, N - 1, -1)) Is there some way to get around this without having to…
Alexander Soare
  • 2,825
  • 3
  • 25
  • 53
1 2 3
…
29 30