Stack Overflow
Stack Overflow

Questions tagged [quantization]

Use this tag for questions related to quantization of any kind, such as vector quantization.

Quantization, in mathematics and digital signal processing, is the process of mapping a large set of input values to a (countable) smaller set.

For more, please read the Wikipedia article for more.

444 questions
4
votes
1 answer

Google Coral Edge TPU compiled model - Inference always almost the same

I'm trying to get a Mobilenetv2 model (retrained last layers to my dataset) to run on the Google edge TPU Coral. I'm able to quantize and compile the model with the 'edgetpu_compiler' (followed this page…
Paulo Ribeiro
  • 153
  • 9
4
votes
1 answer

How can I represent a number in Go using only 4 bits?

I am trying to compress the representation of a number into fewer bits. For example, right now I am using a float64 to represent a number such as 8.0. I only need 4 bits to represent 8.0, so I was trying to find a way to convert the float64…
MUAS
  • 519
  • 1
  • 7
  • 20
4
votes
1 answer

Error in Quantization of Image using NQUANT

I recently acquired the NuGet Package Nquant. I plan to use this to reduce the file size of the bitmap and save it into PNG. But I get this error: The image you are attempting to quantize does not contain a 32 bit ARGB palette. This image has a bit…
TerribleDog
  • 1,237
  • 1
  • 8
  • 31
4
votes
2 answers

How to include alpha channel to calculate color difference with Spatial CIELAB (S-CIELAB)?

Of course I know how to convert RGB to LAB and LAB to RGB, color difference calculation is mentioned at https://en.wikipedia.org/wiki/Color_difference. However, it does not mention anything about opacity. I would like to know how to convert RGBA to…
Miller Cy Chan
  • 897
  • 9
  • 19
4
votes
1 answer

Input image of a fully quantized tensorflow lite model

I've trained a simple CNN model on Cifar-10 in tensorflow with fake quantization (https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/quantize). I then generated a .tflite file using toco. Now I want to use a python interpreter…
DrinkJava
  • 181
  • 1
  • 10
4
votes
1 answer

What are the differences between tf.fake_quant_with_min_max_args and tf.fake_quant_with_min_max_vars

I would love to understand the differences between the tensorflow functions tf.fake_quant_with_min_max_args tf.fake_quant_with_min_max_vars As in their API they have almost the same description. I usually quantize manually the required nodes…
Fernando García Redondo
  • 307
  • 1
  • 7
  • 12
4
votes
1 answer

How to compute Quantization Error for clustering?

I would like to measure the quality of clustering using Quantization Error but can't find any clear info regarding how to compute this metric. The few documents/ articles I've found are: "Estimating the number of clusters in a numerical data set…
solub
  • 1,291
  • 17
  • 40
4
votes
1 answer

Tensorflow build quantization tool - bazel build error

I am trying to compile the quantization script as described in Pete Warden's blog. However I get the following error message after running the following bazel build: bazel build tensorflow/contrib/quantization/tools:quantize_graph ERROR: no such…
Moe
  • 185
  • 1
  • 8
4
votes
1 answer

quantization (Reduction of colors of image)

I am trying to quantize an image into 10 colors in C# and I have a problem in draw the quantized image, I have made the mapping table and it is correct, I have made a copy of the original image and I am changing the color of pixels based on the…
Sarmad Omar
  • 47
  • 7
4
votes
0 answers

reducebit image from 24 bit rgb to 8 bit rgb but result image not beautiful as same as from tinypng.com

i use imagetruecolortopallete to reducebit image from 24 bit rgb to 8 bit rgb but result image not beautiful as same as from tinypng.com how can i do ? original image 24 bit rgb image 8 bit rgb convert by my code image 8 bit rgb convert by…
peat ribersal
  • 93
  • 1
  • 7
4
votes
1 answer

Converting 24 to 16 bit audio using SSE/simd instructions

I wonder if there is any fast method to do a 24 bit to 16 bit quantization on an array of audio samples (using intrinsics or asm). Source format is signed 24 le. Update : Managed to get the conversion done like described : static void __cdecl…
ohrfritz
  • 41
  • 2
4
votes
2 answers

Color quantizing giving very bad results?

I'm trying to get my application to export animated gifs, however, once the colors exceed 256, the quantization algorithm kicks in and things get, wrong. Here's the file I have being converted, and what the algorithm converts it to: The…
yesbutmaybeno
  • 1,078
  • 13
  • 31
4
votes
1 answer

OpenCV quantization sample code not running

I'm running the quantization sample code found in the OpenCV documentation, and it's throwing Traceback (most recent call last): File "QuantizeTest.py", line 13, in…
NoTrueScotsman
  • 305
  • 1
  • 3
  • 10
4
votes
1 answer

Why are JPEG quantization matrices asymmetric?

Why are the basis blocks corresponding to reflected waves in the quantisation matrix given seemingly random priorities in the standard JPEG quantisation matrices. Also, why aren't the priorities monotonic with respect to frequency? I haven't been…
0x7fff4d36c4d0
  • 43
  • 3
4
votes
1 answer

What is vector quantization using k-means?

First of all, can someone explain what vector quantization is, its purpose, and what it does? Secondly, an explanation of how k-means is used to do this would be appreciated as well. For the record, I don't know if this will make a difference in…
Shaayaan Sayed
  • 231
  • 2
  • 5
  • 13
1 2 3
…
29 30