Stack Overflow
Stack Overflow

Questions tagged [quantization]

Use this tag for questions related to quantization of any kind, such as vector quantization.

Quantization, in mathematics and digital signal processing, is the process of mapping a large set of input values to a (countable) smaller set.

For more, please read the Wikipedia article for more.

444 questions
-1
votes
1 answer

Quantized Convolution Layer Operation in TensorflowLite

I want to understand the basic operation done in a convolution layer of a quantized model in TensorflowLite. As a baseline, I chose a pretrained Tensorflow model, EfficientNet-lite0-int8 and used a sample image to serve as input for model's…
Thanasis G
  • 15
  • 5
-1
votes
1 answer

quantization vector with numpy/pytorch

I have created quantized values as: {0.0, 0.5, 1.0} and would like to do quantization for a vector based on the set. For instance, given vector [0.1, 0.1, 0.9, 0.8, 0.6, 0.6] would be transferred into [0.0, 0.0, 1.0, 1.0, 0.5, 0.5]. Please guide me…
hhle88
  • 53
  • 1
  • 4
-1
votes
1 answer

Quantization of a complex signal in Python

I want to make a quantization of a complex signal in Python - need to have this signal as a set of discrete nonuniform levels. For example, a part of this signal can be represented as: x = [0, 0.015, 0.03, 0.04, 0.08, 0.1, 0.11, 0.12, 0.14, 0.17,…
Alina Belova
  • 13
  • 4
-1
votes
1 answer

Big O notation depend on matrix size only or data type(integer, Boolean) as well?

Does the big O notation depend on the dimension of the matrix only? or it will depend on data type(integer or boolean) as well. Suppose if I have two matrices of size mat=M x N and I have to multiply them. If the data type of matrix is integer the…
farzana hassan
  • 3
  • 1
-1
votes
1 answer

How to check if a point is inside a cluster of points

I am asking this question for a homework on vector quantization. I have implemented a rather classic algorithm to detect the center of a cluster of points. However in the input data there are several clusters (number of clusters and total inputs…
Ege Korkan
  • 134
  • 1
  • 10
-1
votes
1 answer

quantization of dct image for steganography

I hav a greyscale image. I did 8x8 blocks and computed each of their DCTs. I want to quantize the DCT coefficients and then replace their LSBs with my secret message bits. How exactly do I quantize the coefficients? Should I use the quantization…
shiladitya
  • 2,290
  • 1
  • 23
  • 36
-2
votes
0 answers

Inference of a neural translator model into a mobile application in offline mode

I need an app translator for an obscure language. I used openNMT with model quantization to reduce its size, but the final size is over 100 MB. This is much more than necessary. Are there any small models or ways to reduce the size of the model so…
ИНДУС Геймдев
  • 9
  • 4
-2
votes
1 answer

How to transform a decimal number into binary form but still keep distance property?

I am try to transform a decimal number into a binary form but still keep distance information. Such as 10-2=8 in euclidean space, but in binary case, hamming(1010-0010)=1, obviously the distance information lost a lot. Is there any possible way to…
Hx Yung
  • 1
  • 1
-3
votes
1 answer

is it possible to get code books from code words in the vector quantization?

i was working in matlab for vector quatization as we know in vector Quantization, if we provide set of code words as an input we get code vectors so what i did , i used LPG and Loyed algorithms to do that by :- training set=randn(2,100)…
Ali Al-Qamaji
  • 3
  • 4
1 2 3
…
29
30