Stack Overflow
Stack Overflow

Questions tagged [quantization]

Use this tag for questions related to quantization of any kind, such as vector quantization.

Quantization, in mathematics and digital signal processing, is the process of mapping a large set of input values to a (countable) smaller set.

For more, please read the Wikipedia article for more.

444 questions
0
votes
0 answers

What is the condition to stop splitting data in Non-Uniform Quantizer?

I'm trying to implement a non-uniform quantizer with N-level of quantization. I have already done some work and it works, the problem is it goes into infinite loop when N(the number of levels) exceeds "4". If anyone can point out any hints to know…
Hossam Houssien
  • 359
  • 4
  • 14
0
votes
4 answers

Reducing sample bit-depth by truncating

I have to reduce the bit-depth of a digital audio signal from 24 to 16 bit. Taking only the 16 most significant bits (i.e. truncating) of each sample is equivalent to doing a proportional calculation (out = in * 0xFFFF / 0xFFFFFF)?
lornova
  • 6,667
  • 9
  • 47
  • 74
0
votes
1 answer

Why is quantized graph inference takes much more time than using the original graph?

I followed this tutorial in order to quantize my graph into 8 bit.I can't share the exact graph here but i can say it's a simple convolutional neural network. When i run the benchmark tool over the original and quantized networks it's clear that the…
yossiB
  • 89
  • 11
0
votes
2 answers

for loop not behaving correctly in MATLAB

I'm working on a program where I have to quantize a sine wave in MATLAB with 16 quantization levels I have developed a for loop that should quantize the values properly, and it does for the positive values of the sine wave, but then displays all…
Blake
  • 250
  • 1
  • 8
  • 18
0
votes
0 answers

quantize/arrange/sequence numbers into a specific format

I'm trying to arrange multiple PCM audio data into a specific sequence. the fact that it's audio data is just for context, the problem itself has nothing to do with audio/DSP. my input is a varying set of files with varying lengths, and I'm trying…
spinalwrap
  • 683
  • 1
  • 4
  • 17
0
votes
1 answer

Entropy decoder in vlc

Where is the place with entropy decoding in VLC player source code (GitHub link)? I found a place with parsing NALs and frame-slices in VLC decoder, but I can't find any mention of entropy decoder or quantized coefficients.
ULTRANj
  • 23
  • 6
0
votes
1 answer

How is signal frequency stored during pulse code modulation?

I am new to the concepts of Sampling and Quantization, and despite reading many articles, I am not clear on how PCM works if I record a simple sound. If I record the note A3 (220hz) at 44.1 KHz sampling rate, 16 bit quantization and mono channel for…
dev
  • 11,071
  • 22
  • 74
  • 122
0
votes
1 answer

Image dequantization in Java giving ramdom points in red , green or blue on Image

I am very new to Java and Image Processing. I am trying to take DCT(Discrete cosine transform) of 8*8 blocks and then do quantization and later get the original image by dequantizing followed by IDCT. But my final decoded image has green,red and…
user1662871
  • 19
  • 2
0
votes
1 answer

Quantization of a measure of error in SOM

I want to count quantization error (qe) in my SOM model using R. This is the code I've tried: data(yeast)## take only complete x <- yeast[[3]][apply(yeast[[3]], 1, function(x) sum(is.na(x))) == 0,] yeast.som <- som(x, somgrid(5, 8,…
Hanima
  • 25
  • 7
0
votes
2 answers

Quantizing a audiorecorder sample in MATLAB

I have a recorded a sound wave (16-bit single channel) using audiorecorder and I need to quantize it into 15 bits (removing LSB) without using toolboxes other than MATLAB built-in ones. Can anyone help on this? fs = 8000; tmax = 2; nbits = 16; nchan…
Bababarghi
  • 571
  • 1
  • 4
  • 23
0
votes
0 answers

How do I write encoded information (JPEG/JFIF) when i used 16x16 quantization table block

I'm trying to develop a JPEG encoder by using 16x16 quantization table block and DCT block instead of 8x8. I faced some issues while I'm writing the JFIF header of this new image. 0xFFDB is the JFIF marker of the quantization table. there is 2 …
Bilal Maskaleh
  • 11
  • 3
0
votes
1 answer

In case of Vector quantization, is the reconstructed intensity at the decoder taken into consideration?

Suppose we have formed the codebook using the RGB training images. This codebook is now present at the encoder and the decoder. Now we have one RGB test image (which is not contained in the training images) which we want to compress, transmit and…
Sara
  • 81
  • 1
  • 9
0
votes
1 answer

Lab color space quantization in Matlab

In Matlab, I've converted an RGB image to CIE Lab color space. Lab = applycform(rgbImage, makecform('srgb2lab')); L = Lab(:, :, 1); a = Lab(:, :, 2); b = Lab(:, :, 3); How to quantize and combine those 3 channels? ... For the comparison, this is…
Yahya
  • 198
  • 1
  • 5
  • 13
0
votes
1 answer

JPEG source-code and quantization mode change - C language

I'm assigned to do a project that consists in changing the quantization in the JPEG source-code, from the quantization tables to Lloyd-Max quantization. The problem is not knowing what to do (I know how to change the quantization), but where to find…
André Monteiro
  • 68
  • 6
0
votes
2 answers

Synchronise to beats per minute / Send quantised messages

I'm getting periodic signals (beats per minute) from a Transmitter and now want to call methods in a fraction of the period, e.g. send 1/1, 1/2, 1/4, 1/8, 1/16,.. notes. My solution for this is to create a thread, do a busy wait and then execute the…
user
  • 211
  • 2
  • 12
1 2 3
…
29 30