Stack Overflow
Stack Overflow

Questions tagged [quantization]

Use this tag for questions related to quantization of any kind, such as vector quantization.

Quantization, in mathematics and digital signal processing, is the process of mapping a large set of input values to a (countable) smaller set.

For more, please read the Wikipedia article for more.

444 questions
0
votes
0 answers

How to use the quantized Tensorflow MobileNet v1 floating point scaling values

There are quantized MobileNet v1 models available at https://github.com/tensorflow/models/blob/master/research/slim/nets/mobilenet_v1.md I see floating point scaling values associated with the weights and biases in the model, but it isn't evident…
Jay Norwood
  • 146
  • 3
0
votes
1 answer

What is the "n" parameter in the JPEG spec's DQT segment?

I'm in the process of writing a JPEG file decoder to learn about the workings of JPEG files. In working through ITU-T81, which specifies JPEG, I ran into the following regarding the DQT segment for quantization tables: In many of JPEG's segments,…
user8370684
0
votes
2 answers

Range in DCT coefficents MATLAB

I am trying to find the range over which the coefficients of an 8 x 8 block varies for various block of an image. Do I need to generate a histogram for the values of is there a function in matlab?
qwop
  • 97
  • 5
0
votes
2 answers

JH Labs Quantize Usage to reduce image color depth

I am trying to use the QuantizeFilter in http://www.jhlabs.com/ip/filters/index.html to reduce the color depth of a screen shot. Here is my very very simple code: Robot robo = new Robot(); BufferedImage notQuantized =…
Nikola Yovchev
  • 9,498
  • 4
  • 46
  • 72
0
votes
0 answers

Quantize normalized vector for network communication

I am wondering if is possible to send Vector3(float x, float y, float z) in some kind of shorter data type, because right now I am sending three floats (3*4 bytes). I am talking about normalized vector, which can hold only range from -1f to +1f for…
MrIncognito
  • 153
  • 12
0
votes
1 answer

Quantize the variable with slicing assignment in Tensorflow

I want to do quantization for only some area of variables. More specifically, I would like to perform only on any channel in the convolution layer. This is binarization code: G = tf.get_default_graph() def binarize(x): with…
Wonseok Choi
  • 53
  • 1
  • 7
0
votes
0 answers

Tensorflow Quantization: Speed of Op "QuantizationConv2D" and "Conv2D"

Basically, I am testing the difference speed of Conv2D and QuantizedConv2D. I found that if I use two constant nodes as input and filter, the QuantizedConv2D is faster than Conv2D; but if I use a placeholder node as input, and a constant node as a…
Vic
  • 1
  • 2
0
votes
2 answers

Use tensor's value to modify another tensor at run-time

In TensorFlow I'd like to apply one of the "fake" quantization functions to one of my tensors. Concretely I'm interested in using this function: fake_quant_with_min_max_args( inputs, min=-6, max=6, num_bits=8, …
karl71
  • 1,082
  • 2
  • 18
  • 29
0
votes
2 answers

how to combine variables.data and saved_model.pb in tensorflow

I am new to tensorflow and keras. I trained a CNN for sentence classification using keras and exported the model using following code K.set_learning_phase(0) config = model.get_config() weights = model.get_weights() new_model =…
manoveg
  • 423
  • 1
  • 3
  • 13
0
votes
1 answer

TensorFlow: Quantization Error "Analysis of target '//tensorflow/tools/graph_transforms:transform_graph' failed; build aborted."

I am working to quantize my existing inception model graph in an attempt to reduce its size from ~89mb so something around 30mb as claimed according to the google tutorial here. The issue I am having is when I try to copy the following code snippet…
Veejay
  • 397
  • 3
  • 10
  • 17
0
votes
2 answers

C# PNG Image saving with selected filter

I couldn't help myself so once again I'm asking you for help. This time I will show the problem better than last time, I hope. I'm writing a program to check if Quantization have any influence on image sizes. To do that I need to have implemented :…
Łukasz Kochanowski
  • 82
  • 1
  • 12
0
votes
1 answer

Tensorflow tanh with quantized values

I am experimenting with the quantization of a neural network in Tensorflow 1.1. According to the documentation, the tanh operation supports floating point inputs as well as fixed point inputs of type qint32. However, I can't get this to work: import…
rerx
  • 1,133
  • 8
  • 19
0
votes
1 answer

Does data get lost during the MP3 quantization process

During the MP3 encoding process, before the data is quantized, it passes through a windowing and MDCT function. I'm wondering, does the quantization of that data mean some of it gets lost? MP3 Encoding
popopret
  • 83
  • 2
  • 9
0
votes
0 answers

Quantized Neural Network Support is gone in TensorFlow 1.0v?

I'm trying to implement quantization in neural network with tensorflow r1.0 and is not working i.e. Bazel BUILD files doesn't find targets and some other files. I realize that in the version r0.9 the quantization folder in "contrib" is more…
Mauric
  • 81
  • 6
0
votes
1 answer

find gradient using sobel mask operator in matlab

I am using following code to mask image using sobel mask . for i=1:size(C,1)-2 for j=1:size(C,2)-2 %Sobel mask for x-direction: Gx=((2*C(i+2,j+1)+C(i+2,j)+C(i+2,j+2))-(2*C(i,j+1)+C(i,j)+C(i,j+2))); %Sobel mask for…
Diwakar Kumar
  • 3
  • 4
1 2 3
…
29 30