Stack Overflow
Stack Overflow

Questions tagged [quantize]

11 questions
3
votes
1 answer

How to use`kmeans` parameter in Pillow Image.quantize?

Doc page for PIL.Image.quantize gives: Image.quantize(colors=256, method=None, kmeans=0, palette=None, dither=1) colors – The desired number of colors, <= 256 kmeans –…
P i
  • 29,020
  • 36
  • 159
  • 267
1
vote
0 answers

Error regarding input node in vitis-ai-quantizer

Im trying to quantize my model using vitis-ai tensoflow in the vitis ai docker image.But an error occurs sating my input node is wrong.I checked my input node via displaying using netron,but it is the only input node.why does this…
Kevini Perera
  • 11
  • 2
1
vote
0 answers

How does quantize work for Decimal object? InvalidOperation: []

So I'm having trouble understanding quantize for Decimal class. I've read the document and saw similar questions for my problem but for some reason, the solutions that were given did not work for me and I think I'm grossly misunderstanding what…
swordlordswamplord
  • 380
  • 4
  • 15
1
vote
0 answers

Model size reduction problem after quantization

I'm doing my project with tensorflow2 and tf-trt(tensorRT) module for deep learning accleration. I used tf-trt to quantize pretrained deep neural network(FP32->FP16), and its latency reduction is amazing. as far as i know, when quantize from…
GodHJ
  • 13
  • 3
0
votes
0 answers

Modify the PyTorch code to call GEMM of INT8

I would like to know how to use cublasGemmEx to infer a .pth model trained by pytorch with int8 quantization. I tried torch.quantization.quantize_dynamic and it seems that it doesn't work on the CUDA. I also tried to convert model to onnx, but it…
火星大王
  • 11
  • 2
0
votes
0 answers

is it possible quantized logsoftmax in pytorch

I try to quantize nn.LogSoftmax in my model. But ı don't find right configs. Any idea ? it's dynamic quantize, ı can try static quantize if it possible. qconfig_spec = { nn.Embedding: float_qparams_weight_only_qconfig, nn.Linear:…
Umut Ucay
  • 5
  • 3
0
votes
0 answers

How can I assign multiple values which are parameter to a function to single variable given the condition that the function is in another file in Py

I have a file num.py in which I have defined a variable as import decimal fivedecimal = Decimal(10) ** -5 This particular variable is being used in other python files as: import decimal from num import fivedecimal b=…
shubham_lv1
  • 1
  • 1
0
votes
0 answers

How to quantize a Sequence classification network in Matlab using calibrate

I used this network as a basic for my own Sequence classification network: https://de.mathworks.com/help/deeplearning/ug/sequence-classification-using-1-d-convolutions.html I used dlquantizer to quantize my network quantObj = dlquantizer(net) Now i…
hajo
  • 314
  • 1
  • 4
  • 12
0
votes
1 answer

Quantization of torch tensors to reduce storage size

I have a number (N) of vectors of size 192 x 1 currently stored as torch tensors. Each element in the tensor are float numbers. These Nvectors are used to be compared against a reference vector by its' similarity. Now, N can be very large and hence…
Oscar Johansson
  • 25
  • 7
0
votes
1 answer

Quantize Float Array to Byte Array in Python

I have a float array like this [0.1,0.4,1.5,2.22,3] where max value is always 3.0 I want to Quantize that to a byte array from 0 to 255 for each value I want to Quantize it like (byte)((value / 3.0) * 255.0) Is there a way to do that in numpy very…
Rob The Quant
  • 347
  • 3
  • 15
0
votes
1 answer

How to reduce number of colors to 1024 with PIL python

I am trying to reduce number of colors to 1024, but got error ValueError: bad number of colors img = img.convert('P', palette=Image.ADAPTIVE, colors=1024) or img = img.quantize(256, 0) What is wrong?
Romkaq
  • 1
  • 1