Stack Overflow
Stack Overflow

Questions tagged [underflow]

An arithmetic underflow is a condition in a computer program where the result of a calculation is a number of smaller absolute value than the computer can actually represent in memory.

In computing, buffer underrun or buffer underflow is a state occurring when a buffer used to communicate between two devices or processes is fed with data at a lower speed than the data is being read from it. This requires the program or device reading from the buffer to pause its processing while the buffer refills.

From Wikipedia

148 questions
0
votes
0 answers

C++: Unsigned integer underflow optimization

I'm having some problem with the following issue in c++: #include using namespace std; int main() { uint8_t l = 200, r = 2, c = 199; bool res1 = (l - c) < (r - c); bool res2 = uint8_t(l - c) < uint8_t(r - c); …
MaPo
  • 613
  • 4
  • 9
0
votes
2 answers

How to deal with underflow in R?

I want to compute this number : 0.34911191 ^ 1157 I am using R programming language and it returns me 0 (this is an underflow problem). How I can fix it? Thanks.
michele
  • 17
  • 4
0
votes
1 answer

How do you create a function to wrap numbers to a specified range?

I'm trying to create a function to underflow a number back up or overflow a number back down into a specified range mathematically. I think I was able to get this to work when the numbers are all positive (taking out Math.Abs (used to positivify…
Matt Arnold
  • 668
  • 2
  • 8
  • 21
0
votes
2 answers

different AWK flavors with diff smallest double value

there appears to be discrepancy regarding what each flavor of awk considers the smallest figure before it underflows to zero: mawk 1.3.4 | mawk 1.9.9.6 | macos built-in nawk all say 2^-1074 or 4.94 x 10^-324 but gawk 5.1.0 (w/o activating GMP or…
RARE Kpop Manifesto
  • 2,453
  • 3
  • 11
0
votes
1 answer

How to avoid underflow trying to zero out elements

I have a big numpy 2d array F which has complex numbers (np.complex64). It has a lot of very small numbers. For the sake of my calculation, I only need precision of ~1e-6 - 1e-9. Since this matrix is very large, I am trying to use a sparse matrix…
adch99
  • 340
  • 1
  • 7
0
votes
0 answers

Why am I getting this underflow error while slicing numpy ndarrays?

I'm trying to make a grid image with pyplot--specifically one that I can use as a "gallery" of a 2d array containing some smaller images of uniform size. I've mostly succeeded, but the blank image I used to put the grid over gets random noise on it.…
aimlesslegs
  • 41
  • 5
0
votes
1 answer

Casting Primitive Values

Value: 1,921,222, is too large to be stored as a short, so numeric overflow occurs and it becomes 20,678. Can anyone demonstrate the process of 1,921,222 becoming 20,678 ? how to “wraps around” to the next lowest value and counts up from there to…
user2017
  • 1
0
votes
1 answer

R - Incorrect calculations for small values

I consider the following function: -x*exp(-x) - (1+exp(-x))*log(1+exp(-x)) It is a continuous function, but while plotting it in R I get a weird discontinuous graph over x from -38 to -34 (which is not really so low as to reach Inf/-Inf in the…
RSP
  • 13
  • 3
0
votes
4 answers

Detecting signed integer multiplication overflow in C

I'm writing this code for more than 3 hours already.. I gave up about the overflow thing and tried to google and look it up on stackoverflow. I did not find any solution besides the one that I wrote in my code as you can see in lines 27-28 (where it…
NoobCoder
  • 513
  • 3
  • 18
0
votes
0 answers

R: Forward probability self-defined function leads to rounding errors

Here is my code. forw<-function(x,Pi,delta) {n <- length(x) m <- 2 nu<- matrix(0, nrow = n, ncol = m) #Initialisation nu[1, ] <- delta[,x[1]]/sum(delta[,x[1]]) ######## for (i in 2:n) { nu[i, ] <- (nu[i - 1, ] %*%…
Lola
  • 97
  • 4
  • 11
0
votes
0 answers

How to compute the variance of very small numerical values?

Today, I heard of the term catastrophic cancellation and would like to know what is the right way to compute the variance of very small numerical values in general? Since I use Python very often, I am mainly interested in whether Numpy's numpy.var()…
Gilfoyle
  • 3,282
  • 3
  • 47
  • 83
0
votes
0 answers

Linux filter in Ruby which maintains continuous data flow for a given time by its internal buffer?

For example when the incoming data stream on the STDIN can have pauses, gaps in it for a few minutes, meanwhile no data is sent, because actual data is still being prepeared, or isn't currently available for some reason. I would like to write a CLI…
Konstantin
  • 2,983
  • 3
  • 33
  • 55
0
votes
2 answers

Is there way of stopping uint underflow in C#?

Use int to test if below 0 . uint in c# wraps around to a huge number once it goes below 0, this is in all programming languages.
user366866
  • 55
  • 1
  • 4
0
votes
0 answers

Level of precision loss when an underflow occurs

I understand what an underflow is and when it could occur based on this. However, my question is that given that an underflow has occurred, what determines the amount of loss in precision? I'm computing an array in C++ and I do see some numerical…
tryingtosolve
  • 763
  • 5
  • 20
0
votes
0 answers

underflow in php 7 with int 32 is return 0

I used the following code to print the sum of two integer numbers with underflow: $z10 = (int)($z & 0xFFFFFFFF); $z11 = (int)($z5 & 0xFFFFFFFF); $z12 = (int)(((int)$z10 + (int)$z11) & 0xFFFFFFFF); $z10_type = gettype…
Moti
  • 897
  • 4
  • 12
  • 21
1 2 3
…
9 10