Stack Overflow
Stack Overflow

Questions tagged [rounding]

Rounding a numerical value means replacing it by another value that is approximately equal but has a shorter, simpler, or more explicit representation.

Rounding a numerical value means replacing it by another value that is approximately equal but has a shorter, simpler, or more explicit representation; for example, replacing £23.4476 with £23.45, or the fraction 312/937 with 1/3, or the expression √2 with 1.414.

5484 questions
2
votes
2 answers

IEEE 754 conformant sqrt() implementation for double type

I'm trying to implement the double __ieee754_sqrt(double x) function which uses hardware instruction to obtain the 1st approximation: double __ieee754_sqrt(double x) { double z; /* get reciprocal of the square root (6.75 bits accuracy) */ …
pmor
  • 5,392
  • 4
  • 17
  • 36
2
votes
2 answers

Round timestamp to microsecond - pandas

I can't seem to find this but I'm sure it's somewhere on SO. I'm hoping to round timestamps to a single microsecond. Other rounding frequencies are working but when I try to round the following to a single decimal place it's not working. df =…
jonboy
  • 415
  • 4
  • 14
  • 45
2
votes
4 answers

PHP calculate integer of division

I would like to calculate an integer part of division. The numerator and denominator (especially their precision) should not be altered because it might change from one calculation to other, also the denominator is as an example, its integer part as…
Jimmix
  • 5,644
  • 6
  • 44
  • 71
2
votes
1 answer

Function test with rounding to one decimal place showing error even with correct answers?

My function is to either input a Fahrenheit and output the conversion to Celsius or input Celsius and output the conversion to Fahrenheit. My test challenges that I round whatever result to the first decimal, which I've done correctly. However my…
James Ross
  • 67
  • 1
  • 7
2
votes
2 answers

Round series to specific values, that are not multiples?

I am trying to round the values of a series based on a list of values that are not multiples. The values I'm trying to round to are 15, 30, 60 and 120. You can see that I cannot use multiples of 15 because that would encompass 45, 75, 90..., which I…
Jason Minhas
  • 23
  • 5
2
votes
3 answers

float precision comparison by specific digit

How to test if two floats are identical until a specific digit? I tried, aa1 = 0.043403 aa2 = 0.043392 print(int(aa1*1000) == int(aa2*1000)) >> True I want to follow this way, but my data include NAN value, it cannot convert it to intro anyhow. I…
Dong-gyun Kim
  • 411
  • 5
  • 23
2
votes
5 answers

Rounding only numeric variables with ifelse

I have a very large dataframe (around 100 rows, 200 columns). A subset of my data looks like this: example <- data.frame("Station" = c("012", "013", "014"), "Value1" = c(145.23453, 1.022342, 0.4432), "Value2" = c(2.1221213, 4445.2231412,…
Sarah
  • 411
  • 4
  • 14
2
votes
2 answers

Round division of unsigned integers with no overflow

I'm looking for an overflow-safe method to perform round division of unsigned integers. I have this: uint roundDiv(uint n, uint d) { return (n + d / 2) / d; } But unfortunately, the expression n + d / 2 may overflow. I think that I will have to…
goodvibration
  • 5,980
  • 4
  • 28
  • 61
2
votes
1 answer

Is there an python function or extension that is is similar to Matlab's format short?

The command format short in Matlab makes all the print outs in the command window be "Short, fixed-decimal format with 4 digits after the decimal point." I know there is np.round, but I would like to have this functionality that Matlab offers in…
Oscar Andersson
  • 45
  • 5
2
votes
1 answer

Round function no longer works with mice output

Since updating R to 4.0.0 and re-installing mice it seems that the round() function no longer works on mice output but yields an error message. For example (using iris dataset): library(missForest) # for the prodNA function library(mice) #…
Lola92
  • 39
  • 5
2
votes
2 answers

Match list of floats to nearest integers without repeating

I have an algorithm I'm trying to implement and I'm struggling to find a good way of doing it. The goal is to take a list of floats, and make a one-to-one mapping to a list of integers such that no two floats gets mapped to the same integer, and the…
MarcTheSpark
  • 473
  • 5
  • 14
2
votes
1 answer

Is there a way to control the Python decimal quantize method when applied to zero?

Consider the following Python3 snippet: (Python 3.7.7 on mac os catalina) >>> from decimal import Decimal as d >>> zero = d('0') >>> one = d('1') >>> for q in range(10): ... one.quantize(d('10') ** -q) ...…
Thruston
  • 1,467
  • 17
  • 23
2
votes
0 answers

Reading numbers with System.Text.Utf8JsonReader

Since Json has only one type for Number, but C# has many types, like int, long, double, decimal, etc., when I use Utf8JsonReader, what is the correct way to retrieve number when after I call Read() and the TokenType is JsonTokenType.Number? There…
Yevgeniy P
  • 1,480
  • 1
  • 15
  • 23
2
votes
0 answers

RoundUp Yielding Unexpected result

I'm using the worksheetfunction.roundup function in my VBA code and it seems to be spitting out a bogus answer. Here is an example code that recreates the problem: Debug.Print WorksheetFunction.RoundUp(0.091, 2), WorksheetFunction.RoundUp(0.091, 2)…
ish_K
  • 21
  • 1
2
votes
2 answers

How to round a very small floating point number with trailing zeros?

Currently my code renders the number: x = 0.000092627861766 p x as something like a BigInt format such that: => 9.0e-05 Is there a method I can call on the variable to return a rounded floating point number (in either number or string format)…
Richard Jarram
  • 899
  • 11
  • 22
1 2 3
…
99 100