Stack Overflow
Stack Overflow

Questions tagged [rounding]

Rounding a numerical value means replacing it by another value that is approximately equal but has a shorter, simpler, or more explicit representation.

Rounding a numerical value means replacing it by another value that is approximately equal but has a shorter, simpler, or more explicit representation; for example, replacing £23.4476 with £23.45, or the fraction 312/937 with 1/3, or the expression √2 with 1.414.

5484 questions
3
votes
1 answer

Float Precision and Removing Rounding Errors in Scheme

I have a procedure that returns a float to three decimal places. >(gpa ’(A A+ B+ B)) 3.665 Is there any way to round this to 3.67 in Scheme? I'm using SCM version 5e7 with Slib 3b3, the additional Simply Scheme libraries (simply.scm, functions.scm,…
usernvk
  • 77
  • 1
  • 8
3
votes
2 answers

lua - How to get decimals to round down on the half

I'm using this function to round currently: function round(val, decimal) if (decimal) then return math.floor( (val * 10^decimal) + 0.5) / (10^decimal) else return math.floor(val+0.5) end end That function works perfectly except that…
Mark Kramer
  • 3,134
  • 7
  • 34
  • 52
3
votes
2 answers

Why do I need to multiply RGB values by 256 instead of 255 while converting from a 0.0-1.0 interval ?

I read about a rounding technique at this question. Converting color value from float 0..1 to byte 0..255 You have a float on the range from 0 to 1 and you want to map it to the byte integer range of 0-255. So you multiply the 0-1 number by 255f,…
Dan Webster
  • 1,185
  • 1
  • 12
  • 27
3
votes
1 answer

How can I use string.Format() to truncate instead of round?

I would like to apply string.Format() in such a way that it performs truncation instead of rounding. For example string.Format("##", 46.5); // output is 47, but I want 46 string.Format("##.#", 46.55); // output is 46.6, but I want 46.5
ChaseMedallion
  • 20,860
  • 17
  • 88
  • 152
3
votes
2 answers

Rounding to the Nearest Ending Digits

I have the following function that rounds a number to the nearest number ending with the digits of $nearest, and I was wondering if there is a more elegant way of doing the same. /** * Rounds the number to the nearest digit(s). * * @param int…
Alix Axel
  • 151,645
  • 95
  • 393
  • 500
3
votes
4 answers

In Ruby how do you round down a number to 2 significant digits

For example, If I have 24987654, I need it to return 24000000, is this possible?
Mhasan
  • 31
  • 1
3
votes
4 answers

floor() behaving strangely

I needed a function to round up a positive double to its closest integer. lurking aorund i found this very elegant way int x = floor(y + 0.5); I wrote a simple testing program: double a = 10.0; for (int i = 0; i < 10; i++) { cout << a << "\t" <<…
user2164587
  • 49
  • 5
3
votes
2 answers

python 3 round function approximation

I am facing a problem with the round function of python. When I round with 2 digits by executing round(0.715,2) the answer is 0.71, similarly when I execute round(0.615,2) the result is 0.61. But when I execute round(0.515,2) it gives 0.52,…
user2146889
  • 31
  • 2
3
votes
2 answers

How to round amounts correct?

I have a little problem with round() in php. I don't know, if I really make it correct. (it is an order system) $amount can be decimal 14,8 $price can be decimal 10,5 (in the database) I am using the following rounding at this moment The $price is…
DjangoSi
  • 395
  • 2
  • 4
  • 16
3
votes
3 answers

Floating point limits code not producing correct results

I am racking my brain trying to figure out why this code does not get the right result. I am looking for the hexadecimal representations of the floating point positive and negative overflow/underflow levels. The code is based off this site and a…
Babbu Maan
  • 149
  • 2
  • 3
  • 7
3
votes
2 answers

SQL rounding from 3 decimals to 2 decimals

I Have an SSRS 2008 Report whose dataset runs a SQL query that returns a calculated column. This dataset fills a table in the report. The calculated column is returning up to 4 decimals. I want to round to the nearest 2 decimals. i.e., 8660.125…
MikeTWebb
  • 9,149
  • 25
  • 93
  • 132
3
votes
4 answers

How to round up in progress programming

I am writing a report for an MRP program and it contains a field I calculate for the quantity to order. I need to round the number up if it is a decimal point. For example: 2.33 needs to be rounded up to 3 and so on. I have tried oder =…
Jim S.
  • 203
  • 8
  • 24
3
votes
2 answers

Python conditionally round up numbers

Today I have a question for you about rounding up numbers using conditions in Python. I am doing a sales website and my client wants to round up the prices depending of the result converting them from USD to Colombian Pesos. For example: 200 USD to…
Cristian Rojas
  • 2,746
  • 7
  • 33
  • 42
3
votes
3 answers

LINQ to SQL Math.Round Problem

I'm having an issue with a query I wrote where for some reason the variable that I'm using to store a decimal value in returns 6 values after the decimal place (they're mostly 0). I have tried the following (and different combinations using…
Overhed
  • 1,289
  • 1
  • 13
  • 41
3
votes
3 answers

Add round corners to a jpeg file

I am trying to add round corners to a jpeg file, but the problem is that after adding round corners, I am getting a black background color. Somehow I am not able to change it to any other color (white, transparent, red). It just simply shows black…
Aditya
  • 395
  • 6
  • 18
1 2 3
…
99 100