Anything related to converting a floating point number to and from other representations.
Questions tagged [floating-point-conversion]
290 questions
4
votes
1 answer
Why is JavaScript's number *display* for large numbers inaccurate?
So in JavaScript, 111111111111111111111 == 111111111111111110000. Just type any long number – at least about 17 digits – to see it in action ;-)
That is because JavaScript uses double-precision floating-point numbers, and certain very long numeric…
purefanatic
- 933
- 2
- 8
- 23
4
votes
1 answer
What are some common strategies different compilers use to deal with overflow in numeric conversions?
I understand that, in C++, when I convert a float/double into an int, whereby the floating-point number is beyond the range that the int can hold, the result is not defined as part of the C++ language. The result depends on the…
dayuloli
- 16,205
- 16
- 71
- 126
4
votes
3 answers
Midpoint 'rounding' when dealing with large numbers?
So I was trying to understand JavaScript's behavior when dealing with large numbers. Consider the following (tested in Firefox and Chrome):
console.log(9007199254740993) // 9007199254740992
console.log(9007199254740994) //…
p.s.w.g
- 146,324
- 30
- 291
- 331
4
votes
3 answers
How do I convert float to double while preserving decimal point precision?
I have a float-based storage of decimal by their nature numbers. The precision of float is fine for my needs. Now I want to perform some more precise calculations with these numbers using double.
An example:
float f = 0.1f;
double d = f; //d =…
Yahor
- 639
- 8
- 16
4
votes
2 answers
How to correctly pass a float from C# to C++ (dll)
I'm getting huge differences when I pass a float from C# to C++.
I'm passing a dynamic float wich changes over time.
With a debugger I get this:
c++ lonVel -0.036019072 float
c# lonVel -0.029392920 float
I did set my MSVC++2010…
JohnYouDontLike
- 71
- 1
- 1
- 8
4
votes
4 answers
Why bother using a float / double literal when not needed?
Why use a double or float literal when you need an integral value and an integer literal will be implicitly cast to a double/float anyway? And when a fractional value is needed, why bother adding the f (to make a floating point literal) where a…
Kevin
- 53,822
- 15
- 101
- 132
4
votes
2 answers
convert double to GLfloat in Haskell
I want to convert a Double to a GLfloat.
I want to use it for comparations.
xM <- newIORef 0.0
zM <- newIORef 0.0
mobs <- newIORef []
mapM_ (\x -> colision x xM) mobs
mobs is fulled with a method.
colision mob xC = do
xcama <- get xC
--zcama <-…
4
votes
1 answer
Bad float conversion on C# using VS2012
I am getting a very weird behavior here with C#:
float num = 144771463f;
// num is 144771456.0
I have also tried
float num = Convert.ToSingle(144771463f);
// num is still 144771456.0
Why does it happen?
MatanKri
- 263
- 1
- 3
- 13
4
votes
4 answers
Parse and convert denorm numbers?
In C++, we can store denorm numbers into variables without problems:
double x = std::numeric_limits::denorm_min();
Then, we can print this variable without…
Vincent
- 57,703
- 61
- 205
- 388
4
votes
2 answers
the Floating-point error
#include
int main()
{
int n;
while ( scanf( "%d", &n ) != EOF ) {
double sum = 0,k;
if( n > 5000000 || n<=0 ) //the judgment of the arrange
break;
for ( int i = 1; i <= n; i++ ) {
k…
Hong Wei
- 416
- 3
- 8
3
votes
2 answers
(float) casting does not work
I', trying this simple code. It shows the first 10 integers that can not be represented in float:
int main(){
int i, cont=0;
float f;
double di, df;
for(i=10000000, f=i; i
salteador
- 31
- 4
3
votes
3 answers
Comparing float and double
#include
int main(void){
float a = 1.1;
double b = 1.1;
if(a == b){
printf("if block");
}
else{
printf("else block");
}
return 0;
}
Prints: else block
#include
int main(void){
…
Chankey Pathak
- 21,187
- 12
- 85
- 133
3
votes
7 answers
Working With Floats and Integers
I've created an ATM like program which holds the money in a person's account. When the person takes out a withdrawal, it subtracts the withdrawal from the account along with a .50 surcharge. The problem I'm having is working with both integers and…
user1064913
- 335
- 2
- 8
- 19
3
votes
2 answers
square of a float number in C
I have written a code in C which works fine for int but when I try to do this with float it is showing error what can i do to make it correct.
#include
int main()
{
float a,y;
float square();
scanf("%f", &a);
y = square( a…
Sudhanshu Gupta
- 2,255
- 3
- 36
- 74
3
votes
3 answers
Bitwise Float To Int
I am trying to figure out the algorithm to this but all I get with google is doing it with casting. I need to know the details.
So if we have a float x and want to return its binary representation what do we need to do?
I know we need to return the…
Jace
- 33
- 1
- 4