Anything related to converting a floating point number to and from other representations.
Questions tagged [floating-point-conversion]
290 questions
-2
votes
2 answers
different result of int multiplication vs float multiplication
I have this code:
float number1 = 2f / 12;
System.out.println(number1);
int number2 = 6;
float x = number1 * number2;
System.out.println(x);
And as output I get:
0.16666667
1.0
Why? When I multiply number1 and number2, number2…
The Nightmare
- 701
- 5
- 16
- 36
-2
votes
1 answer
C++ float rounding (error?)
Why when I use multiplication
float a= 1.0500000f;
float b= a*100.0f;
why b is 104.99999 but not 105.0 ?
and when I
int f= (int)b;
f is 104
mrgloom
- 20,061
- 36
- 171
- 301
-2
votes
2 answers
Understanding float variable comparison in if()
Unable to find the reason for the following piece of code:
#include
int main()
{
float f = 0.1;
if (f == 0.1)
printf("True");
else
printf("False");
return 0;
}
The output is false.
#include
int…
Megharaj
- 1,589
- 2
- 20
- 32
-3
votes
1 answer
Converting HEX string into int
I need help on converting the varbinary data which I acquired from sql, into int by taking 4 bits at a time.
From the code above, I acquired the results as ('0x640761075D075A0.....'). My plan is to take every 4 bits & swap it (ex: 0764 0761 07D5 and…
Crn
- 45
- 1
- 8
-3
votes
2 answers
Why does my floating point value have an `E` character when I convert it to a string?
txtDebugLog.Invoke(new MethodInvoker(delegate()
{
fps.Frame();
ggg = fps.GetFPS();
txtDebugLog.Text = String.Format("{0}\r\n{1}", ggg, txtDebugLog.Text);
})
txtDebugLog is a TextBox.
Using a breakpoint i see on ggg in this example it's…
Harim Abdu
- 23
- 6