Stack Overflow
Stack Overflow

Questions tagged [decimal]

Decimal is the name for our common base-ten numeral system. It may also refer to non-integer values expressed with a decimal point.

Decimal is the base-10 positional numeral system. It is the ordinary numeral system we use; it can be used to represent both integers and (using a decimal point) non-integer values.

Related tags

7532 questions
2
votes
2 answers

Culture-Based String Formatting For Decimal

I've got a decimal value, which is stored in my database (SQL Server 2008) with a precision of 13, and a scale of 10. Examples: 10.6157894734 68.0750000000 96.8723684210 Basically, the numbers represent a "score" out of 100. Now, i want to…
RPM1984
  • 72,246
  • 58
  • 225
  • 350
2
votes
4 answers

How to round a double to two decimal places in Java?

I am currently try to retrieve a latitude and longitude value from a location. When i convert the location to integer values using the following code: LocationManager locMan; Location location; String towers; private static double…
devinefergal
  • 287
  • 1
  • 4
  • 10
2
votes
3 answers

Convert binary to hex to decimal

I have the binary number 1010 1011. I know that this is AB in hex and I know that A = 10 and B = 11 in decimal. But how do I get from 10 and 11 in decimal to the final number of 171? With hex I would do A B 0xAB = (10 * 16^1)…
Frank Vilea
  • 8,323
  • 20
  • 65
  • 86
2
votes
3 answers

What is that decimal can't do but double can and vice versa?

Can someone tell what a decimal variable cannot do but at the same time double can do? Also what is that double cant do but decimal can? I was having trouble with finding power of (sqroot 5) to more than 2000000 e.g. (3 + root(5) )raise to…
Dhananjay
  • 3,673
  • 2
  • 22
  • 20
2
votes
3 answers

Separate decimals of a value into a new variable (eg: get "0.6" from the value "3.6")

I want to create a new variable based on whatever decimal values a variable has. i.e: myValue = 3.6 function twoNewVars{...} newVar1 = 3 newVar2 = 0.6 Using jQuery or javascript. Would one split that by simply looking for the position of the…
RGBK
  • 2,048
  • 5
  • 35
  • 55
2
votes
3 answers

Convert from double to decimal

I am developing a weather application in C# using the Google weather XML file and I am having trouble using calculations in a class file. I am trying to convert farenheit to celcius with the folliowing method: public static class Helper { public…
Rhys Towey
  • 2,355
  • 1
  • 24
  • 35
2
votes
3 answers

Mysterious 2 arising in the output of my JavaScript decimal to binary converter?

I am trying to implement a fairly simple Decimal to Binary converter using the following recursive function: function dectobin(d) { if (0 >= d) { return 0; } else if (1 == d) { return 1; } else { …
user1251380
  • 63
  • 4
2
votes
2 answers

Working with big numbers in Objective-C?

I need to convert values like 1393443048683555715 to HEX. But, first of all, i cann't display it as decimal using NSLog(), for example. Ok, it works: NSLog(@"%qu", 1393443048683555706); But what about converting to HEX. What type i have to use to…
leaf
  • 87
  • 11
2
votes
1 answer

Increase precision on money field for huge table

I have a huge data table (200+ million records) which stores cash amounts for which we are using the Money datatype. I need to increase the precision of this field to 8 or so decimal points. Now I could simply go ALTER TABLE CashTable ALTER COLUMN…
Karl
  • 5,573
  • 8
  • 50
  • 73
2
votes
1 answer

Turning off automatic conversion to Decimal in SqlAlchemy with Oracle DB

How do I tell SqlAlchemy not to convert Oracle Number fields to Decimal when querying an Oracle database? I'm trying to replace a bunch of code containing hand-rolled sql-statements, with SqlAlchemy. A lot of the queries deal get 500 columns of…
AFoglia
  • 7,968
  • 3
  • 35
  • 51
2
votes
2 answers

convert binary to decimal and show in assembly

I have array of word,with 4 cells RESULT DW 4 DUP(0) it will contain Binary number,for example MOV RESULT, 0H MOV RESULT+2, 0H MOV RESULT+4, 35A4H MOV RESULT+6, E900H now result contain 0000000035A4E900H which means 900000000 in Decimal way. not…
Moein Hosseini
  • 4,309
  • 15
  • 68
  • 106
2
votes
1 answer

Decimal type in C# vs IEEE-754 standard

Is Decimal type in C# follow the same rules (formula,normalized/denormalized,implied 1,Exponent bias) of classic double representation (IEEE-754 standard) except the use of base 10 instead of base 2. What does implied the use of base 10 instead of…
Guillaume Paris
  • 10,303
  • 14
  • 70
  • 145
2
votes
2 answers

Javascript regular expression for comma delimited decimal

I need a regular expression that matches currency values similar to here: Regex for Money i.e. '1000', '1000,0' and '1000,00' but I need it to work using javascript: var RE = /^-{0,1}\d*\.{0,1}\d+$/; //validates decimal format such as 1000.50 …
FiveTools
  • 5,970
  • 15
  • 62
  • 84
2
votes
3 answers

Precision of decimal number in Java, C, C++ and others

Let's say we have this kind of loop (pseudocode) double d = 0.0 for i in 1..10 { d = d + 0.1 print(d) } In C with printf("%f", d) I get this: 0.100000 0.200000 0.300000 ... 1.000000 In C++ with cout << d I get this: 0.1 0.2 ... 1 In Java…
user219882
  • 15,274
  • 23
  • 93
  • 138
2
votes
1 answer

How to compare numbers with 2 decimal places in VB.NET

I'm trying to compare Office version numbers in vb.NET How can I determine if: 12.0.64 is greater or less than say 12.0.62 or 14.0.4762 It seems the fact that there is 2 decimals, vb doesn't like it.
Ben
  • 1,000
  • 2
  • 15
  • 36
1 2 3
…
99 100