Stack Overflow
Stack Overflow

Questions tagged [decimal]

Decimal is the name for our common base-ten numeral system. It may also refer to non-integer values expressed with a decimal point.

Decimal is the base-10 positional numeral system. It is the ordinary numeral system we use; it can be used to represent both integers and (using a decimal point) non-integer values.

Related tags

7532 questions
3
votes
2 answers

Value result is unintentionally rounding up?

So I this code, basically applying simple mathematical arithmetic to variables. rrdistance, qsdistance, heartrate give desirable values, while pamp and qamp don't. I think it rounds up? Supposedly, at i = 1, the values of trial[1] is 120, Pycoor[1]…
cookie23
  • 61
  • 1
  • 6
3
votes
6 answers

How to compare one character from a string with another string

Possible Duplicate: How do I compare strings in Java? I'm fairly new to Java and to practice I'm trying to create a hexadecimal to decimal number converter since I've successfully managed to make a binary to decimal converter. The problem I'm…
moka
  • 65
  • 1
  • 1
  • 9
3
votes
7 answers

Setting specific bits in a number

var d = 7; in binary: 7 = (111) What I want to do is to set second place from right to 1 or 0 at disposal, and return the decimal value. For example,if I want to make the second 1 to 0,then after process should return a 5, because 5=(101). How to…
omg
  • 136,412
  • 142
  • 288
  • 348
3
votes
1 answer

NHibernate Profiler is lying to me?

Im using Fluent NHibernate as my ORM and NH Profiler is throwing me this sql query when I execute it INSERT INTO [Location] (Name, Lat, Lon) VALUES ('my address' /* @p0 */, -58.37538459999996 /* @p1 */, …
Javier Acrich
  • 65
  • 4
3
votes
3 answers

How to get format numbers with decimals (XCode)

My objective is to create a customer calculator application for iPhone and I am using Xcode to write my application. My problem, that I cannot find a solution for, is how to format a number that uses decimals (with extra zeros) without switching…
Billy Fleming
  • 65
  • 1
  • 1
  • 6
3
votes
2 answers

Masked TextBox with decimal numbers

In my window application I need masked textbox which accept real decmal numbers. eg. 1) 1.56 2) 22.34 3) 123.34 4) 12312.34 This all value should be valid. Can anyone tell me how can I do this? And ya if anyone have better solution for…
Sagar Upadhyay
  • 819
  • 2
  • 11
  • 31
3
votes
3 answers

why R discard decimal when as.numeric is applied?

Hi to all the community, I have the following DB: ID Distance M1_PRM 54,56 M1_PRM 4147,69 M1_PRM 1723,34 I use the following script to replace "," in "." in Distance as R doesn't like "," (and it…
stefano
  • 601
  • 1
  • 8
  • 14
3
votes
3 answers

Having problems with my JavaScript Equation (Part of Binary to Decimal Converter)

So this is an outrage, a simple equation results in a random number that I can make little sense of. Ok, so I am trying to make an app that converts from Binary to Base Ten. I know one way of doing this is to do this. Say, 11001 to base ten. Take…
austinstout
  • 1,180
  • 1
  • 11
  • 14
3
votes
2 answers

How to represent floating point in binary. IEEE

Similar to decimal binary numbers can have represent floats too. Now I read that it can have floats of the sort 0.5:0.1 , 0.25:0.01 , 0.125:0.001 ... and so on. But then, for example, how is the 0.1(in decimal) represented in binary? Also, given a…
user1868357
3
votes
2 answers

Trying to pad decimal number with leading spaces

for (int iCount = 0; iCount < oForm.LineItems.Count; iCount++) { // cartDetails is a stringbuilder here. cartDetails.Append(String.Format("{0:0}", oForm.LineItems[iCount].Quantity)); cartDetails.Append(String.Format("{0:0.00}",…
NewCoder
  • 99
  • 4
  • 13
3
votes
2 answers

Html.EditorFor for a decimal field with commas

Hi I have a decimal field defined for price as follows: [Required(ErrorMessage = "Asking Price/Rent is required.")] [Display(Name = "Asking Price/Rent*")] [DisplayFormat(DataFormatString = "{0:N0}", ApplyFormatInEditMode = true)] public decimal…
Tripping
  • 919
  • 4
  • 18
  • 36
3
votes
1 answer

Shortest identical decimal representation

Possible Duplicate: drop trailing zeros from decimal How do I get the shortest representation of a Decimal that compares equal? For example: Decimal('89.00') -> '89' Decimal('123.010') -> '123.01' Decimal('0.0') -> '0' Currently, I have my own…
gerrit
  • 24,025
  • 17
  • 97
  • 170
3
votes
0 answers

Why does this SQL Server 2005 decimal division give results with different number of decimals?

Possible Duplicate: T-SQL Decimal Division Accuracy In SQL Server 2005, if I do the following query : select cast(1 as decimal(38,18))/cast(150 as decimal(38,18)) it returns 0.006666 (6 decimals) however, if I do : select cast(1 as…
user1185417
  • 99
  • 6
3
votes
4 answers

Decimal Conversion error

I am writing a program that will convert octal numbers to decimals. It compiles right and everything but there is something majorily wrong with my conversion code. It seems perfectly logic to me, however somehow when I run the program the…
user1714873
  • 135
  • 3
  • 3
  • 10
3
votes
4 answers

Formatting 2 decimal places in java

here's my example: double num = 0; num = 4/3; System.out.println(num); And my output is 1.0 instead of 1.3 Any suggestions?
Jcorretjer
  • 467
  • 2
  • 6
  • 24
1 2 3
…
99 100