-1

I'm calculating product prices after discounts. An example would be product A would be 100 dollars with a 11.5% discount. When I store the 11.5% discount in a decimal as something like

myDecimal = 0.885M

the variable gets rounded to 2 places to become 0.89. Am I using the wrong datatype? I'm not sure what else to use because I was taught not to use floating point values because of the math errors.

What would be a solution?

David
  • 1,560
  • 2
  • 16
  • 32
  • 1
    There shouldn't be any implicit rounding going on. Can you show us more of your code? – Josh Mar 28 '14 at 01:30
  • `decimal myDecimal = 0.885M;` No repro. Show the code that is causing the problem – gunr2171 Mar 28 '14 at 01:30
  • what do you mean when you say store? are you talking about storing the value in a SQL database? if so there is a presicion and a scale element to decimal values in SQL. Take a look at this article: http://technet.microsoft.com/en-us/library/ms190476.aspx – kmacdonald Mar 28 '14 at 01:33

1 Answers1

2

You are just looking at the string representation of your decimal, which appears to be giving you 2 decimals, try using a different string format specification something like this.

static void Main(string[] args)
{
    decimal myDecimal = 0.885M;
    Console.WriteLine(myDecimal.ToString("N"));
    Console.WriteLine(myDecimal.ToString("N3"));
    Console.ReadLine();
}

The default Decimal.ToString() implementation according to MSDN states:

The ToString method formats a Decimal value in the default ("G", or general) format of the current culture.

Mark Hall
  • 53,938
  • 9
  • 94
  • 111
  • How did you know the OP was doing this? – gunr2171 Mar 28 '14 at 01:37
  • @gunr2171 when you look at the value, it uses the ToString method which defaults to the 'G` General Format which I guessed was 2 digits. I just made an example to show him that he still had his precision. It was just a guess and I could be wrong. – Mark Hall Mar 28 '14 at 01:39
  • Well, everything in your post is factually correct, but I don't see how it relates to the question. The OP has provided ZERO CODE, so who knows if the are accidently using the wrong ToString? As the question stands, this is just a shot in the dark. – gunr2171 Mar 28 '14 at 01:48
  • This is actually what is going on. The values are correct when I set them in a variable or put them in the database. On the form to edit a record what happens is the value is being rounded to 2 places. Now I just have to find a fix for that. Maybe the problem is with my viewmodel? – David Mar 28 '14 at 01:50
  • @gunr2171 The only way he could be looking at a rounded up value in mho is that he is converting to a string somewhere, without seeing his code you are right it is an educated guess. – Mark Hall Mar 28 '14 at 01:50
  • 1
    now that I know what is going on here is the problem I'm having:http://stackoverflow.com/questions/5428289/mvc3-decimal-truncated-to-2-decimal-places-on-edit – David Mar 28 '14 at 01:53