0

I have a value being stored as an NSDecimalNumber and when I convert it to a double it's losing precision.

For the current piece of data I'm debugging against, the value is 0.2676655. When I send it a doubleValue message, I get 0.267665. It's truncating instead of rounding and this is wreaking havoc with some code that uses hashes to detect data changes for a syncing operation.

The NSDecimalNumber instance comes from a third-party framework so I can't just replace it with a primitive double. Ultimately it gets inserted into an NSMutableString so I'm after a string representation, however it needs to be passed through a format specifier of "%.6lf", basically I need six digits after the decimal so it looks like 0.267666.

How can I accomplish this without losing precision? If there's a good way to format the NSDecimalNumber without converting to a double that will work as well.

TJ Mott
  • 123
  • 4
  • 8

2 Answers2

4

The NSDecimalNumber instance comes from a third-party framework so I can't just replace it with a primitive double.

Yes you can. NSDecimalNumber is an immutable subclass of NSNumber, which is a little too helpful when it comes to conversion:

double myDub = [NSDecimalNumber decimalNumberWithDecimal:[[NSNumber numberWithDouble:((double)0.2676655)] doubleValue]];

Ultimately it gets inserted into an NSMutableString so I'm after a string representation, however it needs to be passed through a format specifier of "%.6lf", basically I need six digits after the decimal so it looks like 0.267666.

Double precision unfortunately does not round, but getting a string value that's off by one-millionth is not that big of a deal (I hope):

NSDecimalNumber *num = [NSDecimalNumber decimalNumberWithDecimal:[[NSNumber numberWithDouble:((double)0.2676655)] decimalValue]];
NSString *numString = [NSString stringWithFormat:@"%.6lf", [num doubleValue]];
NSLog(@"%@",numString);
CodaFi
  • 43,043
  • 8
  • 107
  • 153
2

I think that your are on a wrong path and somewhere lost in what to do.

First of all, keep in mind that in objective-c lond double is not supported, so you might better want to use something like %f instead of %lf. [to be found in the documentation library under "Type encodings" of the objective c runtime programming guide]

Then I would rather expect that the value is show as being truncated, as the doubleValue returns an approximate value but the range you are using is still within the correct range.

You should use a simple formatter instead of moving numbers around, like:

    // first line as an example for your real value
    NSDecimalNumber *value = [NSDecimalNumber decimalNumberWithString:@"0.2676655"];
    NSNumberFormatter *numFmt = [[NSNumberFormatter alloc] init];
    [numFmt setMaximumFractionDigits:6];
    [numFmt setMinimumFractionDigits:6];
    [numFmt setMinimumIntegerDigits:1];
    NSLog(@"Formatted number %@",[numFmt stringFromNumber:value]);

This has another benefit of using a locale aware formatter if desired. The result of the number formatter is the desired string.

iOS
  • 813
  • 5
  • 14