0

I treated 500 as BigDecimal in java. And i turned into {･UnscaledValue = 5 ･Scale = -2 ･Precision = 0} I really don't understand why precision is 0. I thought it might be 1. Can anyone explain?

Ole V.V.
  • 81,772
  • 15
  • 137
  • 161
Aki
  • 9
  • 2

1 Answers1

2

The precision for 500, constructed in the most straightforward way, is 3.

scala> new java.math.BigDecimal("500").precision()
res0: Int = 3

If you did something different, you'll have to show the code so we know what you did.

The meaning of precision from the documentation:

The precision is the number of digits in the unscaled value. The precision of a zero value is 1.

Chris Martin
  • 30,334
  • 10
  • 78
  • 137
  • If the scale is -2, and the unscaledValue is 5, then it is obviously stored as 5 * 10^2. Then, the precision should be 1, not 0, nor 3. – Rudy Velthuis Sep 29 '16 at 07:05
  • I too get 3. I also tried `new BigDecimal(500.0)`, the presicion is still 3 and the scale 0. I still haven’t figured out how the OP could possibly get a precision of 0. – Ole V.V. Sep 29 '16 at 08:37
  • @OleV.V.: Don't instantiate a BigDecimal from a floating point type (double). But if you do, say, `x = new BigDecimal("5e2")` or something like `x = new BigDecimal("5").movePointRight(2);`, or from a simple calucaltion, you can get an `unscaledValue` of `5` and a `scale` of `-2`. So the only problem is: why is the precision `0`. It should be `1`, by definition. – Rudy Velthuis Sep 29 '16 at 13:30
  • What I am trying to say is: *it does not matter how he got an `unscaledValue`of `5` and a `scale` of `-2`. It is a valid BigDecimal denoting the number 500, and its `precision()` should **not** be `3`, it should be `1`.* – Rudy Velthuis Sep 29 '16 at 13:33
  • Thanks for the suggestions, @RudyVelthuis. You clearly know a lot more than I about `BigDecimal`. However, for the precision of 3 I admit I still tend to believe my Java installation more then you (for the precision of 0 I’m starting to believe the OP made a mistake I never actually got a precision of 0 from his `BigDecimal` — now I’m being frank bordering to rudeness). – Ole V.V. Sep 29 '16 at 13:41
  • @OleV.V.: As I said, if you use the strightforward way, you'll get 3, because the unscaledValue is 500 and the scale is 0. But for a scale of -2 and an unscaledValue,of 5 it **should** be 1, not 0. That is by definition. So if you don't, something is pretty wrong. How did you obtain that value? Note that a precision of 0 is not allowed for any conceivable BigDecimal value. – Rudy Velthuis Sep 29 '16 at 15:23
  • @OleV.V.: FWIW, if I do `BigDecimal b = BigDecimal.valueOf(500).setScale(-2);`, then the scale is (obviously) -2, the unscaledValue is 5 and the precision is 1. That is on my Java installation, which currently uses JDK 1.7. So indeed, the OP must have done something wrong. And sorry I confused you with the OP. – Rudy Velthuis Sep 29 '16 at 15:42
  • @RudyVelthuis No problem, I didn’t discover. :-) PS My 1.8 installation agrees with your 1.7. – Ole V.V. Sep 29 '16 at 16:53