3

Using SQL Server 2012...

I have two columns:

Price [decimal(28,12)]
OustandingShares [decimal(38,3)] -- The 38 is overkill but alas, not my call.

When I do an ALTER TABLE I get a resulting computed column as a [decimal(38,6)]. I need the datatype to be [decimal(28,12)].

ALTER TABLE [xyz].MyTable
ADD Mv AS OustandingShares * Price

How can I effectively get 12 decimals of scale on this computed column? I've tried doing convert on the OutstandingShares to 12 decimal places as well as wrapping a convert around the OutstandingShares * Price. The only thing I get is a computed field at [decimal(28,12)] with six trailing zeros.

Thoughts?

Solomon Rutzky
  • 46,688
  • 9
  • 128
  • 171
jermny
  • 870
  • 2
  • 12
  • 23

2 Answers2

4

The Fix

This does what you want:

CONVERT(DECIMAL(28,12), (
                            CONVERT(DECIMAL(15, 3), [OustandingShares])
                          * CONVERT(DECIMAL(24, 12), [Price])
                         )
       )

Test with this:

SELECT CONVERT(DECIMAL(28,12),
                 (CONVERT(DECIMAL(24,12), 5304.987781883689)
               * CONVERT(DECIMAL(15,3), 3510.88)));

Result:

18625175.503659806036

The Reason

The computation is being truncated due to SQL Server's rules for how to handle Precision and Scale across various operations. These rules are detailed in the MSDN page for Precision, Scale, and Length. The details we are interested in for this case are:

  • Operation: e1 * e2
  • Result precision: p1 + p2 + 1
  • Result scale *: s1 + s2

Here the datatypes in play are:

  • DECIMAL(28, 12)
  • DECIMAL(38, 3)

This should result in:

  • Precision = (28 + 38 + 1) = 67
  • Scale = 15

But the max length of the DECIMAL type is 38. So what gives? We now need to notice that there was a footnote attached to the "Result scale" calculation, being:

* The result precision and scale have an absolute maximum of 38. When a result precision is greater than 38, the corresponding scale is reduced to prevent the integral part of a result from being truncated.

So it seems that in order to get the Precision back down to 38 it chopped off 9 decimal places.

And this is why my proposed fix works. I kept the "Scale" values the same as we don't want to truncate going in and expanding them serves no purpose as SQL Server will expand the Scale as appropriate. The key is in reducing the Precision so that the truncation would be non-existent or at least minimal.

With DECIMAL(15, 3) and DECIMAL(24, 12) we should get:

  • Precision = (15 + 24 + 1) = 40
  • Scale = 15

40 is over the limit so reduce by 2 to get down to 38, which means reduce the Scale by 2 leaving us with a true "Result Scale" of 13, which is 1 more than we need and will even be seeing.

Solomon Rutzky
  • 46,688
  • 9
  • 128
  • 171
  • Thanks for the answers. I'm worried about the accuracy of floats out to the 12th decimal place. Do you have an opinion on this? – jermny Nov 11 '14 at 14:28
  • @jermny : I have **a little** hesitation towards that as well, but so far this was the only means of extending the range of significant digits that I could find. I am still trying other ideas but figured I would share this first. But I am not sure how much concern there needs to be as FLOAT is typically accurate and theoretically imprecise but that should be on the far end of the scale I would think. – Solomon Rutzky Nov 11 '14 at 14:29
  • @jermny : I figured it out and posted the answer a bit ago, and have now updated with the full explanation. – Solomon Rutzky Nov 11 '14 at 15:26
  • 1
    This is an absolutely fantastic post. Thank you so much! – jermny Nov 11 '14 at 15:57
1

Use cast() or convert(). Something like:

ALTER TABLE [xyz].MyTable ADD Mv AS cast(OustandingShares * Price as decimal(12, 6)

or whatever type you want it to be.

EDIT:

Oh, I think I'm getting the idea. The problem is the calculation itself. In that case, do the conversion before the multiplication, so you don't have to depend on SQL Server's (arcane) rules for conforming decimal types.

ALTER TABLE [xyz].MyTable
    ADD Mv AS cast(OustandingShares as decimal(28, 12) * cast(Price as decimal(28, 12))

I believe what is happening in your case is that the maximum precision on the calculated result exceeds the allowed thresholds, so the scale is reduced accordingly. This is explained at the bottom of this page.

Gordon Linoff
  • 1,242,037
  • 58
  • 646
  • 786
  • I had mentioned that I tried that already and all I got was trailing zeros. Also, I want it to NOT be 12, 6 but rather 28, 12. – jermny Nov 11 '14 at 14:11
  • 1
    @jermny . . . You got trailing digits because you have 12 for the scale. "Precision" is the total number of digits. "Scale" is the number to the right of the decimal point. (http://msdn.microsoft.com/en-us/library/ms190476.aspx.) Perhaps you are not using the right type. – Gordon Linoff Nov 11 '14 at 14:12
  • Thank you for the information on terminology. My apologies. – jermny Nov 11 '14 at 14:17
  • Unfortunately this still didn't work: When I did ALTER TABLE [xyz].MyTable ADD Mv as cast((OutstandingShares * Price) as decimal(28, 12)) it still had the result of 3510.88 (shares) * 5304.987781883689 (price) = 18625175.503660000000 (trailing zeros). – jermny Nov 11 '14 at 14:27
  • You still have 28 for your precision and 12 for the scale. That is why it is giving you lots of zeros. Try putting in the numbers that I suggest, or just use 6 for the scale (which seems to be what you are asking for). – Gordon Linoff Nov 11 '14 at 14:28
  • 2
    I want 12 decimal places. I need to be able to represent the number 123,456,789,012.123456789012. – jermny Nov 11 '14 at 14:30