-1

I'm currently facing a problem when calculating the standard deviation in an SQL Server statement.

My problem: I have two tables.

T1:

Reg_Month
---------   
1
2
3
...

T2:

Product  Month   Consumption
-------------------------------
ProdA      1       200
ProdB      1        10
ProdA      1       300
ProdC      2       100
ProdA      2       200
...

Now what I want is something like this, for calculating my STDEV over a year:

Reg_Month   Product   Month   Sum
---------------------------------
   1        ProdA       1     500
   1        ProdB       1      10
   1        ProdC       1       0
   2        ProdA       2     200
   2        ProdB       2       0
   2        ProdC       2       0

So now I don't need this table to be set up, but I need to calculate the STDEV and AVG of the column "Sum" for each Product. The Problem is to include the NULLS.

This gives me the table I want for ProdA, but with NULLS whenever there was no consumption:

SELECT * 
FROM T1
FULL JOIN (SELECT Product, Month, SUM(Consumption) AS Sum, 
           FROM T2
           WHERE (Product = 'ProdA')
           GROUP BY Month, Product) sub ON (T1.Reg_Month = T2.Month)`

This gives me the STDEV:

SELECT Stdev(Sum) 
FROM 
    (SELECT * 
     FROM T1
     FULL JOIN (SELECT Product, Month, SUM(Consumption) AS Sum, 
                FROM T2
                WHERE (Product = 'ProdA')
                GROUP BY Month, Product) sub ON (T1.Reg_Month = T2.Month)) sub 
WHERE Product = 'ProdA'`

But the problem is, that it doesn't give me the correct STDEV for the entire year, if there is a month in which there was no consumption, because the NULLs (that appear due to the join) are ignored.

My approaches:

ISNULL():

SELECT Stdev(Sum) 
FROM 
    (SELECT * 
     FROM T1
     FULL JOIN (SELECT Product, Month, ISNULL(SUM(Consumption), 0) AS Sum, 
                FROM T2
                WHERE (Product = 'ProdA')
                GROUP BY Month, Product) sub ON (T1.Reg_Month = T2.Month)) sub 
WHERE Product = 'ProdA'`

doesn't work (maybe because the null is generated after the join?)

CASE:

SELECT Stdev(Sum) 
FROM 
    (SELECT * 
     FROM T1
     FULL JOIN (SELECT Product, Month, CASE WHEN SUM(Consumption) IS NULL THEN 0 ELSE Sum(Consumption) END AS Sum, 
                FROM T2
                WHERE (Product = 'ProdA')
                GROUP BY Month, Product) sub ON (T1.Reg_Month = T2.Month)) sub 
WHERE Product = 'ProdA'

doesn't work (probably the same reason)

I hope I was able to illustrate my example properly. Now, do you have any idea how get the right results for the STDEV?

Your input would be greatly appreciated!

Thanks a lot,

Clemens

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
user4985694
  • 13
  • 1
  • 4

1 Answers1

0

if you are only doing one product at a time

SELECT sum(isnull(T2month.MonthSum ,0))
     , Stdev(isnull(T2month.MonthSum ,0))
  FROM T1
  LEFT JOIN
       (select Month, sum(Consumption) as MonthSum 
          from T2
         where Product = 'ProdA' 
         group by Month) T2month 
    ON T1.Reg_Month = T2month.Month

for all products

SELECT Product.Name  
     , sum(isnull(T2month.MonthSum ,0))
     , Stdev(isnull(T2month.MonthSum ,0))
  FROM T1 
  cross apply  
       (values ('ProdA'), ('ProdB'), ('ProdC')) Product(Name)
  LEFT JOIN
       (select Product, Month, sum(Consumption) as MonthSum 
          from T2
         group by Product, Month) T2month 
    ON T1.Reg_Month = T2month.Month 
   AND T2month.Product = Product.Name 
 Group By Product.Name

Hey OP left join does NOT leave out periods
That is what a left join does

select lj.val, isnull(jj.val,0)
  from ( values (1), (2), (3), (4) ) lj(val) 
  left join ( values (1), (2), (4) ) jj(val) 
    on lj.val = jj.val 
 order by lj.val
paparazzo
  • 44,497
  • 23
  • 105
  • 176
  • Ok, I can see what you did. But the Stdev(..) doesn't work as a column, does it? because it needs to calculate the Stdev of the entire column not just a row. So if I leave this out I get the desired 0 instead of NULL. However, I can't seem to fit a Select Stdev() around it. I tried `SELECT T1.Reg_Month , sum(isnull(T2month.MonthSum ,0)) AS Needed FROM T1 LEFT JOIN (select Month, sum(sum) as MonthSum from T2 where Product = 'ProdA' group by Month) T2month ON T1.Reg_Month = T2month.Month GROUP T1.Reg_Month` – user4985694 Jun 17 '15 at 15:00
  • `SELECT Stdev(Needed) FROM (SELECT T1.Reg_Month , sum(isnull(T2month.MonthSum ,0)) AS Needed FROM T1 LEFT JOIN (select Month, sum(sum) as MonthSum from T2 where Product = 'ProdA' group by Month) T2month ON T1.Reg_Month = T2month.Month)` you mean like this? throws the error incorrect syntax near ) – user4985694 Jun 17 '15 at 15:11
  • What do yo get if you try exactly what I currently have posted? – paparazzo Jun 17 '15 at 15:12
  • a table with the columns Reg_Month, monthlySums and all NULLS for the STDEV() column – user4985694 Jun 17 '15 at 15:17
  • Then you are not pasting what I have because there is no Reg_Month in that select – paparazzo Jun 17 '15 at 15:19
  • Oh sorry, that was already a modifcation made by me, that yielded the result! Your second answer works just fine! Thanks a lot! – user4985694 Jun 17 '15 at 15:30
  • Just noticed that the Left join leaves out periods for the product. So if there is no demand in T2 the stdev for the product over the year would be wrong. Thats why I used Full join. Do you have another solution on how to deal with the NULL values? – user4985694 Jun 17 '15 at 20:14
  • No that left join does NOT leave out periods. That is what a left join does. – paparazzo Jun 17 '15 at 20:27