0

I'm running into an issue where the output for my dynamic pivot contains thousands of NULL values that I would like to replace with 0. I am not able to use ISNULL before the pivot is generated because there aren't any NULL values in the original table. I've searched everywhere and I havn't located anything that works for me. Here is my code.

DECLARE @col NVARCHAR(MAX);
DECLARE @sql NVARCHAR(MAX);

SET @col =  (SELECT STRING_AGG(CONVERT(NVARCHAR(MAX),CONCAT('[',[order_date],']')),',') FROM
            (
            SELECT DISTINCT TOP 10000 (CAST(CONVERT(datetimeoffset,[purchase-date],127) AS DATE)) AS order_date
            FROM dbo.Amazon_FBAFulfilledShipments_Report
            ORDER BY order_date ASC
            ) trial_table);
SELECT @col
PRINT @col
SET @sql = 
'SELECT ship_country,sku,' + @col + '
FROM
(
SELECT [ship-country] AS ship_country,sku,CAST([quantity-shipped] as INT) AS quantity_shipped,CAST(CONVERT(datetimeoffset,[purchase-date],127) AS DATE) AS order_date
FROM dbo.Amazon_FBAFulfilledShipments_Report
) as Source_table
PIVOT
(SUM(quantity_shipped) FOR order_date IN (' + @col +')
)AS pivot_table'

PRINT @sql
EXECUTE (@sql)
  • As per the question guide, please do not post images of code, data, error messages, etc. - copy or type the text into the question. Please reserve the use of images for diagrams or demonstrating rendering bugs, things that are impossible to describe accurately via text. – Dale K Jun 16 '22 at 23:40
  • Does this answer your question? https://stackoverflow.com/questions/72639331/use-sum-function-inside-a-pivot-table – Dale K Jun 16 '22 at 23:41
  • @DaleK I don't believe it does because I'm dealing with a dynamic pivot output. In the link you gave, the table has set column names, whereas mine does not – AceCharm101 Jun 16 '22 at 23:45
  • You can still build a similar query, just dynamically. Looks very doable, but an actual example of your end query, and min/max numbers of columns would help clarify. – Dale K Jun 16 '22 at 23:47
  • An example of my end query? As in a picture of my result? The min/max numbers of columns is 1000 (or roughly three years worth of days as column headers). Each item (sku) has an order quantity for different days, but obviously not all days. This is why there are NULL values – AceCharm101 Jun 16 '22 at 23:51
  • No pictures please :) but having an idea of the range of values of `@col` - but why not just change your your `@col` query to produce a bunch of case expressions? – Dale K Jun 16 '22 at 23:52
  • 1000 columns? Something makes me thing there is a design flaw there... – Dale K Jun 16 '22 at 23:53
  • @DaleK Hm. Well I'm not super familiar with SQL (just started job) so I haven't used CASE yet. If you're able, could you briefly explain how I would apply CASE to a dynamic pivot? – AceCharm101 Jun 16 '22 at 23:54
  • https://stackoverflow.com/q/72621391/61305 – Aaron Bertrand Jun 16 '22 at 23:55
  • @DaleK Oh, it's definitely a design flaw. My boss wants me to generate a report so that he can load the data into Excel and do some forecasting. His requirement is that each day be a column. That's why I have to do this. – AceCharm101 Jun 16 '22 at 23:56
  • @AceCharm101 the answer I linked to shows you how to use a case expression? But Aarons goes one further and shows a dynamic pivot. – Dale K Jun 16 '22 at 23:57
  • @DaleK Okay, I will look into Aaron's response and try to apply it to my issue. Thank you for your responses. – AceCharm101 Jun 17 '22 at 00:01
  • Just think about it this way: you need two different sets of columns to jam into the dynamic SQL: one for the pivot (`FOR col IN ([day1], [day2], etc.)`) and one for the output (`day1 = COALESCE(day1, 0), day2 = COALESCE(day2, 0), etc.`. [This may also be useful](https://www.mssqltips.com/sqlservertip/2783/script-to-create-dynamic-pivot-queries-in-sql-server/?utm_source=AaronBertrand). – Aaron Bertrand Jun 17 '22 at 00:01
  • @AaronBertrand Ahhh. I see what you're saying. I might reach out to you tomorrow after I've given it a good effort and have some progress to show. Is that okay with you? – AceCharm101 Jun 17 '22 at 00:01
  • Also `DISTINCT TOP 10000 ` why? Please make it make sense – Aaron Bertrand Jun 17 '22 at 00:06
  • @AaronBertrand lol. So in the original table, the [purchase-date] column has NVARCHAR values like yyyy-mm-ddTHH-mm-ss-z. Since I want to only have "day" column headers (such as 2022-06-10) and no repeats: this is why I have the DISTINCT. The TOP 10000 is honestly just because I want current data, and I havn't been able to use an ORDER BY ASC to get recent days. When I had DISTINCT TOP 1000 ........ ORDER BY...... ASC I only got from 2018 to 2020 – AceCharm101 Jun 17 '22 at 00:08
  • Ok, but you're ultimately grouping by just the date right? – Aaron Bertrand Jun 17 '22 at 00:10
  • @AaronBertrand I edited the above comment, but if you need further clarification let me know. – AceCharm101 Jun 17 '22 at 00:11

1 Answers1

2

The key is to think about it this way: you need two different sets of columns to jam into the dynamic SQL: one for the pivot (FOR col IN ([day1], [day2], ...)) and one for the output (day1 = COALESCE(day1, 0), day2 = COALESCE(day2, 0), ...). This previous question and this article may be useful.

Here's a working script (and here's an example db<>fiddle).

DECLARE @OutputCols nvarchar(max),
        @PivotCols  nvarchar(max),
        @sql        nvarchar(max),
        @StartingDate date = '20180101';

;WITH src(d) AS
(
  SELECT CONVERT(nvarchar(max), QUOTENAME(CONVERT(date, [purchase-date])), 120)
    FROM dbo.Amazon_FBAFulfilledShipments_Report
    WHERE [purchase-date] >= @StartingDate
    GROUP BY CONVERT(date, [purchase-date])
)
SELECT @PivotCols  = STRING_AGG(d, ','),
       @OutputCols = STRING_AGG(CONCAT(d, ' = COALESCE(', d, ', 0)'), ',')
FROM src;

SET @sql = N'SELECT ship_country,sku,' + @OutputCols + '
FROM
(
  SELECT [ship-country] AS ship_country,sku,
    CAST([quantity-shipped] as INT) AS quantity_shipped,
    CAST(CONVERT(datetimeoffset,[purchase-date],127) AS DATE) AS order_date
  FROM dbo.Amazon_FBAFulfilledShipments_Report
  WHERE [purchase-date] >= @StartingDate
) AS Source_table 
PIVOT
(
  SUM(quantity_shipped) FOR order_date IN (' + @PivotCols +')
) AS pivot_table'

PRINT @sql;
EXEC sys.sp_executesql @sql, N'@StartingDate date', @StartingDate;

Multiple things:

  • I got rid of the DISTINCT TOP 10000. If you want "current" data, just use a WHERE clause (which will produce more predictable results, like since Jan 2018, not since 10,000 distinct purchase date values ago), and use it in both queries (there's no sense limiting your PIVOT list to 10,000 dates if your dynamic query queries ALL OF TIME, though I'm not sure how you'd consume such a query anyway).
  • Use QUOTENAME(); it is much safer and less tedious than trying to use string concatenation to add your own [ and ]. Dynamic SQL probably isn't on the radar in this specific scenario, but it is not a good practice and sets a bad example.
  • Don't use EXECUTE(), always use sys.sp_executesql.
  • Dynamic SQL advice here.
Aaron Bertrand
  • 272,866
  • 37
  • 466
  • 490
  • @AaronBertrand This is phenomenal. Thank you so much for the query/ links, this is much closer to what my boss wants so you're directly helping my job security haha. I really appreciate it. – AceCharm101 Jun 17 '22 at 03:27