3

I am currently running into some bad performance issues with a strange behaviour on SQL Server 2016 Enterprise.

I created a new schema in a database and then I created a view inside this schema.

Now, when I connect directly to the database, that contains this schema and the view and write a simple query like

SELECT * FROM SCHEMA.VIEW

it takes around 30 minutes (!) to complete. The same happens with a full qualified query like

SELECT * FROM DB_NAME.SCHEMA.VIEW

But now, if I first change the database to master or another user database and then run the query again across the databases, it completes within around 10 seconds (!). The database properties of both databases are the same, as well as the used drives for the database files and the log files.

Does anyone have an idea on what might cause this enormous performance problems?

I used the following code for the view:

CREATE VIEW [controlling].[UnitLoginHistory]
AS

    WITH    cte 
            ( 
                    Jahr, UnityId, Unit_UnityId, Analytical_Code, Unit_Code, Unit_Name, Active
            ,       Show_in_org_chart, Begin_Date, End_Date, Unit_Owner_First_Name, Unit_Owner_Last_Name
            ,       Unit_Owner_Login, Unit_Owner_CorporateID, UnityParentId, UnityTypeId, Unit_Level
            ,       Magnitude_Code, ImportDate, ReplicationLevel 
            ) 
    AS
    (
            SELECT
                    dt.Jahr
            ,       a.UnityId 
            ,       a.UnityId 
            ,       a.Analytical_Code 
            ,       a.Unit_Code 
            ,       a.Unit_Name 
            ,       cast(case when a.Active = 'True' then 1 else 0 end as bit)
            ,       a.Show_in_org_chart 
            ,       a.Begin_Date 
            ,       a.End_Date 
            ,       a.Unit_Owner_First_Name 
            ,       a.Unit_Owner_Last_Name 
            ,       a.Unit_Owner_Login 
            ,       a.Unit_Owner_CorporateID 
            ,       a.UnityParentId 
            ,       a.UnityTypeId 
            ,       a.Unit_Level 
            ,       a.Magnitude_Code
            ,       a.ImportDate
            ,       1
            FROM    [Staging_INPUT].[DBO].[OBS_Workunit]    a
            JOIN    (
                            SELECT
                                    YEAR(IMPORTDATE)        Jahr
                            ,       MAX(IMPORTDATE)         Datum
                            FROM    [Staging_INPUT].[DBO].[OBS_Workunit]
                            GROUP   BY
                                    YEAR(IMPORTDATE) 
                    )                                       dt
            ON      a.ImportDate                    =       dt.datum
            WHERE   a.unitytypeid                   =       12
            UNION   ALL
            SELECT
                    b.Jahr
            ,       b.UnityId 
            ,       a.UnityId 
            ,       b.Analytical_Code 
            ,       a.Unit_Code 
            ,       a.Unit_Name 
            ,       cast(case when a.Active = 'True' then 1 else 0 end as bit)
            ,       a.Show_in_org_chart 
            ,       a.Begin_Date 
            ,       a.End_Date 
            ,       a.Unit_Owner_First_Name 
            ,       a.Unit_Owner_Last_Name 
            ,       a.Unit_Owner_Login      
            ,       a.Unit_Owner_CorporateID 
            ,       a.UnityParentId 
            ,       a.UnityTypeId   
            ,       a.Unit_Level 
            ,       a.Magnitude_Code
            ,       a.ImportDate 
            ,       b.ReplicationLevel + 1
            FROM    [Staging_INPUT].[DBO].[OBS_Workunit]    a
            JOIN    cte                                     b
            ON      a.UnityId                       =       b.UnityParentId
            AND     a.ImportDate                    =       b.ImportDate
            AND     a.UnityTypeId                   >=      6 
    )
    ,       Company
    AS
    (
            SELECT  DISTINCT 
                    Jahr
            ,       UnityId
            ,       LEFT(REPLACE(Magnitude_Code,'XE','U'),4) CompanyUID
            FROM    cte
            WHERE   UnityTypeId     =       7
            AND     Active          =       1

    )
    ,       BUs 
    AS
    (
            SELECT  DISTINCT 
                    a.Jahr                          JAHR
            ,       a.UnityId                       UnityId
            ,       c.Analytical_Code               BU_CODE
            ,       c.Unit_Name                     BU_NAME
            ,       b.CompanyUID
            FROM    cte                     a

            JOIN    Company                 b
            ON      a.Jahr          =       b.Jahr
            AND     a.UnityId       =       b.UnityId
            AND     a.Active        =       1

            JOIN    cte                     c
            ON      a.Jahr          =       c.Jahr
            AND     a.UnityId       =       c.Unit_UnityId
            AND     c.Active        =       1

            WHERE   ISNULL(c.Analytical_Code,'') != ''
    )
    SELECT  DISTINCT
            a.JAHR                          JAHR
    ,       a.BU_CODE                       BU_CODE
    ,       a.BU_NAME                       BU_NAME
    ,       'EUROPE\'
    +       b.Unit_Owner_Login              BU_LOGIN
    ,       b.Unit_Owner_Last_Name 
    +       ', '
    +       b.Unit_Owner_First_Name         BU_LOGIN_NAME
    ,       a.CompanyUID                    COMPANY_UID
    FROM    BUs                             a

    JOIN    cte                             b
    ON      a.Jahr                  =       b.Jahr
    AND     a.UnityId               =       b.UnityId
    AND     b.Active                =       1

    UNION   

    SELECT  DISTINCT
            a.Jahr                          JAHR
    ,       a.BU_CODE                       BU_CODE
    ,       a.BU_NAME                       BU_NAME
    ,       c.BU_LOGIN                      BU_LOGIN
    ,       c.BU_LOGIN_NAME                 BU_LOGIN_NAME
    ,       a.CompanyUID                    COMPANY_UID
    FROM    BUs                             a

    CROSS   JOIN (
                    SELECT  DISTINCT
                            [Last Name]     COLLATE Latin1_General_100_CI_AS
                    +       ', '
                    +       [First Name]    COLLATE Latin1_General_100_CI_AS BU_LOGIN_NAME   
                    ,       'EUROPE\'
                    +       [User Id]       COLLATE Latin1_General_100_CI_AS BU_LOGIN  
                    FROM    NAVISION.dbo.Employee
                    WHERE   [Global Dimension 2 Code]       IN      (
                                                                            10061
                                                                    ,       10062
                                                                    )
                    AND     ISNULL([User Id],'')            !=      ''
                 )                          c

GO 

Execution times and stats:

use NAVISION

GO

set statistics io on 
set statistics time on

select * 
from NAVISION.dbo.UnitLoginHistory 
where Jahr = 2017 
order by 1,2

SQL Server parse and compile time: CPU time = 0 ms, elapsed time = 0 ms.

(34119 row(s) affected) Table 'Worktable'. Scan count 1607, logical reads 253696, physical reads 0, read-ahead reads 1238, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. Table 'U415 Altran Engineering GmbH$Employee'. Scan count 1, logical reads 128, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. Table 'U388 Altran Aviation GmbH$Employee'. Scan count 1, logical reads 42, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. Table 'U354 Altran Service GmbH$Employee'. Scan count 1, logical reads 210, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. Table 'U353 AIH Holding GmbH Co KG$Employee'. Scan count 1, logical reads 934, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. Table 'OBS_Workunit'. Scan count 46286, logical reads 10430933, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

SQL Server Execution Times: CPU time = 1363546 ms, elapsed time = 1455980 ms.

use Staging_INPUT
GO

set statistics io on 
set statistics time on

select * 
from NAVISION.dbo.UnitLoginHistory 
where Jahr = 2017 
order by 1,2

SQL Server parse and compile time: CPU time = 0 ms, elapsed time = 0 ms.

(34119 row(s) affected) Table 'Worktable'. Scan count 582, logical reads 576096, physical reads 0, read-ahead reads 146, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. Table 'Workfile'. Scan count 0, logical reads 0, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. Table 'OBS_Workunit'. Scan count 53573, logical reads 485656, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. Table 'U415 Altran Engineering GmbH$Employee'. Scan count 1, logical reads 128, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. Table 'U388 Altran Aviation GmbH$Employee'. Scan count 1, logical reads 42, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. Table 'U354 Altran Service GmbH$Employee'. Scan count 1, logical reads 210, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. Table 'U353 AIH Holding GmbH Co KG$Employee'. Scan count 1, logical reads 934, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

SQL Server Execution Times: CPU time = 15047 ms, elapsed time = 28007 ms.

  • 3
    Post your view code –  Jul 13 '17 at 08:39
  • This seems quite complex. – Evaldas Buinauskas Jul 13 '17 at 09:01
  • That's a beast of a query. Is it possible the first time, when it took a while was just it figuring out the correct plan to use? In otherwords, you changed databases, and it seemed to run OK. Have you tried running it twice on the same database to make sure there's no compilation time component? – Xedni Jul 13 '17 at 09:02
  • Yes, I tried it various times using various database connections. And always, when connected to databas NAVISION that contains the query, it takes around 30 minutes and always, when connected to master or Staging_INPUT, it's done in seconds.... – Stefan Corsten Jul 13 '17 at 09:05
  • The next step I'd suggest is to pull some statistics from the two runs. run `set statistics io on` and `set statistics time on`. Those will give you indiciations of time, cpu, and IO. Are those two different databases running different versions of SQL perhaps? Even in different compatability modes? It makes me wonder if each is going about the query in a different way. – Xedni Jul 13 '17 at 09:06
  • 1
    Are indexes in place? It would be great if you'd be able to post execution plan – Evaldas Buinauskas Jul 13 '17 at 09:13
  • Is it possible the underlying data changed between executions? If so, the rCTE might have had to do considerably more work during one of them. If the data didn't, and in fact DOESN'T change very much, you could also consider pre-populating a table with these results and querying that instead. Ultimately though I think you'll need to provide some performance metrics such as time, io and execution plan to really say anything more specific. – Xedni Jul 13 '17 at 09:17
  • is_parameterization_forced is the same? – sepupic Jul 13 '17 at 09:38
  • I have posted the statistics time and io in the original post, as there is not enough room for it in the comments. @Xendi, you are right, the NAVISION database is SQL Server 2016 and the other one is in 2012 compatibily mode. All configurable Options of both databases are equal. the paramertization Option on both databases is set to simple. – Stefan Corsten Jul 13 '17 at 10:29
  • You mention it's Navision. How many companies do you have in your navision setup? Reason I ask is that every company will create hundreds of tables. I've seen shipping companies where every vessel was set up as a company, resulting in just expanding the list of tables in SSMS taking minutes – SchmitzIT Jul 13 '17 at 10:29
  • The underlying data changes only once a day, so I am sure, there was no change between the executions. – Stefan Corsten Jul 13 '17 at 10:30
  • We have 4 companies in the Navision setup – Stefan Corsten Jul 13 '17 at 10:31
  • That's not it then. That ought to perform just fine. (I also didn't see you meant you directly queried this view. I read the initial post as you querying the systables. My mistake). – SchmitzIT Jul 13 '17 at 10:32
  • Make sure the tables in both DBs have the same set up (indices etc.). Make sure they are both using a cached plan because if one uses a cached plan and one doesn't, then execution time will be different. Look at the execution plan for both and see what the difference is (you can look the estimated plan so you don't have to wait for the query to execute). If you do not know what execution plans are, search online. – CodingYoshi Jul 13 '17 at 12:47
  • 1
    Are the collations in all databases the same? Why do you use this "COLLATE Latin1_General_100_CI_AS" in your view? – Alex Jul 13 '17 at 13:00
  • There is only one table used in the view ([Staging_INPUT].[DBO].[OBS_Workunit]) and this one already uses Indexes. The CROSS JOIN uses another view (NAVISION.dbo.Employee), that selects data directly from NAVISION. As in NAVISION the collation is different than in the other databases, I need to use the "COLLATE Latin1_General_100_CI_AS" in this part. – Stefan Corsten Jul 13 '17 at 14:37
  • The both execution plans are different as well, but unfortunately they are to big to post them here – Stefan Corsten Jul 13 '17 at 15:07

1 Answers1

0

The different performance must be due to different execution plans. Since the plans differ depending on the database context, this suggests different settings that influence the plan. There are so many default SET options and properties that can vary by database, you have missed one or two.

I suggest generate CREATE DATABASE scripts for the 2 databases and compare the scripts.

EDIT:

A difference in the database compatibility level setting can affect execution plans. SQL Server will use the legacy cardinality estimator for databases in the 110 (SQL Server 2012) level whereas the newer CE will be used for databases in 120 and later unless the LEGACY_CARDINALITY_ESTIMATION database scoped setting is turned on.

Dan Guzman
  • 43,250
  • 3
  • 46
  • 71
  • This does not answer the question. Should be a comment. – CodingYoshi Jul 13 '17 at 12:36
  • @CodingYoshi, why shouldn't this be an answer? Only different plans will cause these symptoms and, plans that differ by database context with all other things being equal must be due to database settings/properties, AFAIK. – Dan Guzman Jul 13 '17 at 12:42
  • The CREATE DATABASE scripts differ only in the compatibility level, which is 130 for NAVISION and 110 for Staging_INPUT, what meens SQL Server 2016 for NAVISION and SQL Server 2012 for Staging_INPUT. The other difference is the recovery mode, which is set to SIMPLE for NAVISION and FULL for Staging_INPUT. All other options are exactly the same – Stefan Corsten Jul 13 '17 at 14:42
  • @StefanCorsten, compatibility level matters. I added more info to my answer. – Dan Guzman Jul 13 '17 at 15:54
  • Hi Dan, setting the LEGACY_CARDINALITY_ESTIMATION ON changed the performance from around 30 minutes down to around 2 minutes on NAVISION. On Staging_INPUT it is still on around 10 seconds. Means, performance is better, but still not the same and still not the best.... – Stefan Corsten Jul 13 '17 at 17:29
  • @StefanCorsten, can up upload the execution plans to https://www.brentozar.com/pastetheplan/? – Dan Guzman Jul 13 '17 at 17:39
  • Hi Dan,I uploaded the plans: – Stefan Corsten Jul 13 '17 at 17:58
  • [Navision](https://www.brentozar.com/pastetheplan/?id=HyTHmESBW) , [Staging_INPUT](https://www.brentozar.com/pastetheplan/?id=Bke_m4Br-) – Stefan Corsten Jul 13 '17 at 17:58
  • @StefanCorsten, the estimated plans look the same now as far as I can tell. Can you upload the actual ones? – Dan Guzman Jul 13 '17 at 18:20
  • Hi Dan, the plans are the actual ones and indeed they seem to be equal now. But for any reason, the execution time still differs... What I can see is, that statistics io still has different results. So, executed from Staging_Input it uses the internal objects "Worktable" and "Workfile", while from Navision it only uses "Worktable" – Stefan Corsten Jul 14 '17 at 07:43
  • @StefanCorsten, I don't see the actual counts in the operators of the uploaded plans. I'd expect the actual counts to be the same now that the plans are identical, assuming data didn't change between executions. If performance varies with the identical plans and counts, it will likely be due to caching and multi-user concurrency that can affect the number of physical reads, IO performance, memory grants, short term blocking, etc. – Dan Guzman Jul 14 '17 at 11:38
  • On a dedicated test system with no shared resources, you can level the playing field by clearing the caches before each query (`CHECKPOINT; DBCC FREEPROCCACHE; DBCC DROPCLEANBUFFERS;`). – Dan Guzman Jul 14 '17 at 11:40