Testing DAX calculation with NBi

Question

I'm doing some research on automated test tool for our SSAS Tabular project. I found NBi and thought it is really cool. I attempted to set it up and successfully ran some basic tests. However, when I attempted to test dax calculation, it says "Function not found" (see screenshot). It sounds like it does not support SUM, but given that SUM is a basic function I would imagine it should work. Since I'm new to this tool, I wanted to double check if I've done something wrong or it is simply what the error is saying... (not supported function).

I went back and review NBi documentation and it mentioned to check out their NCAL.dll for all available expression. Unfortunately, I'm unable to open a readable version of that dll file. Any help is appreciated.

Here is the formula I want to test:

=SUMX(FILTER(MyTable, AND(MyTable[Date] = EARLIER(MyTable[Date]), MyTable[Account] = EARLIER(MyTable[Account]))), MyTable[Amount])

XML code (nbits) file

 <test name="My second test: Calculated column compared to DAX formula">
  <system-under-test>
   <execution>
    <query connectionString="Provider=MSOLAP.7;Data Source...">
     <![CDATA[    
    EVALUATE
    SUMMARIZE (MyTable, MyTable[Date], MyTable[Account], MyTable[Amount], MyTable[CalculatedAmount])
     ]]>
    </query>
   </execution>
  </system-under-test>
  <assert>
    <evaluate-rows>
      <variable column-index="0">Date</variable>
      <variable column-index="1">Account</variable>
      <variable column-index="2">Amount</variable>
      <variable column-index="3">CalculatedAmount</variable>
      <expression column-index="3" type="numeric" tolerance="0.01"> = SUMX(FILTER(MyTable, AND(MyTable[Date] = EARLIER(MyTable[Date]), MyTable[Account] = EARLIER(MyTable[Account]))), MyTable[Amount])</expression>
    </evaluate-rows>
  </assert>
 </test>

Answer 1

NBi supports the evaluation of DAX queries in the query tag but not in an expression tag. Expression and evaluate-rows tags are not designed to compare two queries. To achieve this, change your test to use the assertion equalTo between your two queries. It will be easier and will work.

I guess a better question would be how do I test a measure and a calculated column in term of ensuring that another developer doesn't accidentally change the calculation/expression I entered when designing the Tabular model?

I'll answer at three levels: conceptual, logical and technical.

At the conceptual level, your test is wrong: you should never use the same implementation in your assertion and in your system-under-test. This is not specifc to NBi or to any framework but to all automated tests. The role of a test is not ensure that someone doesn't change something but to ensure that something gives the correct result. Comparing an artifact to itself will always result in a green test even if your implementation is wrong. In this case, you must change your assertion with a concrete static result or you need to create a sql statements resulting in the same calculation of your database or find another query in MDX resulting in the same result.

At the logical level the following sentence is not correct

Here is the formula I want to test:

You've defined this formula in your assert and not in your system-under-test. It means that it's not what you're testing but it's your reference (something you're 100% sure that it's correct). What you're testing is the query EVALUATE SUMMARIZE (MyTable, MyTable[Date], MyTable[Account], MyTable[Amount], MyTable[CalculatedAmount]).

At the technical level, using an evaluate-rows is nopt the correct option. This assertion is not expecting a function or a query but an expression based on row's variable (no DAX, no SQL, ...). The usage of EARLIER is a clear sign that it won't be possible. In your case, you must compare two queries probably something as:

<assert>
  <equalTo>
    <column index="0" role="key" type="dateTime"/>
    <column index="1" role="key" type="numeric"/>
    <column index="2" role="value" type="numeric"/>
    <column index="3" role="value" type="numeric" tolerance="0.01"/>
    <query>
      EVALUATE SUMMARIZE (MyTable, MyTable[Date], MyTable[Account], MyTable[Amount], SUMX(FILTER(MyTable, AND(MyTable[Date] = EARLIER(MyTable[Date]), MyTable[Account] = EARLIER(MyTable[Account]))), MyTable[Amount])
    </query>
  </equalTo>
</assert>

PS: I'm clearly not a specilist of DAX and I'm not sure the query above is valid from a syntax point of view.