I have a U-SQL script which processes some data using some UDOs and then finally outputs a file back to Azure Data Lake.
The expected behaviour is that if the file that is generated is empty, the script should fail, however I am unable to get it to do so.
I tried implementing a simple reducer which counts the number of rows and throws an exception when the count is zero. However, it doesnt get called during execution since the file is empty, and the script succeeds.
Any ideas on how to get this done?
The reduce function is below:
public override IEnumerable<IRow> Reduce(...)
{
long count = 0;
foreach (var row in input.Rows)
{
count++;
break;
}
if (count == 0)
{
throw new Exception("Zero rows found in table");
}
else
{
output.Set("Count", count);
yield return output.AsReadOnly();
}
}