0

When I do an extract from multiple files and include part of the filename in the fields list and in the FROM clause (e.g. FROM "/input/filename-{filedate:*}.nc"), the resulting output file only contains a header row. If I remove "filedate" from the fields list and the FROM clause, I get the correct output.

I noticed in the job graph that when including "filedate", an "Empty Input" and an "Extract Cross" step is added before the "PodAggregate" step, and in the "Extract Cross" no data is written. What is this step?

Also, if I run the original extract including "filedate" locally, I get the correct output, so it's only in ADLA this error occurs.

I use a custom extractor and I don't know if this has anything to do with it. I haven't tested with a built-in extractor.

Magnus Johannesson
  • 432
  • 7
  • 20

1 Answers1

0

We released the new "fast file set" option by default. Unfortunately, it introduced a regression for some plans. Until we fix it, please add the following statement to your script:

SET @@InternalDebug = "FileSetV2:off";

Our apologies for any inconvenience this may have caused.

Michael Rys
  • 6,684
  • 15
  • 23