1

I am ingesting data to ADX from Azure Data Factory using the built-in connector. I have many many pipelines and dataflows that successfully output to ADX without issue. Also, the data flow that is now causing me a problem was working perfectly until a recent change (which I made to similar dataflows without issue), which appears fine when I inspect the data being sent to ADX.

After the recent change the ingestion process stalls and can sit foe hours un-completed until cancelled. I receive the following error in Azure Data Factory after the timeout has been reached.

>     Operation on target SWANTEL OS and Site Mapping failed: {"StatusCode":"DFExecutorUserError","Message":"Job failed due to
> reason: at Sink 'ADXOuput':
> shaded.msdataflow.com.microsoft.kusto.spark.utils.RetriesExhaustedException:
> Failed to move extents after 401
> tries","Details":"shaded.msdataflow.com.microsoft.kusto.spark.utils.RetriesExhaustedException:
> Failed to move extents after 401 tries\n\tat
> shaded.msdataflow.com.microsoft.kusto.spark.utils.KustoClient.moveExtentsWithRetries(KustoClient.scala:203)\n\tat
> shaded.msdataflow.com.microsoft.kusto.spark.utils.KustoClient.moveExtents(KustoClient.scala:241)\n\tat
> shaded.msdataflow.com.microsoft.kusto.spark.utils.KustoClient$$anonfun$1.apply$mcV$sp(KustoClient.scala:316)\n\tat
> shaded.msdataflow.com.microsoft.kusto.spark.utils.KustoClient$$anonfun$1.apply(KustoClient.scala:258)\n\tat
> shaded.msdataflow.com.microsoft.kusto.spark.utils.KustoClient$$anonfun$1.apply(KustoClient.scala:258)\n\tat
> scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)\n\tat
> scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)\n\tat
> scala.concurrent.impl.ExecutionContextImpl$AdaptedForkJoinTask.exec(ExecutionContextImpl.scala:121)\n\tat
> scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)\n\tat
> scala.concurrent.fo"}

When I inspect the operations in ADX I see the following entries, which suggests there is an issue with the materialised view:

ADX Errors

Can't Move Extent Error

Does anyone know what would prevent extents from being moved? The only change I have made is to add some additional rows to the output, with the same columns of data as the original output which was working. This one has me stumped!

James Turner
  • 61
  • 2
  • 7

1 Answers1

3

Please see this section in the materialized views docs about this error:

  • The source table of a materialized view:

    • Must be a table into which data is directly ingested, either using one of the ingestion methods, using an update policy, or from query commands.
      • Using move extents from other tables to the source table of the materialized view is only supported if using setNewIngestionTime property as part of the move extents command (refer to .move extents command for more details).
      • Moving extents to the source table of a materialized view, while not using setNewIngestionTime may fail with one of the following errors:
        • Cannot drop/move extents from/to table 'TableName' since Materialized View 'ViewName' is currently processing some of these extents.
        • Cannot move extents to 'TableName' since materialized view 'ViewName' will not process these extents (can lead to data loss in the materialized view).

Using spark connector to ingest to a source table of a materialized view is supported as of spark connector version 3.0. Please upgrade your connector and check if this resolves the issue.

yifats
  • 2,419
  • 5
  • 6
  • Thanks, I am using the Azure Data Factory connector, not a custom app or anything to do the ingest, which as I said works for other identical ingestion activities. I'm not knowingly trying to move extents between tables. All my pipelines and dataflows output to the same single table, which is the source of the materialized view. – James Turner Jul 26 '22 at 14:59
  • The Data Flow integration does not currently use the latest version with the fix @yifats mentioned. We will work on upgrading this. For now either use a copy activity to Kusto or set a destination table that is not a materialized view. – Ohad Bitton Jul 28 '22 at 08:02