0

I am getting following error while pushing the data into Azure Search Index from OnPremSql database through ADF.

 Copy activity encountered a user error at Sink side: ErrorCode=UserErrorAzuerSearchOperation,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Error happened when writing data to Azure Search Index 'coursemap'.,Source=Microsoft.DataTransfer.ClientLibrary.AzureSearch,''Type=Microsoft.Azure.Search.IndexBatchException,Message=1 of 1000 indexing actions in the batch failed. The remaining actions succeeded and modified the index. Check the IndexResponse property for the status of each index action.,Source=Microsoft.Azure.Search,'.

Any idea how to resolve this issue?

Eugene Shvets
  • 4,561
  • 13
  • 19
Jai
  • 416
  • 6
  • 20
  • It would be useful if you were able to show us your datasets, linked services and pipeline, obviously obfuscating any sensitive information. If they are too large to post here, consider gist. – wBob May 25 '17 at 07:12

1 Answers1

1

It looks like you're using an older version 2.8 of the data management gateway, which doesn't show all the available detail of the IndexBatchException. Consider updating to the latest version 2.9. With that version, you will be able to see item-level errors for each failed document within the batch.

In this case, we've looked at the telemetry for your service and the errors are conflict (409) errors like the ones below: 

Error indexing document with key '100665_1458': status 409
Error indexing document with key '100666_1255': status 409
Error indexing document with key '100665_1280': status 409 
Error indexing document with key '100669_1280': status 409
Error indexing document with key '100670_1280': status 409
Error indexing document with key '100670_1280': status 409

It looks like you have multiple documents with the same document key inside a single batch, and/or you have multiple copy activities indexing data with the same document keys into the same index at the same time, and that creates those conflicts.

Also, version 2.9 of the gateway has a better retry policy that should decrease the occurrence of these conflict errors.

Eugene Shvets
  • 4,561
  • 13
  • 19
  • yeah thanks. I just checked there were some duplicate data. But if i am not wrong i think that if azure search gets duplicate document key then it overwrites the data for that particular document. – Jai May 26 '17 at 10:02
  • 1
    @Jai It depends. If the indexing requests are sent concurrently to the search service, the result might be 409 (Conflict). – Bruce Johnston May 26 '17 at 17:27