0

I have created a restAPI connection with base url:https://heineken-dev.collibra.com/rest/2.0/assets?offset={offset}

and in copy Data activity source pagination rules , I have added as below. paginationRule

but in the restAPI result set: total: 14518 which is the total number of records. I want to get this total 14518 into end condition of offset.Can someone help me do that.

Thanks in advance!!

Sukanya
  • 33
  • 1
  • 1
  • 9

1 Answers1

0

ADF pagination rule to get total row count from rest API and add as end condition.

As per you requirement in the rest API result set you are getting total: 14518 which is the total number of records. And you want to use this in the end condition.

To achieve this:

here I am using this sample Rest Api

  • First you need to get the result set of api from web activity. enter image description here Result set: here you will get that total: 14518 enter image description here
  • Now in copy activity set the pagination rule as below: set Renge as 1::1000 so it will loop the url infinite time on the offset of 1000 and the set the end condition as MaxRequestNumber : @string(activity('Web1').output.total_pages) so it will end the looping when it reaches the max request number. enter image description here

UPDATE

updating answer for other community members. Agreed with @Sukanya by adding $.total the value of total will be taken from the rest API's response in the end condition as below:

enter image description here

Pratik Lad
  • 4,343
  • 2
  • 3
  • 11
  • I could solve this issue by adding $.total in the end condition(in range) which will take the total from rest API. Hope this helps someone in future. – Sukanya Apr 20 '23 at 12:05