Python while loop token not updating

Question

I am exploring azure management APIs. The ADF monitor pipeline, returns only 100 records at a time. So I created a while loop, but for some reason, not sure what, not able to get the next token.

ct = d.get('continuationToken','')
c = 1
while ct!='':
    req_body = self.getDataBody(ct)
    data = self.getResponse(data_url,data_headers,req_body)
    nct = self.getContinuationToken(data,c)
    c = c+1
    print(c)
    if ct == nct:
        print(ct)
        print(nct)
        print('duplicate token')
        break
    ct = nct
    if ct == '':
        break

Here in the next iteration next token is not getting updated.

Update:

following the functions that the above code is using

def getDataBody(self,ct):
    start_date = datetime.now().strftime("%Y-%m-%d")
    end_date = (datetime.now() + timedelta(days=1)).strftime("%Y-%m-%d")
    data_body = {'lastUpdatedAfter': start_date, 'lastUpdatedBefore': end_date}
    if ct!='':
        data_body['continuationToken'] = ct
    return data_body


def getResponse(self,url,headers,body):
    data = requests.post(url,headers=headers,data=body)
    return data.text


def getContinuationToken(self,data,c):
    d = json.loads(data)
    with open(f'data/{c}.json','w') as f:
        json.dump(d,f)
    return d.get('continuationToken','')

Have you tried seeing if the response differs in the loop? maybe you're always getting the same response. — Farhood ET, Aug 02 '21 at 05:30
it differs if a manually run it multiple times by passing the updated continuation token, but in loop the token stays same — Aman Khandelwal, Aug 02 '21 at 05:34
Have you tried writing a custom dummy getContinuationToken that always returns something different to see whether the problem will be gone or not? — Farhood ET, Aug 02 '21 at 05:36

score 0 · Answer 1 · answered Aug 23 '21 at 12:24

0

you can try with increasing the timeout in the ADF activity may be due to the timeout setting in your current ADF activity is less than the actual time taking to execute that API .

answered Aug 23 '21 at 12:24

RKM

1,234
1
4
9

Python while loop token not updating

1 Answers1