(Step Functions Activity Worker) Best practice for handling long polling timeouts in boto?

Question

I am working on my first Step Functions Activity Worker (EC2). Predictably, after 5 minutes of long polling with no activity from the Step Functions state machine, the client connection times out with the error:

botocore.exceptions.ReadTimeoutError: Read timeout on endpoint URL: "https://states.us-east-1.amazonaws.com/"

Would it be better to catch the error and retry the long poll (every 5 minutes when no activity is present), or try to terminate the call early and retry before the error? I've thought about using a different type of loop, but I want to maximize the value of long polling and not repeatedly request against the Step Functions API (although if that's the best way I'll do it).

Thank you,

Andrew

import boto3
import time
import json

region = 'us-east-1'
activity_arn = 'arn:aws:states:us-east-1:754112345676:activity:Process_Imagery' 

while True:
    client = boto3.client('stepfunctions', region_name=region)
    response = client.get_activity_task(activityArn=activity_arn,
                                    workerName='imagery_processor')    
    activity_token = response['taskToken']
    input_params = json.loads(response['input'])

    print("================")
    print(input_params)
    client.send_task_success(taskToken=activity_token, output='true')

I did some more testing today (packet capture), which showed the instance and Step Functions API communicating just about every 1 minute before Botocore eventually kills the attempt (~5 min). This leads me to believe I need to handle the NULL return better. Will research. — Andrew6850, Oct 23 '18 at 21:19

Andrew6850 · Accepted Answer · 2018-10-24T13:19:21.310

I believe I answered my own question here. The AWS documentation states: "The maximum time the service holds on to the request before responding is 60 seconds. If no task is available within 60 seconds, the poll returns a taskToken with a null string."

However, instead of string being returned, I believe the JSON response from StepFunctions has no 'taskToken' at all. This while loop works:

import boto3
import time
import json
from botocore.config import Config as BotoCoreConfig

region = 'us-east-1'
boto_config = BotoCoreConfig(read_timeout=70, region_name=region)
sf_client = boto3.client('stepfunctions', config=boto_config)
activity_arn = 'arn:aws:states:us-east-1:754185699999:activity:Process_Imagery'  

while True:
    response = sf_client.get_activity_task(activityArn=activity_arn,
                                    workerName='imagery_processor')    

    if 'taskToken' not in response:
        print('No Task Token')
        # time.sleep(2)
    else:
        print(response['taskToken']) 
        print("===================")
        activity_token = response['taskToken']
        sf_client.send_task_success(taskToken=activity_token, output='true')

(Step Functions Activity Worker) Best practice for handling long polling timeouts in boto?

1 Answers1