Rate Exceeding in workflow_execution polling

Question

I am currently trying to modify a plugin for posting metrics to new-relic via AWS. I have successfully managed to make the plugin post metrics from swf to new relic (not originally in plugin), but have encountered a problem if the program runs for too long.

When the program runs for a bout 10 minutes I get the following error:

Error occurred in poll cycle: Rate exceeded

I believe this is coming from my polling swf for the workflow executions

domain.workflow_executions.each do |execution|

        starttime = execution.started_at
        endtime = execution.closed_at
        isOpen = execution.open?
        status = execution.status

        if endtime != nil
          running_workflow_runtime_total += (endtime - starttime)
          number_of_completed_executions += 1
        end

        if status.to_s == "open"
          openCount = openCount + 1
        elsif status.to_s == "completed"
          completedCount = completedCount + 1
        elsif status.to_s == "failed"
          failedCount = failedCount + 1
        elsif status.to_s == "timed_out"
          timed_outCount = timed_outCount + 1
        end

      end

This is called in a polling cycle every 60 seconds

Is there a way to set the polling rate? Or another way to get the workflow executions?

Thanks, here's a link to the ruby sdk for swf => link

There's really nothing for us to work from because you only gave us the bare beginning to your loop. See http://sscce.org for the sort of information we need to help you. — the Tin Man, Jul 18 '13 at 05:15
Thanks for the comment and suggestion. I updated the code, let me know if that is more helpful. — user2495753, Jul 18 '13 at 05:20

xaxxon · Accepted Answer · 2013-07-18T07:00:37.097

2

The issue is likely that you are creating a large number of workflow executions and each iteration through the loop in workflow_executions is causing a lookup, which eventually is exceeding your rate limit.

This could also be getting a bit expensive, so be careful.

It's not clear what you're really trying to do, so I can't tell you how to fix it unless you post all your code (or the parts around calls to SWF).

You can see here:

https://github.com/aws/aws-sdk-ruby/blob/05d15cd1b6037e98f2db45f8c2597014ee376a59/lib/aws/simple_workflow/workflow_execution_collection.rb

That a call is made to SWF for each workflow in the collection.

edited Jul 18 '13 at 07:00

answered Jul 18 '13 at 06:37

xaxxon

19,189
5
50
80

But I don't think I'm creating any more workflow executions in this cycle, only looking up the ones that are currently running. This is also the only part of my code where I am doing anything with said workflow executions. Does the number of executions I am looking up factor in directly to my limit? or is the limit based on the total number of broad lookups I am doing? – user2495753 Jul 18 '13 at 06:48
It doesn't pull them all at once. I'm guessing each iteration through your body of code in the {}'s causes a call to SWF to get data for the next execution. It's the number of executions times the number of times you call domain.workflow_executions.each – xaxxon Jul 18 '13 at 07:01

Rate Exceeding in workflow_execution polling

1 Answers1