1

In my Rails project, I'm using VCR and RSpec to test HTTP interactions against an external REST web service that only allows calls to it once per second.

What this means so far is that I end up running my test suite until it fails due to a "number of calls exceeded" error from the web service. At that stage though, at least some cassettes get recorded, so I just continually run the test suite until eventually I get them all recorded and the suite can run using only cassettes (my default_cassette_options = { record: :new_episodes }). This doesn't seem like an optimal way to do things, especially if I find I need to re-record my cassettes in the future often, and I worry that constant calls could land me on a blacklist with the web service (there's no test server they have that I know about).

So, I ended up trying putting calls to sleep(1) in my Rspec it blocks directly before the call to the web service is made, and then refactored those calls up into the VCR configuration:

spec/support/vcr.rb

VCR.configure do |c|
  # ...
  c.after_http_request do |request, response|
    sleep(1)
  end
end

Although this seems to work fine, is there a better way to do this? At the moment, if a call to an external service that doesn't have a cassette already is the final test in the suite, then the suite sleeps unnecessarily for 1 second. Likewise, if the time between 2 web service calls without cassettes in the test suite is more than once second, then there's another unnecessary pause. Has anyone made any kind of logic to test for these kinds of conditions, or is there a way to elegantly do this in the VCR configuration?

Myron Marston
  • 21,452
  • 5
  • 64
  • 63
Paul Fioravanti
  • 16,423
  • 7
  • 71
  • 122
  • 1
    Given aggressive rate limiting like that, I would build throttling right into the client. Presumably if it's blowing up the test suite it could also blow up in production. – willglynn Oct 11 '12 at 03:57
  • +1 Good point; didn't even think of that. The plan is only for these APIs to be called on a cron job once a day, but since during the cron there will be multiple calls done, the issue will still persist. Looks like I have a client design issue to look at. – Paul Fioravanti Oct 11 '12 at 05:09

2 Answers2

3

First off, I would recommend against using :new_episodes as your record mode. It has it's uses, but the default (:once) is generally what you want. For accuracy, you want to record a cassette as a sequence of HTTP requests that were made in a single pass. With :new_episodes, you can wind up with cassettes that contain HTTP interactions that were recorded months apart but are now being played back together, and the real HTTP server may not respond in that same fashion.

Secondly, I'd encourage you to listen to the pain exposed by your tests, and find ways to decouple most of your test suite from these HTTP requests. Can you find a way to make it so that just the tests focused on the client, and the end-to-end acceptance tests make the requests? If you wrap the HTTP stuff in a simple interface, it should be easy to substitute a test double for all the other tests, and more easily control your inputs.

That's a longer term fix, though. In the short term, you can tweak your VCR config like so:

VCR.configure do |vcr|
  allow_next_request_at = nil
  filters = [:real?, lambda { |r| URI(r.uri).host == 'my-throttled-api.com' }]

  vcr.after_http_request(*filters) do |request, response|
    allow_next_request_at = Time.now + 1
  end

  vcr.before_http_request(*filters) do |request|
    if allow_next_request_at && Time.now < allow_next_request_at
      sleep(allow_next_request_at - Time.now)
    end
  end
end

This uses hook filters (as documented) to run the hooks only on real requests to the API host. allow_next_request_at is used to sleep the minimum amount of time necessary.

Myron Marston
  • 21,452
  • 5
  • 64
  • 63
  • Thanks very much for this response. I agree with you and @willglyn that these issues have exposed some design issues on my service client class, and that I need to decouple most of my test suite from these requests, so I'll be having a look into that. In the meantime, your short term fix worked for me, so many thanks for that. However, I did get errors for the `:real?` symbol, and when I removed it from the `filters`, I got a ` undefined method `uri' for nil:NilClass` (after changing `r` to `req`). Since this is only a short term fix, I'll do without referencing the `filters` for now. – Paul Fioravanti Oct 11 '12 at 05:17
  • Glad it helped. I updated the code a bit...I think this should work for you now (although, I'm puzzled by the `:real?` error--that's definitely tested and it works--what error did you get?) – Myron Marston Oct 11 '12 at 06:26
  • Thanks for the edit. The code works as expected now. The error I got originally was `undefined method 'real?' for nil:NilClass`, but now that the number of arguments in the `lambda` block went from 2 to 1, that seemed to fix the issue. Many thanks again for your help! – Paul Fioravanti Oct 11 '12 at 07:18
3

An alternative may be to use APICache as a proxy around your HTTP library, as it will handle rate limiting on your behalf.

APICache.get("my_albums", period => 1) do
  FlickrRb.get_all_sets
end

This will raise APICache::CannotFetch when you attempt to call the API more often than your limit.

Here's a link to the APICache Github repo

qnm
  • 521
  • 3
  • 14