4

My specs will pass in master branch. If I create a new branch and modify some code completely unrelated to the subscriptions, they'll fail with this. The only way I can get them to pass is to change my vcr.rb to have :record => :new_episodes.

If I leave that option on, then almost every time my specs run I have new modified data files for cassettes that end up being committed which really dilute the logs for Git.

Any suggestions on how to handle this? A lot of the specs that break are based on this matcher:

  describe "#change_plan_to", vcr: {match_requests_on: [:method, :uri, :body]} do

Is this matcher too open to changes? I wasn't able to get the specs to pass any other way with stripe api calls.

Failure/Error: @subscription.create_stripe_customer
     VCR::Errors::UnhandledHTTPRequestError:


       ================================================================================
       An HTTP request has been made that VCR does not know how to handle:
         POST https://api.stripe.com/v1/customers

       VCR is currently using the following cassette:
         - /Users/app/spec/data/Subscription/_change_plan_to/stripe_customer_subscription_plan_/name/.json
         - :record => :once
         - :match_requests_on => [:method, :uri, :body]

       Under the current configuration VCR can not find a suitable HTTP interaction
       to replay and is prevented from recording new requests. There are a few ways
       you can deal with this:

         * If you're surprised VCR is raising this error
           and want insight about how VCR attempted to handle the request,
           you can use the debug_logger configuration option to log more details [1].
         * You can use the :new_episodes record mode to allow VCR to
           record this new request to the existing cassette [2].
         * If you want VCR to ignore this request (and others like it), you can
           set an `ignore_request` callback [3].
         * The current record mode (:once) does not allow new requests to be recorded
           to a previously recorded cassette. You can delete the cassette file and re-run
           your tests to allow the cassette to be recorded with this request [4].
         * The cassette contains 109 HTTP interactions that have not been
           played back. If your request is non-deterministic, you may need to
           change your :match_requests_on cassette option to be more lenient
           or use a custom request matcher to allow it to match [5].

       [1] https://www.relishapp.com/vcr/vcr/v/2-5-0/docs/configuration/debug-logging
       [2] https://www.relishapp.com/vcr/vcr/v/2-5-0/docs/record-modes/new-episodes
       [3] https://www.relishapp.com/vcr/vcr/v/2-5-0/docs/configuration/ignore-request
       [4] https://www.relishapp.com/vcr/vcr/v/2-5-0/docs/record-modes/once
       [5] https://www.relishapp.com/vcr/vcr/v/2-5-0/docs/request-matching
       ================================================================================
     # ./app/models/subscription.rb:83:in `create_stripe_customer'
     # ./spec/models/subscription_spec.rb:68:in `block (3 levels) in <top (required)>'
     # -e:1:in `<main>'

I figured out some more. The specs only break when I add a new spec to the stack. How come they break when more things are added to the suite?

Ecl1pse Armageddon
  • 393
  • 4
  • 18

1 Answers1

5

The behavior you are seeing suggests that the one of the attributes used to match requests is non-deterministic and changes each time you run your tests. You mention using the match_requests_on: [:method, :uri, :body] option -- I'm guessing it's the body. Bear in mind that VCR's built-in body matcher does a direct body_string == body_string comparison, and it's easy to have situations where the bodies are semantically equivalent but aren't the same string. For example, a JSON string like {"a": 1", "b": 2} vs {"b": 2, "a": 1}.

My suggestion is to not match on body at all: it's there if you need it in certain situations, but if you match more leniently, VCR usually works fine, since it records the HTTP interactions in the order they originally occur, and then, during playback, it plays back the first unused matched interaction -- which, if your test makes the requests in the same order it originally did, will cause the correct response to be played back for each request.

To get more insight into exactly what's happening, you can use the debug logger option, which will give you detailed output as it is trying to match, to show why it is doing what it is doing.

Myron Marston
  • 21,452
  • 5
  • 64
  • 63
  • Thank you so much for your insightful reply. If I remove body, then the specs don't pass... any suggestions on how to handle that? I'll turn debugger on and paste what I see. – Ecl1pse Armageddon Jul 24 '13 at 21:52
  • Here is the error I get: https://gist.github.com/anonymous/0661a0e821ae907de0e3 and spec: https://gist.github.com/anonymous/588697967840ee97317f – Ecl1pse Armageddon Jul 24 '13 at 22:00
  • also I read about the before thing so I tried this and it still fails: https://gist.github.com/anonymous/672bf704fd7ae3aec989 – Ecl1pse Armageddon Jul 24 '13 at 22:07
  • I played around with it more and I still get errors... user doesn't update on email? https://gist.github.com/anonymous/adc73d906da075389f40 – Ecl1pse Armageddon Jul 24 '13 at 22:12
  • I can't troubleshoot your problem remotely, sorry. However, one possibility is if some of the things you are asserting on are non-deterministic. For example, if the user id is different on each test run, but gets recorded in the cassette as a particular value, then the next time on playback, the expected value could be different (due to the non-determinism) but the cassette would playback the recorded value and it could fail. – Myron Marston Jul 25 '13 at 06:19
  • What the test is doing is that it's creating a user. Then a stripe customer which has a "customer token", that customer token is saved to this subscription model. When I grab the subscription model it's saying it has a different customer. It's pulling an "old" one vs the "proper one" - if new espisodes is on this doesn't happen. In this case it's pulling the api data from stripe and figuring out the email. The response from vcr/stripe is an old email and not the email of the actual object in the test. – Ecl1pse Armageddon Jul 25 '13 at 21:25
  • I'm sorry - I really just want to figure this out... "factory user: bob1@example.com." "factory subscription bob1@example.com" "factory subscription customer token after api call: cus_2GTDs43sbCtUN9" "stripe customer by token's email: bob2@example.com" - on stripe test data that customer token goes to bob2example.com but during the factory phase we're referencing bob1@example.com. VCR is returning a different response/old. How do I fix that? – Ecl1pse Armageddon Jul 25 '13 at 21:37
  • 1
    In general, there are 3 strategies that tend to work for these issues: (1) find a way to remove the non-determinism. (2) Insert placeholders into the cassette for the non-deterministic values. That's what this option is for: https://relishapp.com/vcr/vcr/v/2-5-0/docs/configuration/filter-sensitive-data (3) Write a custom VCR request matcher that ignores the non-deterministic parts that do not identify the request. You can also look into using VCR's hooks -- https://relishapp.com/vcr/vcr/v/2-5-0/docs/hooks -- to do some even more complex stuff. – Myron Marston Jul 26 '13 at 06:35