3

Say I have a feature requirement that when a user successfully signs up for a paid account, several things happen:

  1. The user's account is created
  2. The user's credit card is billed, resulting in a new transaction record
  3. A receipt is emailed to the user

Most of the rspec integration or cucumber tests I've seen focus pretty exclusively on what's displayed on screen, e.g. expect(page).to have_text(X) or Then I should see X

How much should an integration test cover here?

Should the integration test check...

  • that an account record was created?
  • that a transaction was created?
  • that the transaction was for the correct amount?
  • that the receipt email was sent?

Why or why not? If not, where/how should these types of side effects be tested?

Dave Schweisguth
  • 36,475
  • 10
  • 98
  • 121
Jared
  • 2,885
  • 2
  • 28
  • 31

1 Answers1

2

If you develop outside in, most of your integration tests will be acceptance tests. These are about what the user sees, so they interact with the system from outside. They don't look in the database or otherwise go behind the scenes. They don't need to: if there was a reason to save something in the database, we'll know if it worked if it showed up on the screen. They do need to test that an email was sent or that an account was created in the third-party credit card processing system, because those are part of what the user sees.

After you've implemented all of your acceptance tests you might look at the system as an engineer and decide that some aspects of how classes in the system interact with one another or with external systems aren't adequately tested. First, decide whether these are telling you about requirements that should really be in the acceptance tests (which is often the case), and, if they are, improve the acceptance tests.

You might still feel the need to write some more integration tests that aren't acceptance tests, and they might need to look directly in the database or whatever. In my experience these are in the minority.

That said, sometimes it makes pragmatic sense for an acceptance test to stop in mid-scenario and just test side effects, because another acceptance test has already described the rest of the scenario. (For example, maybe there are 18 different ways to create an account in your app. Maybe they all result in the same account in the database in mid-scenario.) Similarly, you might want to fake up some database state and start in mid-scenario to avoid duplication; it's common to do this with user login.

After your application implements all of its acceptance tests and has good integration coverage, test the details with unit tests, which do test side effects. But if your acceptance/integration tests already test a given piece of code adequately, it doesn't need unit tests and its side effects might never be directly tested -- which is fine.

Dave Schweisguth
  • 36,475
  • 10
  • 98
  • 121
  • Thanks for the thorough answer. To test a side effect then would you recommend navigating to where it is visible to the user and verify it's there? e.g. after a payment is processed on signup navigate to the payment history page and look for the payment? How do you handle that when it's an external service - say I post to twitter; how does my acceptance test verify that a tweet was indeed posted? The effect of that would be "visible" to the user in production but my test is not going to actually post to twitter - it'll be mocked out. – Jared May 13 '14 at 00:58
  • For side effects internal to the application, exactly what you said. External services are harder and the techniques are diverse. A few examples: Rails has a test mode for mail that doesn't send it but lets you check that it was sent. The most recent credit card processing system I used had a test server that one could run integration tests against. For something like Twitter you could use a test account on the real service. Sometimes there's no good way to do it and you have to stub or mock even in the integration test. – Dave Schweisguth May 13 '14 at 01:08