6

I'm loving the BDD approach to development, but I've bumped into a concern with how far to go. This comment from ThoughtWorks most recent Radar gives me pause:

"The advent of behavior-driven design (BDD) testing frameworks like Cucumber, combined with browser automation tools like Selenium, has encouraged widespread use of acceptance testing at the browser level. This unfortunately encouraged doing the bulk of testing where the cost to run the tests is the greatest. Instead, we should test at the appropriate level, as close to the code as possible, so that tests can be run with maximum efficiency. Browser-level tests should be the icing on the cake, supported by acceptance and unit tests executed at appropriate layers."

So here's my scenario (pun intended):

I've got a basic CRUD app with a business requirement to filter the data displayed based on criteria the end user is permitted to select. For ease of discussion let's say it's an app for the power company, and I'm displaying current month-to-date power usage for each customer. The user of this app could narrow the data by selecting a single customer, multiple customers, no customers, or "All customers". The displayed data will change based on what they select.

For a product stakeholder, these really represent 4 separate scenarios. However from a developer perspective they are practially identical, with the only difference being a parameter passed to the DB. So the question becomes: does the benefit of having each permutation spelled out outweigh the cost of running and maintaining them?

I think BDD principles would probably say "yes" because the communication of the expected behavior is more explicit, but I'm not sure. It certainly has the feel to me of overkill. The scenarios would likely be a lot of copy-paste with minor changes.

My inclination is to cover this functionality with a single scenario that captures the core business value ("when I select a customer I see the power usage data"), and then cover the other permutations with a set of non-UI-based integration tests that validate the service/query returns the correct data. Is this thinking wrong? What's the best answer to making sure these scenarios are covered, without surrendering the benefits of BDD?

Ryan Nelson
  • 4,466
  • 5
  • 29
  • 45

2 Answers2

4

My rule for BDD is that the developers should easily be able to derive the scenarios from whatever behavior is described, and if they can't, illustrate the behavior with scenarios.

BDD is at its most useful when it's describing things that are tricky; either when passing expert knowledge to the developers, or narrowing down behavior until uncertainty is discovered. In a CRUD application with basic filters, behavior is really easy to understand.

What you're describing is probably best covered Dan North's "Ginger Cake" pattern: take the recipe for something else, but with one aspect of behavior different from another (or in this case with one additional, easy-to-understand aspect of behavior). He also uses copy-paste a bit, and I suspect particularly for this kind of behavior.

So, your inclination is perfectly correct. If automating, I'd probably automate just one example and let unit and integration tests cover the rest.

I'd also want to know why this project was being pursued. There has to be something interesting about it, or it wouldn't be happening. Find that, and it's probably a great place to start discussing scenarios.

Lunivore
  • 17,277
  • 4
  • 47
  • 92
  • Thanks, Liz, that's helpful. I think the pitfall we're getting into here is still thinking of the behaviors like testing. We want comprehensive testing, so we want comprehensive scenarios. Gotta shift that mindset. Also, where can I learn a little more about this "Ginger Cake" pattern? Google didn't turn up much for me except some nice cake recipes ... :) – Ryan Nelson Sep 25 '12 at 16:08
  • @ryan-nelson Dan North will hopefully be writing a book at *some* point... other than that, watch his videos. Essentially you take a recipe for some well-understood piece of software (chocolate cake), and replace the bit you need to replace with something else (chocolate replaced by ginger). You don't need to explain the entire recipe for a chef - or a dev - to get it. – Lunivore Sep 25 '12 at 21:34
  • Found one video here where he explains it: http://oredev.org/2011/sessions/patterns-of-effective-delivery – Ryan Nelson Sep 25 '12 at 23:33
  • I don't mean to sound flippant, but that sounds like reusability to me. That has been the holy grail of software development for, what - decades? – Robin Green Sep 26 '12 at 19:24
  • 1
    @robin-green Right, except it's reusability in *analysis*, for which you need to make the assumption that the devs aren't idiots, and that cut-and-paste might be OK. Since software development has been treating the devs like idiots for decades, it may actually be a little bit revolutionary. Go Dan. – Lunivore Sep 27 '12 at 08:59
0

If you're refactoring scenarios, usually by extracting little tables from duplicate scripts, then the maintenance cost likely won't hurt you at all. This doesn't solve the problem of the execution time cost, though.

In such a situation, I'd probably suggest automating both the simplest scenario (no customers) and the most complicated (many customers matching a filter, but not all), and leave the other permutations to more focused programmer tests. I would include the "no customers" case only because people have a tendency to get that one horribly wrong, as in occasionally crashing the program. (You didn't run the seed data script?!)

J. B. Rainsberger
  • 1,193
  • 9
  • 23
  • 1
    "Given you have no clients... what do you mean, you have no clients? You're a client manager! Why are you even trying to use this program? Get on the phone and get some bloody clients!" <- the conversation about what to do when you have no customers, with a real head of client management! – Lunivore Sep 25 '12 at 21:36