4

Say a Chef can make Recipes, and Sous-Chefs can create Recipes that must be approved by a Head Chef.

You want to test that, when a Head Chef views her homepage, she sees Recipes that she herself created. You also want to test that she sees there are Recipes awaiting her approval.

I can think of two ways to do this:

  1. Test that the view contains certain words, like "Your recipes" and "Recipes awaiting your approval"
  2. Add unnecessary attributes to the html elements you're using so that you can check for an element with "id=recipe_1" or "data-for-the-sake-of-testing=1"

I very much dislike both of these approaches.

Why approach #1 sucks

  1. Incredibly brittle tests. Every time you want to make minor updates to the copy, tests will break.
  2. i18n? How will that work with this approach?

There are probably more reasons, but those two are pretty huge.

Why approach #2 sucks

How annoying to have superfluous markup just for the sake of testing! The user should not have an increased download size for the sake of tests.

What is a good approach to this? I'm interested to hear any alternatives at all, in whatever language you think in. I mostly think in Ruby, Test::Unit, Minitest, RSpec, and Cucumber (though my Cuke skills are stale), but if other languages/frameworks have this figured out, I'd love to see what they're doing, too.

chadoh
  • 4,343
  • 6
  • 39
  • 64
  • aren't these links? e.g.`href=ddd?recipeid=1234`? and you can test that all links that should be present are present (and no other links are present), so you need to have an independent lists of recipes (e.g. from previous test cases or a database query).. – Aprillion May 18 '12 at 22:05
  • In this example, they would probably be links. But not necessarily. And URLs can change and are thus also somewhat brittle. (i18n could change the URL, as could restructuring it from "/recipes?id=1" to "/recipes/1".) However, I am more interested in the general case of not-necessarily-link text. How to test that? – chadoh May 18 '12 at 22:20
  • 1
    if the application can decode "/recipes/1" link, it is obviously possible to do it => so you can write a test to do it. and localisation strings can be tracked back to the original ids if you need to.. but i didn't write a book about testing, so i don't know what is the "right" way to do it ;) – Aprillion May 18 '12 at 22:28

4 Answers4

4

Use a page paradigm.

Phrase the steps in as human a way as you can, at the level of capabilities (high level) wherever possible, and use specific examples. For instance, if I'm using Cucumber I might say:

Given the sous-chef has created a recipe for Frog Pie
When the chef looks for recipes to approve
Then the recipe for Frog Pie should be in the list.

Inside the code for these steps, instantiate or find the particular page you're looking for, where the page is an object that represents the capabilities of the page. That page can then have all the things that the user can do with the page - look for recipes, approve recipes, move to another page, etc.

This way, if you need to change the underlying code for the step, you only have to change it in one place, and all the changes for a particular page will be together. Because you've phrased the scenario in terms of capabilities you're delivering, it's unlikely that the scenario will need to change much (unless you discover that your business need different capabilities to the ones you're delivering).

This also works pretty well for window-based apps too, with each widget or module being a particular page.

It's also fine to have extra ids just for testing. Sometimes designers like to use them too.

Lunivore
  • 17,277
  • 4
  • 47
  • 92
4

I see at least two options:

  1. Avoid testing business logic through the UI. Write a "service" or "use case controller" object that returns plain data structures. In other words, you build an API to your system. Your unit test accesses the system through the API. Your UI accesses the system through the same API, but then there should be almost no logic in the view. See http://www.confreaks.com/videos/759-rubymidwest2011-keynote-architecture-the-lost-years or http://www.cleancoders.com/codecast/clean-code-episode-7/show.

  2. Use the "page object" pattern. Write an object that reads in the HTML code that your app produces, parses it, and makes interesting data available through getters. This will do wonders to make your test code clear. Your objection could be that you still have problem #2. In fact I don't think it's really a problem. If you use structural HTML markup, it should be fairly easy to extract the information you need. It will be much easier if you attach an ID to a key element of the page; in your example I would have a div with id="my-recipes" and another div with id="to-be-approved". That should be enough; anything else should be easy to find with xpath or css selectors. Why do you find this objectionable? These IDs will probably be useful for other purposes, such as attaching behaviour with unobtrusive JavaScript or attaching styles with a CSS stylesheet.

xpmatteo
  • 11,156
  • 3
  • 26
  • 25
  • Great answer and links! I'll have to take some time to process it. But to answer your question: Adding superfluous IDs _just for tests_ feels like I'm now writing code for me instead of my customer. My customer now has an increased file-size with no benefit to them. _Iff_ JS or CSS need to hook onto the element, then the (arguably negligible, I get that) increased file-size has benefit to the customer. – chadoh May 30 '12 at 01:08
  • 1
    Thank you chadoh. I understand your concern, but I still feel that having a well tested product *does* benefit your customer :-) – xpmatteo May 30 '12 at 12:13
1

Live with #2, perhaps using brief comments (no i18n issues and not visble to the end user):

<!-- APPROVAL -->

The documentation of simpletest has a nice take on it:

Next chance you get, look at a circuit board, perhaps the motherboard of the computer you are looking at right now. On most boards you will find the odd empty hole, or solder joint with nothing attached or perhaps a pin or socket that has no obvious function. Chances are that some of these are for expansion and variations, but most of the remainder will be for testing.

If a small amount of superfluous markup makes your product more testable and reliable then just live with it!

Community
  • 1
  • 1
Ken
  • 77,016
  • 30
  • 84
  • 101
  • #2 is clearly better than #1, but it still makes me sigh. While it's interesting that EEs have to leave in superflous pins, we in the digital world may be able to come up with a more elegant solution. – chadoh May 19 '12 at 13:11
  • HTML comments are very difficult to use in a unit test. It's much better to use id or class attributes, so that you can use them in xpath queries. – xpmatteo May 26 '12 at 14:16
  • assertPattern('#RECIPES_FOR_APPROVAL#', $output) doesn't seem that hard. It seems clearer to make your test labels explicit, rather than using ids and class attributes that can/should change with your business requirements. – Ken May 27 '12 at 09:28
1

I personally try to do not test views at all. I mean generated markup, since those tests appearing to be much fragile.

Instead, I focus on "data-provider" side, in case of MVC web framework is Controller. As soon as controller is covered by unit tests that check what kind of data controller prepares, you are pretty much safe. The view you create is easy to test by just running the application and see that it looks OK.

Nethertheless, there some approaches of view testing. First one is based on "end-to-end" testing simulation with Selenium Driver. It runs the browsers and initialize requests to you application. Tests are checking the output HTML. Tests logon to "known" edition, it means tests know that current localization is EN, for instance.

You should basically combine the approaches, where it works use HTML markup values ("Recipies") otherwise use HTML elements id's or classes. I would not add any additional markup for the testing.

Another approach you can try is Approval Testing. I believe there is a Ruby driver for that - http://approvaltests.sourceforge.net/. With approvals you render the view and save the HTML as golden master. The test will fail in case of View had changed. It much more easier to implement than Selenium tests.

Alexander Beletsky
  • 19,453
  • 9
  • 63
  • 86