1

I've started a new role in my life. I was a front end web developer, but I've now been moved to testing web software, or more so, automating the testing of the software. I believe I am to pursue a BDD (Behavior Driven Development) methodology. I am fairly lost as to what to use, and how to piece it together.

The code that is being used/written is in Java to write a web interface for the application to test. I have documentation of the tests to run, but I've been curious how to go about automating it.

I've been directed to Cucumber as one of the "languages" to help with the automation. I have done some research and come across a web site for a synopsis of BDD Tools/Frame works, 8 Best Behavior Driven Development (BDD) Tools and Testing Frameworks. This helped a little but then I got a little confused of how to implement it. It seems that Selenium is a common denominator in a lot of the BDD frameworks for testing a GUI, but it still doesn't seem to help describe what to do.

I then came across the term Functional Testing tool, and I think that confused me even more. Do they all test a GUI?

I think the one that looked like it was all one package was SmartBear TestComplete, and then there is, what seems to be, another similar application by SmartBear called, SmartBear TestLeft, but I think I saw that they still used Cucumber for BDDing it. There a few others that looked like they might work as well, but I guess the other question is what's the cheapest route?

I guess the biggest problem I have is how to make these tests more dynamic, as the UI/browser dimensions can easily change from system to system, and how do I go about writing automation that can handle this, and tie into a BDD methodology?

Does anyone have any suggestions here? Does anybody out there do this?

Thanks in advance.

Matthew Dewell
  • 563
  • 7
  • 30
  • 2
    Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, [describe the problem](https://meta.stackoverflow.com/questions/254393/what-exactly-is-a-recommendation-question) and what has been done so far to solve it. – undetected Selenium Feb 15 '18 at 18:51
  • @DebanjanB Where should I ask a question like this? Are there other developer environments/sites that this can be done? – Matthew Dewell Feb 15 '18 at 19:02
  • Questions about general computing hardware and software are off-topic for Stack Overflow unless they directly involve tools used primarily for programming. You may be able to get help on [Super User](https://superuser.com/tour). – undetected Selenium Feb 15 '18 at 19:04
  • @DebanjanB Thanks. I will try there. Sorry for the inconvenience. – Matthew Dewell Feb 15 '18 at 19:28
  • 2
    I think you would do better to post this on [Software Quality Assurance & Testing](https://sqa.stackexchange.com/questions). It is a good question. – Bill Hileman Feb 15 '18 at 19:55
  • This question is perfectly suited to the BDD Google Group - please come ask it there and we'll be happy to help you! https://groups.google.com/d/forum/behaviordrivendevelopment – Lunivore Feb 19 '18 at 07:08

3 Answers3

2

BDD Architecture

BDD automation typically consists of a few layers:

  1. The natural language steps
  2. The wiring that ties the steps to their definition
  3. The step definitions, which usually access page objects
  4. Page objects, which provide all the capabilities of a page or widget
  5. Automation over the actual code being exercised, often through the GUI.

The wiring between natural language steps and the step definitions is normally done by the BDD tool (Cucumber).

The automation is normally done using the automation tool (Selenium). Sometimes people do skip the GUI, perhaps targeting an API or the MVC layer instead. It depends how complex the functionality in your web page is. If in doubt, give Selenium a try. I've written automation frameworks for desktop apps; the principle's the same regardless.

Keeping it maintainable

To make the steps easy to maintain and change, keep the steps at a fairly high level. This is frequently referred to as "declarative" as opposed to "imperative". For instance, this is too detailed:

When Fred provides his receipt
And his receipt is scanned
And the cashier clicks "Refund to original card"
And the card is inserted...

Think about what the user is trying to achieve:

When Fred gets a refund to his original card

Generally a scenario will have a few Givens or Thens, but typically only one When (unless you have something like users interacting or time passing, where both events are needed to illustrate the behaviour).

Your page objects in this scenario might well be a "RefundPageObject" or perhaps, if that's too large, a "RefundToCardPageObject". This pattern allows multiple scenario steps to access the same capabilities without duplication, which means that if the way the capabilities are exercised changes, you only need to change them in one place.

Different page objects could also be used for different systems.

Getting started

If you're attacking this for the first time, start by getting an empty scenario that just runs and passes without doing anything (make the steps empty). When you've done this, you'll have successfully wired up Cucumber.

Write the production code that would make the scenario run. (This is the other way round from the way you'd normally do it; normally you'd write the scenario code first. I've found this is a good way to get started though.)

When you can run your scenario manually, add the automation directly to the steps (you've only got one scenario at this point). Use your favourite assertion package (JUnit) to get the outcome you're after. You'll probably need to change your code so that you can automate over it easily, by e.g.: giving relevant test ids to elements in your webpage.

Once you've got one scenario running, try to write any subsequent scenarios first; this helps you think about your design and the testability of what you're about to do. When you start adding more scenarios, start extracting that automation out into page objects too.

Once you've got a few scenarios, have a think about how you might want to address different systems. Avoid using lots of "if" statements if you can; those are hard to maintain. Injecting different implementations of page objects is probably better (the frameworks may well support this by now; I haven't used them in a while).

Keep refactoring as you add more scenarios. If the steps are too big, split them up. If the page objects are too big, divide them into widgets. I like to organize my scenarios by user / stakeholder capabilities (normally related to the "when" but sometimes to the "then") then by different contexts.

So to summarize:

  1. Write an empty scenario
  2. Write the code to make that pass manually
  3. Wire up the scenario using your automation tool; it should now run!
  4. Write another scenario, this time writing the automation before the production code
  5. Refactor the automation, moving it out of the steps into page objects
  6. Keep refactoring as you add more scenarios.

Now you've got a fully wired BDD framework, and you're in a good place to keep going while making it maintainable.

A final hint

Think of this as living documentation, rather than tests. BDD scenarios hardly ever pick up bugs in good teams; anything they catch is usually a code design issue, so address it at that level. It helps people work out what the code does and doesn't do yet, and why it's valuable.

The most important part of BDD is having the conversations about how the code works. If you're automating tests for code that already exists, see if you can find someone to talk to about the complicated bits, at least, and verify your understanding with them. This will also help you to use the right language in the scenarios.

See my post on using BDD with legacy systems for more. There are lots of hints for beginners on that blog too.

Lunivore
  • 17,277
  • 4
  • 47
  • 92
  • 1
    Thank you so much. I'm getting a good idea of what needs to go on now. I keep getting Cucumber as the way to go for using a BDD. As well, for doing the GUI testing Selenium is the one mentioned the most, and it tying into Cucumber, to perform BDD with the GUI. I still have not gotten to run a Cucumber/Selenium test on our code, but I have done several tutorials that gives me a good idea. Thanks again. – Matthew Dewell Feb 28 '18 at 15:42
  • I would take a look at Ginkgo4j. A Java lambda-based BDD framework based on Ruby's RSpec. Very simple to pick up and use and very flexible. – Paul Warren Jun 02 '18 at 06:25
  • @PaulWarren Please make it clear when you are affiliated with a product you're advertising. https://stackoverflow.com/help/behavior – Lunivore Jun 02 '18 at 10:26
  • 1
    Apologies. Yes, indeed, I am the author of the framework Ginkgo4j. Great post btw @Lunivore – Paul Warren Jun 02 '18 at 17:11
  • Thanks, looks like a solid framework too ;) Though it looks like it's probably better suited to class-level BDD (unit tests) than full-stack scenarios; please feel free to show me an example if you think that's not the case. – Lunivore Jun 02 '18 at 19:11
0

Since you feel lost as to where to start, I will hint you about some blogs I have written that talks a bit about your problem.

Some categories that may help you:

This, rather long and old post, might give you hints as well:

http://www.thinkcode.se/blog/2012/11/01/cucumberjvm-not-just-for-testing-guis

Notice that versions are dated, but hopefully it can give some ideas as what too look for.

Thomas Sundberg
  • 4,098
  • 3
  • 18
  • 25
  • Sunderberg Thank you for the advice. I think I have a potential Idea of where I need to go, and please let me know, if you have, or don't, the knowledge to verify this. I think I need a GUI tool, like Selenium, but also a graphical tool. Where I can decipher the simple HTML web page layouts of inputs and such. Yet I need a stronger graphical tool to handle some clicks within dynamic graphics. I came across a few things that might help: AppliTools, LeanFT, SauceLabs, TestComplete, TestLeft. – Matthew Dewell Feb 19 '18 at 14:49
  • These tools are all going to cost, but I may need it, unless you know a different way to go about finding objects, that you know what they are supposed to look like, and a dynamically produced graphic, from software, like Flash/Flex, KeyLines or even Cytoscape. If you have any more intput, it would be greatly appreciated. Thanks Again – Matthew Dewell Feb 19 '18 at 14:50
  • It will have a cost to know if stuff works. Either in time or in tooling. Most likely both. Sometimes it is worth the money to buy a tool. Sometimes free software is a better option. I don't mind paying myself, I pay for my Java dev environment. It saves me so much time that it is worth the cost. – Thomas Sundberg Feb 21 '18 at 07:41
0

I am not an expert on the test automation but I am currently working on this part. So let me share some idea and hope it will help you at the current stage. We have used selenium+cucumber+intellij for testing web application. We have used testcomplete+cucumber+intellij for testing java desktop application.

As to the test of web application, we have provided a test mode in our web application, which allows us to get some useful details of the product and the environment; and also allows us to easily trigger events through clicking the button and inputting text into the test panel under test mode.

I hope these are helpful for you.