2

I have setup cucumber to handle both service (e.g. web service API) level testing and UI (e.g. browser DOM) level testing.

I do this using tags. I tag each API level scenario with the @api tag. In my @Before scenario hook is an if statement which checks if scenario.getSourceTagNames() contains @api. If @api is not detected then driver is initialized and a browser instance is created.

Since my tests are using the PageObject paradigm the step definitions are shielded from each page's HTML details. That lead me to wonder if I could use dependency injection to control which of two sets of PageObject classes get loaded. The first set would bypass the Selenium browser and directly hit server endpoints to obtain values to be processed and sent back to the step definitions. The second set would load the selenium PageObject classes and go through the browser to satisfy step definition calls. Clearly the first set would execute in less (elapsed) time than the second.

So I began an internet search to see what others have done. I ran across this blog post by Aslak Hellesøy. Aslak has this to say:

*"Cucumber power users who understand the benefits of the test pyramid will prefer to run most scenarios against the middle layer (without going through a UI). Then they'll pick a small subset of the same scenarios to run through the UI and run those separately.

If you keep all the automation logic behind an AutomationApi interface you can provide two different implementations - one that talks directly to the domain model, and another one that uses Selenium WebDriver or another UI automation library."*

Given a Jenkins CI server, step 1 is build the software, step 2 is run unit tests, step 3 is run the domain (Web server API) tests and step 4 is run the UI (e.g. web) tests. Thus the testing pyramid is realized.

Has anybody implemented Aslak's idea? If yes, was there actual overlap between scenarios? That is, did you run the same scenarios both with an API implementation of PageObjects (say in your dev environment) and also with an HTML (Selenium) PageObject implementation (say on the CI)? Or were the test scenarios exclusively API xor web?

MikeJRamsey56
  • 2,779
  • 1
  • 20
  • 34
  • You should be able to answer all these questions by reading on the purpose of end to end testing, integration testing and unit testing. The subject it too large to provide a straight answer. – Florent B. Apr 13 '16 at 18:10
  • Aslak isn't talking about two sets of tests using the same page objects, IMO. He's talking about one set of behaviors (scenarios), two different interfaces (maybe one has page objects for selenium, other using rest or other direct interaction with a servers mock services). The more stable interface gets the larger set of tests, while the less stable (GUI) interface gets the smaller set of tests. – Dave McNulla Apr 13 '16 at 18:18
  • @Florent Thank you. I do know quite a bit about the topics that you mention. I just wanted to check with folks before I dove down the rabbit hole. – MikeJRamsey56 Apr 13 '16 at 19:18
  • @DaveMcNulla Yes, Aslak was talking about replacing step definition files one for one and I can see that as being pretty straight forward. The idea buzzing around in my head is that I could keep the same step definition files but replace the PageObject files. One set of PageObject classes would directly invoke the end point(s) to set or get values called for by the step definition files. The other would use Selenium (or Waitr, I guess) and work through the browser to set or get those same values. I suppose that I am just going to have to invest the times to see if it pans out or not. – MikeJRamsey56 Apr 13 '16 at 19:29
  • I meant same scenarios, different step definition files. Only pages have page objects, so I wouldn't expect them on rest services. Maybe service objects. – Dave McNulla Apr 13 '16 at 20:29
  • @DaveMcNulla Sorry, I am not being clear. I realize that PageObjects are intended for pages. But pages are invoking end points (mostly) to get and set values. So if I had a PageObject to start, then I wanted to replace the PageObject class with a class that makes calls to web server end points so that the same test could run sans browser if the second class file was linked in rather than the first. I am coming at the problem from the other direction; PageObject first, page object with API calls second. – MikeJRamsey56 Apr 13 '16 at 20:40
  • The reason I am interested in retrofitting PageObject back into vehicles for directly invoking the APIs is time. It takes too long to run a lot of UI tests. Switching to the end point version of the PageObjects could speed things up for mid-day CI builds. Just an idea. – MikeJRamsey56 Apr 13 '16 at 22:07
  • @MikeJRamsey56 Can you please provide info in case you're able to use Cucumber + Test Pyramid successfully ? Any github repo to showcase it will be of great help. Thanks in advance. – vikramvi Jul 07 '16 at 14:44
  • 1
    @vikramvi The testing pyramid is a concept. The base of the pyramid consists of unit tests. The middle tier is services and the top tier is UI. I keep a starter project that supports the middle (rest-assured) layer and top layer (selenium) both of which use cucumber-jvm. You can find it [here](https://github.com/mikejramsey56/cuc_jav_webdriver). – MikeJRamsey56 Jul 22 '16 at 01:38
  • @MikeJRamsey56 Thanks for input and link to your project. In my new company I'm planning to follow concepts of BDD as strictly as possible. – vikramvi Jul 22 '16 at 08:47

0 Answers0