1

The problem which I am encountering is related to E2E tests which will run all the time for new app builds (maybe even every few hours on CircleCi). I have ( and will have much more in the future ) features that contain a lot of setups ( necessary the same setup for each scenario to run). For example, before the scenario will run ( many in the feature ) need some users, contents, configuration etc. After the scenario runs probably the best practice is to delete/remove all that users, content etc (or at least after all the scenarios had run for the feature ). I am struggling to understand what is the best practice.

If I add a background then it will run before each scenario, but then I head to remove all that data from the background ( I could add a cleanup function in the last scenario step but that seems bad, correctly if I am wrong). I could add hooks that will clean up after each scenario and keep adding more hooks for new features ( maybe use tags for the scenarios to distinguish for which they should run ).

There are options but it does feel so inefficient... Those tests will be running in a live environment ( not integration or unit tests which are fast, but E2E ). Very often the setup/background will take much more time than one scenario to run and then it will run over and over for each tinny scenario. For example, had to run in e.g. background bunch of endpoints to create users, some content and in many cases ( when we don't have an endpoint for it yet ) I will have to write an automated journey through the UI to add something or change specific settings and then same way add the end delete everything and also through UI change the setting to the state before the feature had run. It feels so slow and inefficient...

The only other thing which comes to my mind ( but will not probably work for all the cases ). Is to create a huge hooks script where I will be adding all the necessary "stuff" before the whole suite run and after the whole thing run I clean the whole stack/instance DB ( or reset to some preset DB snapshot ) to make it state as before the whole suite run.

Please help me to understand what are the best practices in such a cases

Regards

Jakub Haraśny
  • 11
  • 2

1 Answers1

1

Generally with Cuking the idea is that the database is reset after every scenario. This is done by things like:

  • running the scenario in a transaction (which is then rolled back)
  • emptying the database after every scenario

Which you do depends on which flavour of cuke you are using.

The inefficiencies you talk about can be mitigated in a number of ways without compromising the idea that the database should be reset after every scenario. Basically you can think of Cukes as setting up state (Givens) doing something (When) and validating (Thens). Only Whens have to use the UI.

So with Givens you can set up state by either

  • writing directly to the database using factories or fixtures
  • calling services (the same ones your UI controllers use) to create things

The second one is much preferred.

With most of the work being done outside the UI this means that you can get rapid cukes that do complex setup quickly.

This really is the way to go when cuking. Setup everything in Givens (using background when appropriate) without using the UI, then login, then do your When using the UI, and validate the result in the UI with your Thens.

Using this approach my current project has approx 450 scenarios that run in about 5 mins on my Mac mini, and that includes

  • several scenarios that step through UI wizards using a browser (super slow)
  • many scenarios with complex setup of multiple entities

This idea what you have to work around standard practices to make your suite efficient and fast is common and almost always wrong.

You can actually go way faster than I am going (though it takes quite a bit of work)

diabolist
  • 3,990
  • 1
  • 11
  • 15