2

In Cucumber, is it possible to run a Background step for the whole feature? So it doesn't get repeated every scenario?

I am running some tests on a search engine, and I need to pre-seed the search engine with test data. Since this data can be quite long to generate and process (I'm using Elasticsearch and I need to build the indices), I'd rather do this background only once, but only for all tests under the same feature.

Is it possible with Cucumber?

Note that I am using MongoDB, so I don't use transactions but truncation, and I believe I have DatabaseCleaner running automatically after each test, that I suppose I'll have to disable (maybe with an @mention?)

EDIT :

Yes I'm using Cucumber with Ruby steps for Rails

EDIT2 : concrete examples

  • I need to test that my search engine always return relevant results (eg. when searching for "buyers" it should return results with "buyer", "buying", "purchase", etc. (has to do with ES configuration), and other contextual information gets updates correctly (eg in the sidebar

  • I have categories/filters with the number of hits in parenthesis, I must make sure those number gets refreshed as the user plays with filters)

For this I pre-seed the search engine with a dozen of results, and I run all those tests that are based on the same inputs. I often have "example" clauses that just do something slightly different, but based on the same seeding

Cyril Duchon-Doris
  • 12,964
  • 9
  • 77
  • 164
  • Do you also need to remove the data after the scenarios that use it have run? – Dave Schweisguth Sep 02 '16 at 14:38
  • I'd have addressed the part about disabling DatabaseCleaner, but I don't know how you've configured it to work with Cucumber and Mongoid. Do you use cucumber-rails (which sets up DatabaseCleaner) or do you run DatabaseCleaner yourself, perhaps in an Around hook? – Dave Schweisguth Sep 02 '16 at 14:49

2 Answers2

1

Supposing the search data is a meaningful part of the scenario, something that someone reading the feature should know about, I'd put it in a step rather than hide it in a hook. There is no built-in way of doing what you want to do, so you need to make the step idempotent yourself. The simplest way is to use a global.

In features/step_definitions/search_steps.rb:

$search_data_initialized = false

Given /^there is a foo, a bar and a baz$/ do
  # initialize the search data
  $search_data_initialized = false
end

In features/search.feature:

Feature: Search

  Background:
    Given there is a foo, a bar and a baz

  Scenario: User searches for "foo"
  ...
Dave Schweisguth
  • 36,475
  • 10
  • 98
  • 121
1

There are a number of approaches for doing this sort of thing:

  1. Make the background task really fast.

Perhaps in your case you could put the search data outside of your application and then symlink it into the app in your background step? This is a preferred approach.

  1. Use a unit test tool.

Consider if you really get any benefit out of having scenarios to 'test' search. If you don't use a tool that allows you greater control because your tests are being written in a programming language

  1. Hack cucumber to work in a different way

I'm not going to go into this, because my answer is all about looking at the alternatives

For your particular example of testing search there is one more possibility

  1. Don't test at all

Generally search engines are other peoples code that we use. They have thousands of unit tests and tens of thousands of happy customers, so what value do your additional tests bring?

diabolist
  • 3,990
  • 1
  • 11
  • 15
  • Concerning your n°2/4, I'm not testing the search engine itself, but how we configured it. ElasticSearch is very hard to configure, and I must make sure the engine still returns "expected results". For example, that it still handles stemming, english/french correctly, etc. I'm also testing the code around the search engine : map-reduce scripts to provide more info about the search, etc. – Cyril Duchon-Doris Sep 07 '16 at 12:06
  • 1
    Testing your configuration of ElasticSearch seems a very reasonable (though difficult) thing to do. However I don't think Cucumber really is the right tool for this. Its great for ensuring that your search returns results and that you can interact with those results. But not so good for ensuring the results are correct. For that I would use a unit test tool and test your configuration of the engine directly without necessarily going through your UI. – diabolist Sep 08 '16 at 16:47