0

I 'd like to call the scenario -Let's say 500 times- in Gherkin test without tableized items. The reason is I 'd like to use randomized variables instead of written by myself. I know how to implement random functionality to the tests but it is called just once.

For example : 

Scenario Outline: I want to test speed with different values
    When I set the speed to <speed>
    And I wait for 5 seconds
    Then it plays at <speed>

    Examples:
    | speed       |
    | 10   |
    | 20   |
    | 30   |
    | 40   |
    | 50   |

import random

speeds = ['10', '20', '30', '40', '50']

def next_speed(): return random.choice(speeds)

If I use random functionality like this, how can I call this scenario 500 times?

Thanks in advance.

allworldfree
  • 55
  • 1
  • 2
  • 7

5 Answers5

0

I've thought about this one on and off for the last couple of years, while I've been using Behave. I use it to drive a functional test environment for critical communciations radios, and also the environment they're in (play / record audio, do some deep learning on the WAV files to confirm content, work with their UI and navigate around to create messages, stuff like that).

I've had the need to try to loop things before, outside of a single line with a supplied table of data.

While I was investigating some of the recesses of the Content object, I delved into the interpreter, and I'm wondering if I can manage to get it to implement something like this:

    When I loop i 5 times
      And I perform this action
      And I perform another action
    Then I should see this result
      And loop back to i

This won't break the gherkin syntax, and provided I can implement something in the loop steps which will rewind the parser somehow, it should run them again. The hard part would be ensuring the results of all the steps are preserved, I suspect: I'd need to delve into the structures used for storing the results, so that the outputs would show all the iterations.

Has anyone else looked into implementing this into Behave via defined steps?

AndyW
  • 71
  • 1
  • 5
0
from __future__ import print_function
import functools
from behave.model import ScenarioOutline


def patch_scenario_with_autoretry(scenario, max_attempts=3):
    """Monkey-patches :func:`~behave.model.Scenario.run()` to auto-retry a
    scenario that fails. The scenario is retried a number of times
    before its failure is accepted.
    This is helpful when the test infrastructure (server/network environment)
    is unreliable (which should be a rare case).
    :param scenario:        Scenario or ScenarioOutline to patch.
    :param max_attempts:    How many times the scenario can be run.
    """
    def scenario_run_with_retries(scenario_run, *args, **kwargs):
        for attempt in range(1, max_attempts+1):
            if not scenario_run(*args, **kwargs):
                if attempt > 1:
                    message = u"AUTO-RETRY SCENARIO PASSED (after {0} attempts)"
                    print(message.format(attempt))
                return False    # -- NOT-FAILED = PASSED
            # -- SCENARIO FAILED:
            if attempt < max_attempts:
                print(u"AUTO-RETRY SCENARIO (attempt {0})".format(attempt))
        message = u"AUTO-RETRY SCENARIO FAILED (after {0} attempts)"
        print(message.format(max_attempts))
        return True

    if isinstance(scenario, ScenarioOutline):
        scenario_outline = scenario
        for scenario in scenario_outline.scenarios:
            scenario_run = scenario.run
            scenario.run = functools.partial(scenario_run_with_retries, scenario_run)
    else:
        scenario_run = scenario.run
        scenario.run = functools.partial(scenario_run_with_retries, scenario_run)

Reference: https://github.com/behave/behave/blob/master/behave/contrib/scenario_autoretry.py

סטנלי גרונן
  • 2,917
  • 23
  • 46
  • 68
Dhvanil Jariwala
  • 1
-1

If you try to use gherkin as a scripting tool, you're gonna have a bad time. There are much better tools for that, like python itself or robot framework. Ask yourself what advantage you expect from your gherkin test. Your gherkin should answer 'why' you are doing something, it should have examples that explain sufficiently the different cases - and preferably only the interesting ones.

Szabo Peter
  • 506
  • 3
  • 7
  • Thanks! You are probably right however, this doesn't fix my problem I've now. Do you have any other suggestion? Thanks in advance. – allworldfree Jul 06 '16 at 16:41
  • You can use background ? – N.. Jul 06 '16 at 17:58
  • Thanks.If I am not wrong, background allows me use my default random function, it is not about calling scenario several times, is it? – allworldfree Jul 07 '16 at 07:02
  • You could use a normal scenario but you would need to adjust your step definitions. This would not execute 5 times, of course. `Scenario: I want to test speed with different values When I set the speed to a random value And I wait for 5 seconds Then it plays at the chosen value ` – Szabo Peter Jul 07 '16 at 14:43
  • You can have repeated random test with a scenario like this: `Scenario: I want to test speed with different values Given 500 random speeds to test When I set the speed and store the playing speed after 5 seconds for each speed Then the playing speeds should match the chosen speeds` The solution will be overcomplicated and you'd be much better off with a simple python unittest. – Szabo Peter Jul 07 '16 at 14:54
  • Ofcourse, you can call the function 500 times, but how can you verify the result on "Then" state of each random result? If you have an error, how do you trigger it? – allworldfree Jul 11 '16 at 13:18
  • You store the individual outcomes in the context, as usual. An exception would be just another outcome. – Szabo Peter Jul 25 '16 at 12:55
-1

You need to add row dynamic each time for test. It will never show in feature file but it needs to add rows.

Following links have few function that will create dynamic rows for you in step definition

http://www.programcreek.com/java-api-examples/index.php?api=gherkin.formatter.model.DataTableRow

or call step definition from step definition

http://www.specflow.org/documentation/Calling-Steps-from-Step-Definitions/

N..
  • 906
  • 7
  • 23
  • The OP is asking about [Behave](https://github.com/behave/behave), which is written entirely in Python. The links you provide are to some other software, written in Java. Whatever it is possible to do in those softwares is not necessarily possible with Behave. (Conversely, there may be things Behave does that other tools do not.) – Louis Jul 07 '16 at 16:50
  • I agree with Louis. I just checked at weekend and it looks it is not possible to implement. – allworldfree Jul 11 '16 at 13:20
-1

Hopefully I understand your question :)

What about this:

Change the step: "When I set the speed to speed" to

When I set the speed to {speed} so that it takes an argument.

In your feature: When I test the speed 500 times And in that step: When I test the speed 500 times you:

==>create a for loop 500 times:

=====>choose a random speed

=====>execute the other steps with context.execute_steps and format(speed)

You'll have to work around this a bit because it takes unicodes, not integers.

==> However, one might agree with Szabo Peter that it is a little awkward to use gherkin/python-behave for this :). It kind of messes up the purpose. Also, even in line of my thinking here, it might be done more elegantly.

You can find some nice stuff here: https://jenisys.github.io/behave.example/tutorials/tutorial08.html Cheerz

So edit after comment: (after editing and writing this example it looks even more silly then I thought it would, so yeah: don't use behave for this. Example:

Feature: test feature Scenario: test scenario given I open the app when I test the app 500 times at random speed then the console says it is done

steps:

@given(u'I open the app')

=>def I_open_the_app(context):

==>#code to open the app

@when(u'I test the app 500 times at random speed')

=>def I_test_the_app_500_times_at_random_speed(context):

==>for times in range(1,500):

===>random_speed = random.randint(min_speed,max_speed)

===>context.execute_steps(u'''when I play at {speed}'''.format(speed=str(random_speed))

@when(u'I play at {speed}')

=>def I_play_at(context,speed)

==>play_at_speed(int(speed))

@then(u'the console says it is done')

=>def the_console_says_it_is_done ==>print('it is done')

Chai
  • 1,796
  • 2
  • 18
  • 31