Why isn't Cucumber considered as a testing tool?

Question

I'm new to Cucumber, and I'm trying to understand the tool. While reading the documentation, I found that it is defined shortly as "a tool that supports BDD":

Cucumber is a tool that supports Behaviour-Driven Development(BDD).

Also it is described as a "validation tool":

Cucumber reads executable specifications written in plain text and validates that the software does what those specifications say.

In the other side, I noticed the excessive use of the word "test" on the 10-minute tutorial.

AFAIK, what does this tool is agile testing, since it is used massively in e2e testing (test basis = Gherkin feature specs + step definitions). However, the blog says something different:

Finally, remember that Cucumber is not a testing tool. It is a tool for capturing common understanding on how a system should work. A tool that allows you, but doesn't require you, to automate the verification of the behaviour of your system if you find it useful.

Now if this tool is not really about testing, then what use is it intended for?

Answer 1

TL;DR

Cucumber is a BDD framework. Its main components are:

1. Gherkin: a ubiquitous language used as communication tool than anything, and can be used as a springboard for collaboration. It helps manage the expectations of the business, and if everyone can see the changes that you are making in a digestible format, they'll hopefully get less frustrated with the development teams, but also you can use it to speed up the reaction time of your team if there are bugs, by writing the tests that cucumber wraps with the mindset that someone is going to have to come back and debug it at some point.
2. CLI implementation (or the CLI): a test runner based on Gherkin. It is developed by volunteers who are all donating part of their spare time. Every implementation is specific to a programming language supporting a production code that is ready for test. It is considered as the concrete tool / utility.

The Long Version

The intended use of Gherkin as a communication tool, describing interactions with a system from multiple perspectives (or actors), and it just so happens that you can integrate it with test frameworks, which aids in ensuring that the system in place correctly handles those interactions.

Most commonly, this is from a users perspective:

Given John has logged in
When he receives a message from Brenda
Then he should be able to click through to his message thread with Brenda via the notification

But it can also be from a component/pages perspective too:

Given the customer history table is displayed
And there have been changes to the customer table for that user since the page was first loaded
When it receives a click to refresh the data
Then the new changes should be displayed

It's all about describing the behaviours, and allowing the business and developers to collaborate freely, while breaking down the language barriers that usually end up plaguing communication, and generally making both sides frustrated at each other because of a lack of mutual understanding of an issue

This is where the "fun" begins _{- Anakin, Ep III}

You could use these files to create an environment of "living documentation" throughout your development team (and if successful, the wider business), and in theory - worded and displayed correctly, it would be an incredible boon for customer service workers, who would more easily be able to keep up with changes, and would have extremely well described help documentation - without any real additional effort, but this isn't something that I've seen much in the wild. I've written a script at work that does this by converting the features into markdown, and alongside various other markdown tools (mermaid for graphs, tsdoc-plugin-markdown to generate the API docs, and various extensions for my chosen HTML converter, docsify) I've managed to generate something that isn't hard to navigate and open up communication between teams that previously found it harder to communicate their issues to the dev team (most people know a little markdown these days, even if it has to be described as "the characters you type in reddit threads and youtube comments to make text bold and italics, etc" for people to get what it is, but it means everyone can contribute to it)

It is an extremely useful tool when it comes to debugging tests, especially when used with the screenplay pattern (less so with the standard page object model, because of the lack of additional context that the pom provides, but it's still useful), as everything is described in a way that breeds replication of the issue from a users or components perspective if it fails.

I've paired it with flow charts, where I draw out the user interactions, pinning the features to it and being able to see in a more visual way where users will be able to do something that we might not have planned for, or even figure out some garish scenario that we somehow missed.

The Long Version Longer

My examples here will mostly in javascript, as we've been developing in a node environment, but if you wanted to create your own versions, it shouldn't be too different.

The Docs

Essentially, this is bit is just for displaying the feature files in a way that is easily digestible by the business (I have plans to integrate test reports into this too, and give the ability to switch branches and such)

First, you want to get a simple array of all of the files in your features folder, and pick out the ones with ".feature" on the end.

Essentially, you just need to flatten an ls here (this can be improved, but we have a requirement to use the LTS version of node, rather than the latest version in general)

const fs = require('fs');
const path = require('path');
const walkSync = (d) => fs.statSync(d).isDirectory() ? fs.readdirSync(d).map(f => walkSync(path.join(d, f))) : d;

const flatten = (arr, result = []) => {
  if (!Array.isArray(arr)){
    return [...result, arr];
  }
  arr.forEach((a) => {
    result = flatten(a, result)
  })
  return result
}

function features (folder) {
  const allFiles = flatten(walkSync(path.relative(process.cwd(), folder)))
  let convertible = []
  for (let file of allFiles) {
    if (file.match(/.feature$/)) {
      convertible.push(file)
    }
  }
  return convertible
}

...

Going through all of those files with a Gherkin parser to pull out your scenarios requires some set up, although it's pretty simple to do, as Gherkin has an extremely well defined structure and known keywords.

There can be a lot of self referencing, as when you boil it down to the basics, a lot of cucumber is built on well defined components. For example, you could describe a scenario as a background that can have a description, tags and a name:

class Convert {

   ...

   static background (background) {
    return {
      cuke: `${background.keyword.trim()}:`,
      steps: this.steps(background.steps),
      location: this.location(background.location),
      keyword: background.keyword
    }
  }

  static scenario (scenario) {
    return {
      ...this.background(scenario),
      tags: this.tags(scenario.tags),
      cuke: `${scenario.keyword.trim()}: ${scenario.name}\n`,
      description: `${scenario.description.replace(/(?!^\s+[>].*$)(^.*$)/gm, "$1<br>").trim()}`,
      examples: this.examples(scenario.examples)
    }
  }

  ...
}

You can flesh it out fully to write to either a single file, or output a few markdown files (making sure to reference them in a menu file)

Flowcharts

Flow charts make it easier to help visualise an issue, and there are a few tools that use markdown to help generate them like this:

In the back, it'll end up looking like this:

### Login

Customers should be able to log into their account, as long as they have registered.

...


```mermaid
 graph TD
        navigateToLogin["Navigate to Login"] -->logIn{"Login"}
        logIn -->validCredentials["Valid<br>Credentials"]
        logIn -->invalidCredentials{"Invalid<br>Credentials"}
        invalidCredentials -->blankPass["Blank Password"]
        invalidCredentials -->wrongPass["Wrong Password"]
        invalidCredentials -->blankEmail["Blank Email"]
        invalidCredentials -->wrongEmail["Wrong Email"]
        ...

        click blankPass "/#/areas/login/scenario-blank-password" "View Scenario"
        ...
 ```

It's essentially just a really quick way to visualise issues, and links us to the correct places in the documentation to find an answer. The tool draws out the flowchart, you just have to make the connections between key concepts or ideas on the page (i.e. a new customer gets a different start screen)

Screenplay Pattern, Serenity and Debugging

I think all that really needs to be said here is that when you run a test, this is your output:

✓ Correct information on the billing page
    ✓ Given Valerie has logged into her account
        ✓ Valerie attempts to log in
            ✓ Valerie visits the login page
                ✓ Valerie navigates to '/login'
            ✓ Valerie waits up to 5s until the email field does become visible
            ✓ Valerie enters 'thisisclearlyafakeemail@somemailprovider.com' into the email field
            ✓ Valerie enters 'P@ssword!231' into the password field
            ✓ Valerie clicks on the login button
            ✓ Valerie waits for 1s

It will break down any part of the test into descriptions, which means if the CSS changes, we won't be searching for something that no longer exists, and even someone new to debugging that area of the site will be able to pick up from a test failure.

Communication

I think all of that should show how communication can be improved in a more general sense. It's all about making sure that the projects are accessible to as many people who could input something valuable (which should be everyone in your business)