In Tavern, how to load a common stage?

Question

I am using https://github.com/taverntesting/tavern but the following is likely really a PyYAML question.

We would like to have a directory of tests where each file matches an API endpoint.

api/v1/test_thing1.tavern.yaml
api/v1/test_thing2.tavern.yaml

and so on. Each YAML document will need a login which could go in a common_stages.yaml at the top of the tree. Most anything I have tried ends up with this error from PyYAML

yaml.scanner.ScannerError: mapping values are not allowed here

common_stages.yaml looks like this:

---
stages:
- name: &login_required
  request:
    url: "{host}/api/v1/login"
    json:
      username: "{username}"
      password: "{password}"
    method: POST
    headers:
      content-type: application/json
  response:
    status_code: 201
    cookies:
      - session
    headers:
      content-type: application/json

and a test file looks like:

---
test_name: Get thing1 list

includes:
  - !include ../../common.yaml

stages:
  - name: Get category list
    request:
      url: "{host}/api/v1/thing1"
      method: GET
    response:
      status_code: 200
      headers:
        content-type: application/json

I have tried adding the common_stages include to the list with common. I have tried including it on the stages line of test_thing1.tavern.yaml. No joy.

All of the Tavern examples show the YAML document as one long file. Which is fine for a demo but not a real world use.

Answer 1

I made a slightly simplified version (since you don't have any aliases in your file, there is no need for the cross file anchor support) of the loader that tavern uses.

And your files load fine, without any error if you !include the correct file. In your question you reference common_stages.yaml, but in your test_thing1.tavern.yaml you !include ../../common.yaml. So most likely your common.yaml has the syntax error, that common_stages.yaml that you present here does not.

Proof that your files load fine:

import os
from pathlib import Path
import yaml

from yaml.reader import Reader
from yaml.scanner import Scanner
from yaml.parser import Parser
from yaml.composer import Composer
from yaml.constructor import SafeConstructor
from yaml.resolver import Resolver

# this class slightly simplified from 
# https://github.com/taverntesting/tavern/blob/master/tavern/util/loader.py
class IncludeLoader(Reader, Scanner, Parser, Composer, Resolver,
        SafeConstructor):
    """YAML Loader with `!include` constructor and which can remember anchors
    between documents"""

    def __init__(self, stream):
        """Initialise Loader."""

        # pylint: disable=non-parent-init-called

        try:
            self._root = os.path.split(stream.name)[0]
        except AttributeError:
            self._root = os.path.curdir
        self.anchors = {}
        Reader.__init__(self, stream)
        Scanner.__init__(self)
        Parser.__init__(self)
        SafeConstructor.__init__(self)
        Resolver.__init__(self)


def construct_include(loader, node):
    """Include file referenced at node."""

    # pylint: disable=protected-access
    filename = os.path.abspath(os.path.join(
        loader._root, loader.construct_scalar(node)
    ))
    extension = os.path.splitext(filename)[1].lstrip('.')

    if extension not in ('yaml', 'yml'):
        raise BadSchemaError("Unknown filetype '{}'".format(filename))

    with open(filename, 'r') as f:
        return yaml.load(f, IncludeLoader)


IncludeLoader.add_constructor("!include", construct_include)

Path('common.yaml').write_text("""\
---
stages:
- name: &login_required
  request:
    url: "{host}/api/v1/login"
    json:
      username: "{username}"
      password: "{password}"
    method: POST
    headers:
      content-type: application/json
  response:
    status_code: 201
    cookies:
      - session
    headers:
      content-type: application/json
""")

t_yaml = 'test_thing1.tavern.yaml'
Path(t_yaml).write_text("""\
---
test_name: Get thing1 list

includes:
  - !include common.yaml

stages:
  - name: Get category list
    request:
      url: "{host}/api/v1/thing1"
      method: GET
    response:
      status_code: 200
      headers:
        content-type: application/json
""")


data = yaml.load(open(t_yaml), Loader=IncludeLoader)

print(data['includes'][0]['stages'][0]['request']['url'])

Output:

{host}/api/v1/login

Answer 2

For those who have the same question.

You can use the TAVERN_INCLUDE environment variable.

In it, you must specify the path to the folders in which Tavern will search for the files specified in "include". The path must be absolute.

Example:

TAVERN_TESTS_HOME = /tests/home
TAVERN_INCLUDE = $TAVERN_TESTS_HOME/common

By default, it looks for files in the same folder as the tests. In "include" you can also specify files from a subfolder.

Example:

!include subfolder/file.yml

Documentaion: https://tavern.readthedocs.io/en/latest/basics.html#including-external-files