0

I am Python programmer, so I'd like to use Tavern for API testing.

It means writing a program (API test) in YAML.

Problem is, when I edit YAML input, it often breaks with error BadSchemaError. Tavern has pykwalify embedded, but error messages are not very helpful.

Often I get error message like this:

=================================================== test session starts ===================================================
platform linux2 -- Python 2.7.5, pytest-4.5.0, py-1.8.0, pluggy-0.12.0
rootdir: /home/*****/tavern
plugins: tavern-0.26.3
collected 0 items / 1 errors

========================================================= ERRORS ==========================================================
_________________________________________ ERROR collecting test_echo.tavern.yaml __________________________________________
/usr/lib/python2.7/site-packages/tavern/testutils/pytesthook/file.py:306: in collect
    raise_from(exceptions.BadSchemaError, e)
/usr/lib/python2.7/site-packages/future/utils/__init__.py:456: in raise_from
    raise e
E   BadSchemaError
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================= 1 error in 0.13 seconds ======

Is there a better way to validate Tavern YAML syntax? Are some params to pykwalify/tavern/whatever to make error messages more helpful?

I found Tavern examples and cookbook which gives me some idea about the syntax, but writing my own YAML is hit and miss, and more miss than hit.

Edit: according to pykwalify docs, I could run pykwalify in CLI like:

pykwalify -d data.yaml -s schema.yaml

but where is located the YAML schema for tavern?

Peter M. - stands for Monica
  • 1,102
  • 12
  • 20

3 Answers3

1

I found tests schema file (source: https://github.com/taverntesting/tavern/blob/master/tavern/schemas/tests.schema.yaml) in /usr/local/lib/python3.10/site-packages/tavern/schemas/tests.schema.yaml

So you may run pykwalify as:

pykwalify -d test_hello_world.tavern.yaml -s /usr/local/lib/python3.10/site-packages/tavern/schemas/tests.schema.yaml

Hope this helps somebody

Waseem
  • 11
  • 2
0

Self answer:

Yes, YAML is incredibly picky with whitespace. EXACTLY 2 spaces per indent. My vim was doing 4 spaces.

Once your basic YAML schema is valid, error messages are much more helpful.

Start with simplest examples and run tests often (for every few changed lines).

Peter M. - stands for Monica
  • 1,102
  • 12
  • 20
0

Another self-answer, few months later: Just use pytest and requests library - https://pypi.org/project/requests/2.7.0/

Tavern is OK for one-off tests like different error conditions, but unless you have a hard requirement for non-programmers at least READING you API tests, you will be much happier with real Python. Tavern has almost no code reuse, so it is hard to refactor the code, there is LOTS of copy-paste and little tweak (no procedure calls with parameters), and debugging your code is far from trivial.

requests library with pytest and --pdb (drop to python debugger on error) makes for incredible productive environment to write regression tests: instead of reading docs, you ask the code what the value is, add assert, rerun the test. Using requests, you can implement API test 10 times faster with 10% of the lines, compared to Tavern.

Peter M. - stands for Monica
  • 1,102
  • 12
  • 20