2

Can anyone help get me started on how to use RobotFramework to validate json responses via a json-schema?

Ideally, the json-schema is externally referenced via an http request: Example http://api-bl-uk.northeurope.cloudapp.azure.com/api/v1/crm/schemas/contact

Progress so far:

pip install robotframework
pip install robotframework-jsonvalidator
pip install robotframework-jsonschemalibrary
robot .\mytest.robot

Where mytest.robot is:

Library JsonValidator
Library JSONSchemaLibrary schemas
*** Test Cases ***
  My Test Case:
   Validate Json  service.schema.json  {"foo": "bar"}

I have a schema in the subdirectory schemas called service.json

When I run the test I get...

$ robot .\mytest.robot
==============================================================================
Mytest
==============================================================================
My Test Case:                                                         | FAIL |
No keyword with name 'Validate Json' found.
------------------------------------------------------------------------------
Mytest                                                                | FAIL |
1 critical test, 0 passed, 1 failed
1 test total, 0 passed, 1 failed
==============================================================================
Output:  E:\GitLab\customer-api\test\output.xml
Log:     E:\GitLab\customer-api\test\log.html
Report:  E:\GitLab\customer-api\test\report.html

So it seems I'm missing a fairly basic piece of the puzzle:

No keyword with name 'Validate Json' found

UPDATE

The problems of blindly following 'sample code'

The problem was I was missing the *** Settings *** header prior to the Library statements, plus the name of the schema to use was wrong (easy to solve after the header was fixed).

Full example: 

*** Settings ***
Library  JSONSchemaLibrary  schemas

*** Test Cases ***
My Test Case:
    Validate Json  service.json  {"foo": "bar"}

Now... How do I use external referenced schema files? The quest continues!

:)

Guy
  • 9,720
  • 7
  • 38
  • 42
  • Are you certain that the JSONSchemaLibrary has a keyword named "validate json"? Is JSONSchemaLibrary an actual robot framework keyword library, or is it just a python module? – Bryan Oakley Jun 08 '18 at 15:55
  • @BryanOakley - TBH, I'm a total newb at robotframework and python so my answer is, I don't know! I've done some googling but the results are disappointingly sparse. At this time, I'd just be happy to get a 'json schema validation failed' error message back - at least then I know that I've got the modules / libraries all setup right. My test is based on: https://github.com/jstaffans/robotframework-jsonschemalibrary – Guy Jun 08 '18 at 16:13
  • 1
    Try with the import of `JsonValidator` commented - it has a method `_validate_json`, which might cause a conflict with the method/keyword `validate_json` you're trying to call from the other library. Don't know will that fix it, trying a lucky shot after checking the libs sources. – Todor Minakov Jun 08 '18 at 19:15
  • @todor - Thanks. Please see my update to the OP. Although you didn't directly provide the answer, it did lead me to it by another route. :) – Guy Jun 08 '18 at 20:38
  • Case solved, that's the important part :) BTW if I were you, I'd delete the first part of the question (it was a simple syntax typo), thus not diluting it, and allowing anyone knowledgeable in the libraries concentrate on the remaining issue. – Todor Minakov Jun 09 '18 at 03:03
  • For the second question - please provide a sample of your framework's directory structure, plus your cwd when you run the tests. The library gets the schemas directory as given - relative or absolute path, and does some unorthodox (for python) concatenation [of the file name to it](https://github.com/jstaffans/robotframework-jsonschemalibrary/blob/master/JSONSchemaLibrary/__init__.py#L31), which might be the culprit of your issue. – Todor Minakov Jun 09 '18 at 07:47
  • @Todor - Yup, I'm making some progress and will spend some time getting a working example going so this can become a reference piece. (For me, when I forget next time) – Guy Jun 09 '18 at 12:32

1 Answers1

3

I'm not sure if this will work with the library you are using, but I'm using the library jsonschema (https://python-jsonschema.readthedocs.io/).

There are two ways I came up with for using a schema from a file. I'd go with the first.

First way

In your virtualenv, run pip install jsonschema.

Then create a new file, mySchema.json in the same directory as your test case file. Test case file:

*** Settings ***
# For the "Get Binary File" task
Library     OperatingSystem
# For the "validate" task
Library    jsonschema


*** Test Cases ***
Load json schema from file, and validate json
    # Load the file as a string, usually sufficent for most methods, but not validate() below
    ${schema}    Get Binary File    ./mySchema.json
    # Load the string as a binary object, you could then use this like ${schema}[someProperty] if you wanted to
    ${schema}    evaluate    json.loads('''${schema}''')    json
    # Do a simple validation, using the schema, and your json data. Remember ${instance} needs to be a json object, not just some string
    ${instance}    evaluate    json.loads('''{"someField":[1,2,3]}''')    json
    validate    instance=${instance}    schema=${schema}

Second way

In your virtualenv, run pip install jsonschema.

Then create a new file, mySchema.json in the same directory as your test case file. Test case file:

*** Settings ***
# For the "Get Binary File" task
Library     OperatingSystem
# For the "validate" task
Library    jsonschema


*** Test Cases ***
Load json schema from file, and validate
    # Create a schema
    ${schema}    concat
    ... {
    ...   "type": "object",
    ...   "properties": {"$ref": "file:/absolute/path/to/mySchema.json"}
    ... }
    ${schema}    evaluate    json.loads('''${schema}''')    json
    # Do a simple validation, using the schema, and your json data. Remember ${instance} needs to be a json object, not just some string
    ${instance}    evaluate    json.loads('''{"someField":[1,2,3]}''')    json
    validate    instance=${instance}    schema=${schema}

If you want to get the schema file from an external source, have a look at the requests library. Something like:

*** Settings ***
Library     RequestsLibrary

*** Test Cases ***
Test case
    Create Session    yourSession    http://localhost
    ${file}    Get Request    yourSession    /filename
dǝɥɔS ʇoıןןƎ
  • 1,674
  • 5
  • 19
  • 42