1

Background: I have multiple json schemas referring large same objects. These objects are moved to a subdirectory. In the example below, the following dependencies appear:

  1. main_schema => positive_integer
  2. main_schema => date
  3. date => positive_integer
  4. date => month

The jsonschema library fails to resolve only the last dependency, processing all other fine.

Project Tree

/
+-- code.py
+-- schemas /
|   +-- dependencies /
|   |   +-- date.json
|   |   +-- month.json
|   |   +-- positive_integer.json
|   +-- main_schema.json

code.py:

from jsonschema import validate import json

contact = \
{
    "name": "William Johns",
    "age": 25,
    "birthDate": { "month": "apr", "day": 15 }
}

def main():
    schema = json.load(open("schemas/main_schema.json"))
    validate(contact, schema)

if (__name__ == '__main__'):
    main()

JSON Schemas

main_schema json

{
    "title": "MainSchema",

    "properties":
    {
        "name":      { "type": "string" },
        "age":       { "$ref": "file:schemas/dependencies/positive_integer.json" },
        "birthDate": { "$ref": "file:schemas/dependencies/date.json" }
    },
    "additionalProperties": false
}

date.json

{
    "title": "date",
    "type": "object",

    "properties":
    {
        "month": { "$ref": "file:month.json" },
        "day":   { "$ref": "file:positive_integer.json" }
    },
    "additionalProperties": false
}

month.json:

{
    "title": "month",
    "type": "string",
    "enum": [ "jan", "feb", "mar", "apr" ]
}

positive_integer.json:

{
    "title": "positiveInteger",
    "type": "integer",
    "minimum": 1
}

Problem

When I run this, program fails with the stacktrace:

"C:\Program Files (x86)\Python\3.6.0\python.exe" D:/Code/python/test/json_schema_testing/validator.py
Traceback (most recent call last):
  File "C:\Program Files (x86)\Python\3.6.0\lib\site-packages\jsonschema\validators.py", line 380, in resolve_from_url
    document = self.store[url]
  File "C:\Program Files (x86)\Python\3.6.0\lib\site-packages\jsonschema\_utils.py", line 23, in __getitem__
    return self.store[self.normalize(uri)]
KeyError: 'file:///schemas/dependencies/month.json'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files (x86)\Python\3.6.0\lib\urllib\request.py", line 1474, in open_local_file
    stats = os.stat(localfile)
FileNotFoundError: [WinError 3] Системе не удается найти указанный путь: '\\schemas\\dependencies\\month.json'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files (x86)\Python\3.6.0\lib\site-packages\jsonschema\validators.py", line 383, in resolve_from_url
    document = self.resolve_remote(url)
  File "C:\Program Files (x86)\Python\3.6.0\lib\site-packages\jsonschema\validators.py", line 474, in resolve_remote
    result = json.loads(urlopen(uri).read().decode("utf-8"))
  File "C:\Program Files (x86)\Python\3.6.0\lib\urllib\request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Program Files (x86)\Python\3.6.0\lib\urllib\request.py", line 526, in open
    response = self._open(req, data)
  File "C:\Program Files (x86)\Python\3.6.0\lib\urllib\request.py", line 544, in _open
    '_open', req)
  File "C:\Program Files (x86)\Python\3.6.0\lib\urllib\request.py", line 504, in _call_chain
    result = func(*args)
  File "C:\Program Files (x86)\Python\3.6.0\lib\urllib\request.py", line 1452, in file_open
    return self.open_local_file(req)
  File "C:\Program Files (x86)\Python\3.6.0\lib\urllib\request.py", line 1492, in open_local_file
    raise URLError(exp)
urllib.error.URLError: <urlopen error [WinError 3] Системе не удается найти указанный путь: '\\schemas\\dependencies\\month.json'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:/Code/python/test/json_schema_testing/validator.py", line 16, in <module>
    main()
  File "D:/Code/python/test/json_schema_testing/validator.py", line 13, in main
    validate(contact, schema)
  File "C:\Program Files (x86)\Python\3.6.0\lib\site-packages\jsonschema\validators.py", line 541, in validate
    cls(schema, *args, **kwargs).validate(instance)
  File "C:\Program Files (x86)\Python\3.6.0\lib\site-packages\jsonschema\validators.py", line 129, in validate
    for error in self.iter_errors(*args, **kwargs):
  File "C:\Program Files (x86)\Python\3.6.0\lib\site-packages\jsonschema\validators.py", line 105, in iter_errors
    for error in errors:
  File "C:\Program Files (x86)\Python\3.6.0\lib\site-packages\jsonschema\_validators.py", line 304, in properties_draft4
    schema_path=property,
  File "C:\Program Files (x86)\Python\3.6.0\lib\site-packages\jsonschema\validators.py", line 121, in descend
    for error in self.iter_errors(instance, schema):
  File "C:\Program Files (x86)\Python\3.6.0\lib\site-packages\jsonschema\validators.py", line 105, in iter_errors
    for error in errors:
  File "C:\Program Files (x86)\Python\3.6.0\lib\site-packages\jsonschema\_validators.py", line 216, in ref
    for error in validator.descend(instance, resolved):
  File "C:\Program Files (x86)\Python\3.6.0\lib\site-packages\jsonschema\validators.py", line 121, in descend
    for error in self.iter_errors(instance, schema):
  File "C:\Program Files (x86)\Python\3.6.0\lib\site-packages\jsonschema\validators.py", line 105, in iter_errors
    for error in errors:
  File "C:\Program Files (x86)\Python\3.6.0\lib\site-packages\jsonschema\_validators.py", line 304, in properties_draft4
    schema_path=property,
  File "C:\Program Files (x86)\Python\3.6.0\lib\site-packages\jsonschema\validators.py", line 121, in descend
    for error in self.iter_errors(instance, schema):
  File "C:\Program Files (x86)\Python\3.6.0\lib\site-packages\jsonschema\validators.py", line 105, in iter_errors
    for error in errors:
  File "C:\Program Files (x86)\Python\3.6.0\lib\site-packages\jsonschema\_validators.py", line 212, in ref
    scope, resolved = validator.resolver.resolve(ref)
  File "C:\Program Files (x86)\Python\3.6.0\lib\site-packages\jsonschema\validators.py", line 375, in resolve
    return url, self._remote_cache(url)
  File "C:\Program Files (x86)\Python\3.6.0\lib\site-packages\jsonschema\validators.py", line 385, in resolve_from_url
    raise RefResolutionError(exc)
jsonschema.exceptions.RefResolutionError: <urlopen error [WinError 3] Системе не удается найти указанный путь: '\\schemas\\dependencies\\month.json'>
(eng: System could not find such file: ...)

Process finished with exit code 1

As I investigated, the 'grandchild' dependencies could be resolved only if they were preloaded earlier. So, if remove the "date => month" dependency, or forcibly "preload" it from the rool level, all would work fine.

Workaround

Modify main_schema.json to be something like this:

{
    "title": "MainSchema",

    "not":
    {
        "comment": "This is preload of 'grandchild' dependencies. It is required due to the jsonschema lib issues.",
        "anyOf":
        [
            { "$ref": "file:schemas/dependencies/month.json" },
            { "$ref": "file:schemas/dependencies/date.json" },
            { "$ref": "file:schemas/dependencies/positive_integer.json" }
        ]
    },


    "properties":
    {
        "name":      { "type": "string" },
        "age":       { "$ref": "file:schemas/dependencies/positive_integer.json" },
        "birthDate": { "$ref": "file:schemas/dependencies/date.json" }
    },
    "additionalProperties": false
}

If do so, validation would be successful. However, I really do not like this workaround. If you have any ideas how to fix it, please, tell me.

System Info:

  • Windows 7
  • Python 3.6.0
  • jsonschema 2.6.0
Peter Zaitcev
  • 316
  • 1
  • 14
  • you're missing a `,` in the `date.json` is it just a typo? (you're also missing the starting `{` and the ending `}` but I am assuming that it's a part of a larger json but the `,`) – iam.Carrot Apr 14 '18 at 20:21
  • @iam-carrot No, I've lost heading "month.json" (or it was eaten). Thank you! – Peter Zaitcev Apr 14 '18 at 20:27
  • can you put in your complete json from the file as a single piece? It'll a lot more easy to understand, you're loading a single file from `main_schema.json`, where do the date.json and month.json come from? – iam.Carrot Apr 14 '18 at 20:27
  • No. I have 5 JSON Schemas of ~100 lines each and each referring to the 3 to 5 same dependencies of ~20 lines. Originally, each of these files contained all these dependencies, but today I was trying to split it. – Peter Zaitcev Apr 14 '18 at 20:31
  • you can also start to narrow down the error by first passing in the complete path instead of `\\schemas\\dependencies\\month.json`. See if it works, since that's the one which started the exception. If you do want to use relative paths I suggest you use the `r` prefix `r"schemas\dependencies\month.json"` – iam.Carrot Apr 14 '18 at 20:32
  • I tried it before posting the issue. I've spent about ~3h to investigate how it's _actually_ works – Peter Zaitcev Apr 14 '18 at 20:34
  • alright then, let's see more of your code then because it's too less to figure out something – iam.Carrot Apr 14 '18 at 20:35
  • Just try to run code I've posted. Should be enough. It is a shorter version of my current project. – Peter Zaitcev Apr 14 '18 at 20:43
  • can you replace `validate(contact, schema)` with `validate(schema)` and tell if it starts to work? – iam.Carrot Apr 14 '18 at 20:57
  • This is not a valid call. `validate` requires 2 arguments at least – Peter Zaitcev Apr 14 '18 at 21:24
  • I suggest you clean up your question it's way too confusing. – iam.Carrot Apr 14 '18 at 21:34
  • The behaviour of a URI with a "file:" protocol is not defined, so this issue is related to the specific library. Have you opened an issue on the libraries github? I'll ping the author to come take a look for you. – Relequestual Apr 16 '18 at 09:12
  • @Relequestual, [issue on GitHub](https://github.com/Julian/jsonschema/issues/398) – Peter Zaitcev Apr 17 '18 at 22:12
  • Why does main_schema.json have a URI reference containing `file:` and `schema/`? Just supply a URI Reference relative to the file's location, which should be `dependencies/positive_integer.json`. – awwright Apr 24 '18 at 07:02
  • @awwright This also does not work. I've tried it before posting this question. – Peter Zaitcev Apr 24 '18 at 16:48

0 Answers0