How to validate a complex nested data structure with Pydantic?

Question

I had complex and nested data structure like below:

{   0: {   0: {'S': 'str1', 'T': 4, 'V': 0x3ff},
           1: {'S': 'str2', 'T': 5, 'V': 0x2ff}},
    1: {   0: {'S': 'str3', 'T': 8, 'V': 0x1ff},
           1: {'S': 'str4', 'T': 7, 'V': 0x0ff}},
......
}

It's a 2D dictionary basically. The innermost dict follows {Str: str, str:int, str:int}, while its outer dict always has integer as the key to index.

Is there a way for Pydantic to validate the data-type and data structure? I mean if someone changes the data with a string as a key to the outer dict, the code should prompt an error. Or if someones tweaks the inner dict with putting 'V' value to a string, a checker needs to complain about it.

I am new to Pydantic, and found it always requires a str-type field for any data... Any ideas?

Are the keys in the inner-most dicts always the same? If so, could you make those a data class? Then you should be able to use `Dict[int, Dict[int, ThatClass]]` — tobias_k, Sep 25 '21 at 15:17
yes, keys in the innermost dicts are always the same. emm... I am thinking how to turn to words into solution — Dave Hwang, Sep 25 '21 at 15:22
I mean, can't you define `class ThatClass(BaseModel): S: str, T: int, V: int` and then use a type hint as above wherever you want to refer to that structure? — tobias_k, Sep 25 '21 at 16:49
that's probably simplest way to do that. a class model would reduce the headache of dict type validation *considerably*. — rv.kvetch, Sep 25 '21 at 17:22
I did not understand type hint until I saw below solution. Anyway, thanks both for your great hints. — Dave Hwang, Sep 25 '21 at 23:59

score 5 · Accepted Answer · answered Sep 25 '21 at 19:45

You could use Dict as custom root type with int as key type (with nested dict). Like so:

from pydantic import BaseModel, StrictInt
from typing import Union, Literal, Dict

sample = {0: {0: {'S': 'str1', 'T': 4, 'V': 0x3ff},
              1: {'S': 'str2', 'T': 5, 'V': 0x2ff}},
          1: {0: {'S': 'str3', 'T': 8, 'V': 0x1ff},
              1: {'S': 'str4', 'T': 7, 'V': 0x0ff}}
          }


# innermost model
class Data(BaseModel):
    S: str
    T: int
    V: int


class Model(BaseModel):
    __root__: Dict[int, Dict[int, Data]]


print(Model.parse_obj(sample))

This the right solution. Appreciate your help. – Dave Hwang Sep 25 '21 at 23:58 — Dave Hwang, Sep 25 '21 at 23:58

How to validate a complex nested data structure with Pydantic?

1 Answers1