In JSON created from a pydantic.BaseModel exclude Optional if not set

Question

I want to exclude all the Optional values that are not set when I create JSON. In this example:

from pydantic import BaseModel
from typing import Optional


class Foo(BaseModel):
    x: int
    y: int = 42
    z: Optional[int]


print(Foo(x=3).json())

I get {"x": 3, "y": 42, "z": null}. But I would like to exclude z. Not because its value is None, but because it is Optional and there was no keyword argument for z. In the two cases below I would like to have z in the JSON.

Foo(x=1, z=None)
Foo(x=1, z=77)

If there is any other solution to set z to optional in this sense, I would like to see it.

alex_noname · Accepted Answer · 2020-12-19T06:00:14.817

You could exclude only optional model fields that unset by making of union of model fields that are set and those that are not None.

Pydantic provides the following arguments for exporting method model.dict(...):

exclude_unset: whether fields which were not explicitly set when creating the model should be excluded from the returned dictionary; default False.

exclude_none: whether fields which are equal to None should be excluded from the returned dictionary; default False

To make union of two dicts we can use the expression a = {**b, **c} (values from c overwrites values from b). Note that since Python 3.9 it could be done just as a = b | c.

from pydantic import BaseModel
from typing import Optional
from pydantic.json import pydantic_encoder
import json


class Foo(BaseModel):
    x: int
    y: int = 42
    z: Optional[int]

def exclude_optional_dict(model: BaseModel):
    return {**model.dict(exclude_unset=True), **model.dict(exclude_none=True)}

def exclude_optional_json(model: BaseModel):
    return json.dumps(exclude_optional_dict(model), default=pydantic_encoder)
    


print(exclude_optional_json(Foo(x=3)))  # {"x": 3, "y": 42}
print(exclude_optional_json(Foo(x=3, z=None)))  # {"x": 3, "z": null, "y": 42}
print(exclude_optional_json(Foo(x=3, z=77)))  # {"x": 3, "z": 77, "y": 42}

Update

In order for the approach to work with nested models, we need to do a deep union( or merge) of two dictionaries, like so:

def union(source, destination):
    for key, value in source.items():
        if isinstance(value, dict):
            node = destination.setdefault(key, {})
            union(value, node)
        else:
            destination[key] = value

    return destination

def exclude_optional_dict(model: BaseModel):
    return union(model.dict(exclude_unset=True), model.dict(exclude_none=True))

class Foo(BaseModel):
    x: int
    y: int = 42
    z: Optional[int]

class Bar(BaseModel):
    a: int
    b: int = 52
    c: Optional[int]
    d: Foo


print(exclude_optional_json(Bar(a=4, d=Foo(x=3))))
print(exclude_optional_json(Bar(a=4, c=None, d=Foo(x=3, z=None))))
print(exclude_optional_json(Bar(a=4, c=78, d=Foo(x=3, z=77))))

{"a": 4, "b": 52, "d": {"x": 3, "y": 42}}
{"a": 4, "b": 52, "d": {"x": 3, "y": 42, "z": null}, "c": null}
{"a": 4, "b": 52, "c": 78, "d": {"x": 3, "y": 42, "z": 77}}

It is brilliant! I use other models in the main model, and it works on the whole structure fortunately. — Arpad Horvath -- Слава Україні, Dec 18 '20 at 22:17
Actually, no. First approach does not work if `None` is explicitly set for optional field of nested model, but I updated the answer for such cases. — alex_noname, Dec 19 '20 at 06:02

Roshin Jay · Answer 2 · 2023-02-17T17:08:16.760

If you're using FastAPI then using exclude_none doesn't seem to work when a response_model is mentioned in the route decorator.

@app.post("/items/", response_model=Item)
async def create_item(item: Item):
    return item.dict(exclude_none=True)

Fast api seems to reprocess the dict with the pydantic model

So overriding the dict method in the model itself should work

def Item(BaseModel):
   name: str
   description: Optional[str]
   ...
   def dict(self, *args, **kwargs) -> Dict[str, Any]:
        kwargs.pop('exclude_none', None)
        return super().dict(*args, exclude_none=True, **kwargs)

(an actual solution would put this definition in separate subclass of BaseModel for reuse)

Note: just changing the default value of the exclude_none keyword argument is not enough: it seems FastAPI always sends exclude_none=False as an argument.

Source:
https://github.com/tiangolo/fastapi/issues/3314#issuecomment-962932368

In JSON created from a pydantic.BaseModel exclude Optional if not set

2 Answers2

Update