7

I have following Pydantic model type scheme specification:

class RequestPayloadPositionsParams(BaseModel):
    """
    Request payload positions parameters
    """

    account: str = Field(default="COMBINED ACCOUNT")
    fields: List[str] = Field(default=["QUANTITY", "OPEN_PRICE", "OPEN_COST"])


class RequestPayloadPositions(BaseModel):
    """
    Request payload positions service
    """

    header: RequestPayloadHeader = Field(
        default=RequestPayloadHeader(service="positions", id="positions", ver=0)
    )
    params: RequestPayloadPositionsParams = Field(
        default=RequestPayloadPositionsParams()
    )


class RequestPayloadOrdersParams(BaseModel):
    """
    Request payload orders parameters
    """

    account: str = Field(default="COMBINED ACCOUNT")
    types: List[str] = Field(default=["WORKING", "FILLED", "CANCELED"])


class RequestPayloadOrders(BaseModel):
    """
    Request payload orders service
    """

    header: RequestPayloadHeader = Field(
        default=RequestPayloadHeader(service="order_events", id="order_events", ver=0)
    )
    params: RequestPayloadOrdersParams = Field(default=RequestPayloadOrdersParams())


class RequestPayload(BaseModel):
    """
    Request payload data
    """

    payload: List[Union[RequestPayloadPositions, RequestPayloadOrders]] = Field(...)

Now, I want to create a payload object for both, orders and positions service:

positions = requests.RequestPayload(payload=[requests.RequestPayloadPositions()])
orders = requests.RequestPayload(payload=[requests.RequestPayloadOrders()])

Now, positions has type requests.RequestPayload[payload= requests.RequestPayloadPositions ... but orders has not requests.RequestPayload[payload= requests.RequestPayloadOrders but the same like positions. This is wrong.

I can fix this by change the model specification from payload: List[Union[RequestPayloadPositions, RequestPayloadOrders]] = Field(...) to payload: List[Any] = Field(...) ... but I want to explicitly define the allowed types.

Any idea how to solve this, or should I explain more detailed? Do you understand my problem?

EDIT Working code sample, shows that the second assert in the last line fails, but should not fail ...

from typing import List, Union
from pydantic import BaseModel, Field


class RequestPayloadHeader(BaseModel):
    """
    Request payload header
    """

    service: str = Field(...)
    id: str = Field(...)
    ver: int = Field(...)


class RequestPayloadLoginParams(BaseModel):
    """
    Request payload login parameters
    """

    domain: str = Field(default="TOS")
    platform: str = Field(default="PROD")
    token: str = Field(...)
    accessToken: str = Field(default="")
    tag: str = Field(default="TOSWeb")


class RequestPayloadLogin(BaseModel):
    """
    Request payload login service
    """

    header: RequestPayloadHeader = Field(
        default=RequestPayloadHeader(service="login", id="login", ver=0)
    )
    params: RequestPayloadLoginParams = Field(...)


class RequestPayloadPositionsParams(BaseModel):
    """
    Request payload positions parameters
    """

    account: str = Field(default="COMBINED ACCOUNT")
    fields: List[str] = Field(default=["QUANTITY", "OPEN_PRICE", "OPEN_COST"])


class RequestPayloadOrdersParams(BaseModel):
    """
    Request payload orders parameters
    """

    account: str = Field(default="COMBINED ACCOUNT")
    types: List[str] = Field(default=["WORKING", "FILLED", "CANCELED"])


class RequestPayloadService(BaseModel):
    """
    Request payload service
    """

    header: RequestPayloadHeader = Field(...)
    params: Union[RequestPayloadPositionsParams, RequestPayloadOrdersParams] = Field(
        ...
    )


class RequestPayload(BaseModel):
    """
    Request payload data
    """

    payload: List[Union[RequestPayloadLogin, RequestPayloadService]] = Field(...)


if __name__ == "__main__":
    positions = RequestPayload(
        payload=[
            RequestPayloadService(
                header=RequestPayloadHeader(service="positions", id="positions", ver=0),
                params=RequestPayloadPositionsParams(),
            )
        ]
    )
    assert isinstance(positions.payload[0].params, RequestPayloadPositionsParams)
    orders = RequestPayload(
        payload=[
            RequestPayloadService(
                header=RequestPayloadHeader(
                    service="order_events", id="order_events", ver=0
                ),
                params=RequestPayloadOrdersParams(),
            )
        ]
    )
    assert isinstance(orders.payload[0].params, RequestPayloadOrdersParams)

EDIT2 The solution by Alex does not cover the scenario when I have two models with same field names but different types for them, like this:

class ResponseReplacePatchStr(BaseModel):
    op: str = Field(default="replace")
    path: str = Field(...)
    value: str = Field(...)

    class Config:
        extra = "forbid"

class ResponseReplacePatchFloat(BaseModel):
    op: str = Field(default="replace")
    path: str = Field(...)
    value: float = Field(...)

    class Config:
        extra = "forbid"

It is always converted to type str for value field if ResponseReplacePatchStr is the first type mentioned in

Union[
            ResponseReplacePatchStr, ResponseReplacePatchFloat
        ]

How can I also solve this, so that Pydantic take care about my types?

alex_noname
  • 26,459
  • 5
  • 69
  • 86
Martin Fischer
  • 469
  • 1
  • 7
  • 21

1 Answers1

12

This is one of the features of Pydantic matching Union, which is described as:

However, as can be seen above, pydantic will attempt to 'match' any of the types defined under Union and will use the first one that matches.[...]

As such, it is recommended that, when defining Union annotations, the most specific type is included first and followed by less specific types.

At the same time, extra fields are ignored by default and default values of declared fields are used in your case.

Therefore, the solution might be adding extra = 'forbid' model config option:

class RequestPayloadPositionsParams(BaseModel):
    """
    Request payload positions parameters
    """
    account: str = Field(default="COMBINED ACCOUNT")
    fields: List[str] = Field(default=["QUANTITY", "OPEN_PRICE", "OPEN_COST"])

    class Config:
        extra = 'forbid'



class RequestPayloadOrdersParams(BaseModel):
    """
    Request payload orders parameters
    """
    account: str = Field(default="COMBINED ACCOUNT")
    types: List[str] = Field(default=["WORKING", "FILLED", "CANCELED"])

    class Config:
        extra = 'forbid'
alex_noname
  • 26,459
  • 5
  • 69
  • 86
  • Alex, I found a second issue which relates to this topic. Could you please have a look at my EDIT2 extension to my initial question here? thanks a lot – Martin Fischer Oct 25 '21 at 13:09
  • 1
    I've updated answer. Swap them as `Union[ResponseReplacePatchFloat, ResponseReplacePatchStr]` – alex_noname Oct 25 '21 at 13:24
  • I understand that by setting extra='forbid', extra attributes are forbidden. But why are all the attributes extra? – Baenka Mar 10 '22 at 08:41
  • `extra = 'forbid'` in config means that additional attributes, that are not defined by given schema are not allowed. That makes pydantic not accepting the first type, but proceeding to the next ones. – Mikaelblomkvistsson Feb 10 '23 at 09:00