TypedDict does not allow optional keys?

Question

I wanted to have a schema validation using pydantic, and also TypedDict to define the part of a nested dict schema. However, I realised that the Optional does not work if it is specified within the TypedDict class.

I read that this class will render all the keys within as required, and a way to make ALL of them as optional was to at total=False. However, I only wanted one of the keys to be optional and the rest required. Is there a way to overcome this limitation?

from typing import List, Optional
from pydantic import BaseModel
from typing_extensions import TypedDict

class _trending(TypedDict):
    allStores: Optional[bool] = False
    category: str
    date: str
    average: List[int]

class RequestSchema(BaseModel):
    storeId: str
    trending: _trending

EDIT

I tried this previously as I thought it is similar to a nested list.

from typing import List, Optional, Dict
from pydantic import BaseModel


class _trending(BaseModel):
    allStores: Optional[bool] = False
    category: str
    date: str
    average: List[int]

class RequestSchema(BaseModel):
    storeId: str
    trending: Dict[_trending]

but faced an error msg saying that Dict requires 2 parameters. Apparently Dict works differently and I can't define it as a class like I hope to be?

Answer 1

Since Python 3.11, as per PEP 655, what you need is NotRequired:

class _trending(TypedDict):
    allStores: NotRequired[bool]
    category: str
    date: str
    average: List[int]

notice that you shouldn't use Optional in TypedDict, and only use (Not)Required

if you want to use TypedDict with Pydantic, you could refer this article

Answer 2

I used this question as duplicate target, but noticed that another option is missing here.

If you don't like NotRequired (for example, if you have many required and many optional keys and don't want to repeat NotRequired many times) or don't want to bother with typing_extensions (rare case), you can tweak totality.

The following definitions of Main* are equivalent:

import sys
# You may also pick one without version check, of course
if sys.version_info < (3, 11):
    from typing_extensions import TypedDict, Required, NotRequired
else:
    from typing import TypedDict, Required, NotRequired


class Main1(TypedDict):
    foo: int
    bar: str
    baz: NotRequired[int]
    qux: NotRequired[str]


class Main2(TypedDict, total=False):
    foo: Required[int]
    bar: Required[str]
    baz: int
    qux: str


class _Main3(TypedDict):
    foo: int
    bar: str

class Main3(_Main3, total=False):
    baz: int
    qux: str


class _Main4(TypedDict, total=False):
    baz: int
    qux: str
    
class Main4(_Main4):
    foo: int
    bar: str

Here's PEP explanation of totality:

The totality flag only applies to items defined in the body of the TypedDict definition. Inherited items won’t be affected, and instead use totality of the TypedDict type where they were defined. This makes it possible to have a combination of required and non-required keys in a single TypedDict type.

Here's example of checking with definitions above:

Main1(foo=1, bar='bar', baz=2, qux='qux')
Main1(foo=1, bar='bar', baz=2)
Main1(foo=1, bar='bar')
Main1(foo=1, baz=2, qux='qux')  # E: Missing key "bar" for TypedDict "Main1"  [typeddict-item]
Main1(foo=1, bar='bar', who=None)  # E: Extra key "who" for TypedDict "Main1"  [typeddict-item]

Main2(foo=1, bar='bar', baz=2, qux='qux')
Main2(foo=1, bar='bar', baz=2)
Main2(foo=1, bar='bar')
Main2(foo=1, baz=2, qux='qux')  # E: Missing key "bar" for TypedDict "Main2"  [typeddict-item]
Main2(foo=1, bar='bar', who=None)  # E: Extra key "who" for TypedDict "Main2"  [typeddict-item]

Main3(foo=1, bar='bar', baz=2, qux='qux')
Main3(foo=1, bar='bar', baz=2)
Main3(foo=1, bar='bar')
Main3(foo=1, baz=2, qux='qux')  # E: Missing key "bar" for TypedDict "Main3"  [typeddict-item]
Main3(foo=1, bar='bar', who=None)  # E: Extra key "who" for TypedDict "Main3"  [typeddict-item]

Main4(foo=1, bar='bar', baz=2, qux='qux')
Main4(foo=1, bar='bar', baz=2)
Main4(foo=1, bar='bar')
Main4(foo=1, baz=2, qux='qux')  # E: Missing key "bar" for TypedDict "Main4"  [typeddict-item]
Main4(foo=1, bar='bar', who=None)  # E: Extra key "who" for TypedDict "Main4"  [typeddict-item]

You can fiddle with this in playground