(...) mark specific dictionaries/mappings as total/exhaustive, meaning, (...) mypy will give an error about the definition of lookup in the BAD example
IOW, what the bold sentence says is: Create a type hint dependency of totality/exhaustiveness from lookup to Foo. Roughly in UML:
Dependency meaning lookup depends on changes to Foo, but with the conflated quadruple requirement that:
A. The dependency be implemented using only type hints (hence a static type checker dependency, not a run-time dependency).
B. The dependency automatically reflect changes in Foo to TypedDict without a need to rewrite type hints upon changing Foo. (This one is completly over the top.)
C. The dependency cause mypy to issue a warning when it's not satisfied.
D. The lookup dictionary keep a Totality relation to the Foo members.
Simple answer is: NO. Python doesn't have one native type hint to establish such a dependency; and it can't be achieved by combining static type hints without requiring rewrites that keep Foo and the TypedDict in sync. So the only choice is resorting to a run-time implementation, or rewriting the TypedDict definition to reflect changes to Foo. (I.e: It's not possible to satisfy requirements A thogether with B.)
(The hard part is demonstrating "why not", so the following points try to build up an incremental demonstration addressing the several possibilities the question mentions.)
1. Declaration
1.1. Literal and TypedDict have to be written in full at declaration, their syntax rules don't allow writing a dynamic declaration. So writing the dependency between lookup: dict and Foo: Enum into the type hints at declaration can't be done. (It's not possible so satisfy requirement B and A together.)
See the PEP quotes below: It's not possible to declare Literal[*Foo] by unpacking or other run-time means, and the same goes for TypedDict because it doesn't have a constructor (other than explicit class syntax and the alternative syntax) that would allow the declaration to be populated as a function of Enum Foo or dict lookup type hint to capture the dependency without writing it explicitly in full.
PEP 586 - Illegal parameters for Literal at type check time
The following parameters are intentionally disallowed by design:
Arbitrary expressions like Literal[3 + 4] or Literal["foo".replace("o", "b")].
(...)
Any other types: for example, Literal[Path], or Literal[some_object_instance] are illegal. This includes typevars: if T is a typevar, Literal[T] is not allowed. Typevars can vary over only types, never over values.
And specific to the TypeDict:
PEP 589 – TypedDict: Type Hints for Dictionaries with a Fixed Set of Keys
Abstract
This PEP proposes a type constructor typing.TypedDict to support the use case where a dictionary object has a specific set of string keys, each with a value of a specific type.
Class-based Syntax
String literal forward references are valid in the value types
This PEP proposes a type constructor typing.TypedDict to support the use case where a dictionary object has a specific set of string keys, each with a value of a specific type.
1.2. Using forward references wouldn't change the fact that references to the Enum members can only be written into the values not the keys (using class syntax) (requirement A is not met).
class Movie1(TypedDict):
cool: "Literal[Foo.X]"
whatever: "Literal[Foo.Y]"
class Movie2(TypedDict):
cool: Literal[Foo.X]
whatever: Literal[Foo.Y]
1.3. The keys in the TypedDict have to be strings but the strings can't have dots (it conflicts with dotted syntax) and can't be written as string literals. So the following three examples won't work (requirement A is not met):
class Movie3(TypedDict): # Illegal syntax
"Foo.X": str
"Foo.Y": str
class Movie4(TypedDict): # Illegal syntax
Foo.X: str
Foo.Y: str
# using a dotted syntax that has no corresponding variable also doesn't work
class Movie5(TypedDict): # Illegal syntax
a.x: str
b.y: str
1.4. The previous point also means you could use an alias for the Enum members in order to write them into the TypedDict, the following code would work:
However, this would again defeat the question's main purpose of not having to write out and maintain a second group of declarations that need to be updated to reflect changes to the Enum. (Requirement B is again not met.)
some_alias1 = Foo.X
some_alias2 = Foo.Y
class Movie6(TypedDict):
some_alias1 : str
some_alias2 : str
lookup_forward_ref3: Movie3 = {'some_alias1': "cool", 'some_alias2': "whatever"}
1.5 TypeDict's Alternative Syntax
Using the Alternative Syntax of TypeDict (opposed to class syntax) allows to circumvent the dotted syntax problem mentioned earlier (in 1.3.), the following passes with mypy 0.931
class Foo(Enum):
X = auto()
Y = auto()
Z = auto() # NEW
Movie = TypedDict('Movie',
{
'Foo.X': str,
'Foo.Y': str,
'Foo.Z': Literal[Foo.X]}, # just as a Literal example
total=False)
lookup2: Movie = {'Foo.X': "cool", 'Foo.Y': "whatever", 'Foo.Z': Foo.X}
This is one step closer to the possible alternatives you were asking for:
I'm happy to change the exact syntax with which we build the dictionary, but the closer it is to {Foo.X: "cool", Foo.Y: "whatever"} the better.
However, you'll still have to maintain the TypedDict declaration in sync with changes to the Enum Foo (so it doesn't satisfy requirements B but requirements A, C and D are pretty close). If for example you tried populating the key-values with something more dynamic:
part_declaration = {
'Foo.X': str,
'Foo.Y': str,
'Foo.Z': Literal[Foo.X]}
Movie = TypedDict('Movie',
part_declaration,
total=False)
Mypy would remind you that:
your_module.py:27: error: TypedDict() expects a dictionary literal as the second argument
your_module.py:31: error: Extra keys ("Foo.X", "Foo.Y", "Foo.Z") for TypedDict "TypedDict"
1.6 Use of Final Values and Literal Types
It should be emphasized that using Literals as key's to the TypedDict is only legal for string literals not Enum literals (notice the bolds in the PEP quote). So, the TypedDict has to be declared in full; looking at Enum Literals for a solution won't change that fact. (Requirement B again not met).
PEP 589 – TypedDict: Type Hints for Dictionaries with a Fixed Set of Keys
Use of Final Values and Literal Types
Type checkers should allow final names (PEP 591) with string values to be used instead of string literals
Type checkers are only expected to support actual string literals, not final names or literal types,
Mypy also considers Enum Literals as final, see Extra Enum checks but that doesn't superseed the above mentioned string literal limitation.
2 Relation between Literal[YourEnum.member] and YourEnum
In most cases there's no difference between typing a variable as the_var: Foo or the_var: Literal[Foo.X, Foo.Y, Foo.Z]] if the Literal has all the Enum members because it would accept the exact same types.
The question mentions using Literals over just Foo (the Enum members are subclasses of the Enum so nominal subtyping rules apply). But for the purpose of the question using Literals won't solve the problem of creating a type hint dependency between lookup and Foo that reflects changes to the later without requiring rewrites (again requirement B not satisfied).
The following two declarations are equivalent:
class Foo(Enum):
X = auto()
Y = auto()
Z = auto()
var1: Foo
var2: Literal[Foo.X, Foo.Y, Foo.Z]
var1 = Foo.X
var1 = Foo.Y
var1 = Foo.Z
var2 = Foo.X
var2 = Foo.Y
var2 = Foo.Z
Lets now look at the 2 properties mentioned in the question:
3. Totality
A property of the TypeDict. As said before the TypeDict definition has to be written in full at declaration - there's no way for the TypedDict keys to reflect changes to Enum Foo without writing those changes explicitly in the declaration. (Requirement B again not met.)
TypedDict is the definition of a dependency on the type of it's values and the strings values of its keys. What totality aims to capture is a dependency between instances of TypedDict and the type itself. So trying to express a relationship of totality dependence to another type can only be done by explicitly coding that dependency. (Without satisfying requirement B it's possible to satisfy requirements A and C but you'll have to manually maintain those dependencies up-to-date).
4. Exhaustiveness
This property of the Enum's (see PEP 596 - Interactions with enums and exhaustiveness checks and mypy - Exhaustiveness checking) is mentioned in the question but it's orthogonal to requirements A, B.
Exhaustiveness is logic (the if/else brach) related to data (the Enum). It allows a run-time implementation that the static type checker verifies, it is not a type hint! (So it's not even in the league of requirement A - because it's not a type hint; it again doesn't satisfy requirement B; but it can satisfy requirement C as you've implemented it; it circumvents requirement D by implementing an explicit run-time mapping to the string constants instead of using a type hint TypedDict to maintain the dependency between lookup strings and Enum members).
5. Conclusion
If you notice requirement B is never satisfied using static type hint checks (you have to write the type hints and mantain them). Most developers would go straight for a unittest or run-time check (or just let the KeyError be thrown because it's easier to...):
class Foo(Enum):
X = auto()
Y = auto()
Z = auto() # NEW
@classmethod
def totality(cls, lookup: dict[str, Any]):
for member in cls:
if '{}.{}'.format(cls.__qualname__, member.name) not in lookup.keys():
raise KeyError # lookup isn't total to Enum.
The main use of type hints is hinting to the developer what types are acceptable. Your use departs from that by trying to establish a mapping between two sets of permissable values and turning those values together with the mapping into a type.
The point of such use isn't giving you a warning if you forget to maintain something in your code, but to remind you what types (in this case mapping between values) are acceptable.
6. Addressing the questions:
I'd love to be able to mark specific dictionaries/mappings as total/exhaustive
Can be done with TypedDict. Declare the type and maintain it current to Enum Foo.
meaning, for instance, mypy will give an error about the definition of lookup in the BAD example above.
Orthogonal to the previous statement! That has nothing to do with totality, the TypedDict keeps totality in relation to its type definition. Keep it's totality in relation to Foo's definition up to date and problem solved.
- Can this be annotated with Python's current type hints as a generic type, for any Enum or Literal input? (For instance, lookup: ExhaustiveDict[Foo, str] = {...}.)
This question doesn't make sense. The type hint you give as example works for the Enum members as shown in (2.) and you don't specify what any Literal means...? I don't see how Generic would help here.
- If not, can it be done for a specific pair of keys/values? (For instance, lookup: ExhaustiveDictFooTo[str] = {...} and/or lookup: ExhaustiveDictFooToStr = {...}.
Depends on the possible lookup dictionaries, you only give a 1:1 mapping between Enum members and string Literals so nothing could be simpler, it would look like this:
combo = tuple[tuple[Literal[Foo.X], Literal['cool']], tuple[Literal[Foo.Y], Literal['whatever']]]
Problem being it's not possible to express a 1:1 key-value relationship in a dictionary using type hints. So this is what turning values into types looks like in the extreme...
but it's annoying to go from a compact dict to a whole function
The straightforward solution is writing a TypedDict mapping to the Enum members' names as keys (as mentioned in 1.5) together with a lookup instance. The type hint itself could be written as
class Foo(Enum):
X = auto()
Y = auto()
Z = auto() # NEW
Movie = TypedDict('Movie',
{
'Foo.X': str,
'Foo.Y': Literal['whatever'], # just as a Literal example
'Foo.Z': Literal[Foo.X]}, # just as a Literal example
total=False)
lookup2: Movie = {'Foo.X': "cool", 'Foo.Y': 'whatever', 'Foo.Z': Foo.X}
If you want mypy to give you a warning you can also use the Exhaustiveness check (in 4.) but that warning is meant to remind you of oversights in writing your logic not the data! nor that type hints are out-of-date.
