What is the meaning of __total__ dunder attribute in Python 3?
TypedDict
was accepted in Python 3.8 via PEP 589. From Python, it appears __total__
is a boolean flag set to True
by default:
tot = TypedDict.__total__
print(type(tot))
print(tot)
# <class 'bool'>
# True
As mentioned in other posts, details on this method are limited in the docs, but @Yann Vernier's link to the CPython source code strongly suggests __total__
is related to the new total
keyword introduced in Python 3.8:
# cypthon/typing.py
class _TypedDictMeta(type):
def __new__(cls, name, bases, ns, total=True):
"""Create new typed dict class object.
...
"""
...
if not hasattr(tp_dict, '__total__'):
tp_dict.__total__ = total
...
How does it work?
Synopsis: by default, all keys are required when instantiating a defined TypedDict
. total=False
overrides this restriction and allows optional keys. See the following demonstration.
Given
A test directory tree:
Code
Files in the test directory:
# rgb_bad.py
from typing import TypedDict
class Color(TypedDict):
r: int
g: int
b: int
a: float
blue = Color(r=0, g=0, b=255) # missing "a"
# rgb_good.py
from typing import TypedDict
class Color(TypedDict, total=False):
r: int
g: int
b: int
a: float
blue = Color(r=0, g=0, b=255) # missing "a"
Demo
If a key is missing, mypy will complain at the commandline:
> mypy code/rgb_bad.py
code\rgb_bad.py:11: error: Key 'a' missing for TypedDict "Color"
...
Setting total=False
permits optional keys:
> mypy code/rgb_good.py
Success: no issues found in 1 source file
See Also
- Tweet by R. Hettinger demonstating totality
- PEP section on totality in PEP 589
- Article Section on types and
TypedDict
in Python 3.8 by Real Python typing-extensions
package to useTypedDict
in Python 3.5, 3.6
I am guessing that the __total__
field signifies whether instances must be complete (the default) or not (all fields optional). I started my search at PEP 589, which introduced TypedDict
and describes totality as such. It used a total
argument, which it would make sense to rename dunder-style for the class
syntax. However, I did not find when such a renaming took place.
Looking into MyPy, which is the actual type checker that cares about these annotations, there is similar documentation on TypedDict
and totality, but again no reference to the dunder syntax. Digging into its implementation led to more confusion, as TypedDictType
in types.py doesn't have a total field, but separate items
and required_keys
. Totality would imply that items.keys()==required_keys
but the implementation makes different assumptions, such as can_be_false
relying on items
alone. total=False
should in principle mean required_keys
is empty.
The CPython source for _TypedDictMeta at least reveals that the total
argument and __total__
dunder are one and the same, although the source describes TypedDict
itself as "may be added soon".