Why does list ask about __len__?
See the Rationale section from PEP 424 that introduced __length_hint__
and offers insight on the motivation:
Being able to pre-allocate lists based on the expected size, as estimated by
__length_hint__
, can be a significant optimization. CPython has been observed to run some code faster than PyPy, purely because of this optimization being present.
In addition to that, the documentation for object.__length_hint__
verifies the fact that this is purely an optimization feature:
Called to implement
operator.length_hint()
. Should return an estimated length for the object (which may be greater or less than the actual length). The length must be an integer>= 0
. This method is purely an optimization and is never required for correctness.
So __length_hint__
is here because it can result in some nice optimizations.
PyObject_LengthHint
, first tries to get a value from object.__len__
(if it is defined) and then tries to see if object.__length_hint__
is available. If neither is there, it returns a default value of 8
for lists.
listextend
, which is called from list_init
as Eli stated in his answer, was modified according to this PEP to offer this optimization for anything that defines either a __len__
or a __length_hint__
.
list
isn't the only one that benefits from this, of course, bytes
objects do:
>>> bytes(Foo())
len
getitem 0
...
b'\x00\x01\x04\t\x10\x19'
so do bytearray
objects but, only when you extend
them:
>>> bytearray().extend(Foo())
len
getitem 0
...
and tuple
objects which create an intermediary sequence to populate themselves:
>>> tuple(Foo())
len
getitem 0
...
(0, 1, 4, 9, 16, 25)
If anybody is wandering why exactly 'iter'
is printed before 'len'
in class Bar
and not after as happens with class Foo
:
This is because if the object in hand defines an __iter__
Python will first call it to get the iterator, thereby running the print('iter')
too. The same doesn't happen if it falls back to using __getitem__
.
list
is a list object constructor that will allocate an initial slice of memory for its contents. The list constructor attempts to figure out a good size for that initial slice of memory by checking the length hint or the length of any object passed into the constructor . See the call to PyObject_LengthHint
in the Python source here. This place is called from the list constructor -- list_init
If your object has no __len__
or __length_hint__
, that's OK -- a default value of 8 is used; it just may be less efficient due to reallocations.