In Python, how do I determine if an object is iterable?
Checking for
__iter__
works on sequence types, but it would fail on e.g. strings in Python 2. I would like to know the right answer too, until then, here is one possibility (which would work on strings, too):from __future__ import print_function try: some_object_iterator = iter(some_object) except TypeError as te: print(some_object, 'is not iterable')
The
iter
built-in checks for the__iter__
method or in the case of strings the__getitem__
method.Another general pythonic approach is to assume an iterable, then fail gracefully if it does not work on the given object. The Python glossary:
Pythonic programming style that determines an object's type by inspection of its method or attribute signature rather than by explicit relationship to some type object ("If it looks like a duck and quacks like a duck, it must be a duck.") By emphasizing interfaces rather than specific types, well-designed code improves its flexibility by allowing polymorphic substitution. Duck-typing avoids tests using type() or isinstance(). Instead, it typically employs the EAFP (Easier to Ask Forgiveness than Permission) style of programming.
...
try: _ = (e for e in my_object) except TypeError: print my_object, 'is not iterable'
The
collections
module provides some abstract base classes, which allow to ask classes or instances if they provide particular functionality, for example:from collections.abc import Iterable if isinstance(e, Iterable): # e is iterable
However, this does not check for classes that are iterable through
__getitem__
.
Duck typing
try:
iterator = iter(theElement)
except TypeError:
# not iterable
else:
# iterable
# for obj in iterator:
# pass
Type checking
Use the Abstract Base Classes. They need at least Python 2.6 and work only for new-style classes.
from collections.abc import Iterable # import directly from collections for Python < 3.3
if isinstance(theElement, Iterable):
# iterable
else:
# not iterable
However, iter()
is a bit more reliable as described by the documentation:
Checking
isinstance(obj, Iterable)
detects classes that are registered as Iterable or that have an__iter__()
method, but it does not detect classes that iterate with the__getitem__()
method. The only reliable way to determine whether an object is iterable is to calliter(obj)
.
I'd like to shed a little bit more light on the interplay of iter
, __iter__
and __getitem__
and what happens behind the curtains. Armed with that knowledge, you will be able to understand why the best you can do is
try:
iter(maybe_iterable)
print('iteration will probably work')
except TypeError:
print('not iterable')
I will list the facts first and then follow up with a quick reminder of what happens when you employ a for
loop in python, followed by a discussion to illustrate the facts.
Facts
You can get an iterator from any object
o
by callingiter(o)
if at least one of the following conditions holds true:
a)o
has an__iter__
method which returns an iterator object. An iterator is any object with an__iter__
and a__next__
(Python 2:next
) method.
b)o
has a__getitem__
method.Checking for an instance of
Iterable
orSequence
, or checking for the attribute__iter__
is not enough.If an object
o
implements only__getitem__
, but not__iter__
,iter(o)
will construct an iterator that tries to fetch items fromo
by integer index, starting at index 0. The iterator will catch anyIndexError
(but no other errors) that is raised and then raisesStopIteration
itself.In the most general sense, there's no way to check whether the iterator returned by
iter
is sane other than to try it out.If an object
o
implements__iter__
, theiter
function will make sure that the object returned by__iter__
is an iterator. There is no sanity check if an object only implements__getitem__
.__iter__
wins. If an objecto
implements both__iter__
and__getitem__
,iter(o)
will call__iter__
.If you want to make your own objects iterable, always implement the
__iter__
method.
for
loops
In order to follow along, you need an understanding of what happens when you employ a for
loop in Python. Feel free to skip right to the next section if you already know.
When you use for item in o
for some iterable object o
, Python calls iter(o)
and expects an iterator object as the return value. An iterator is any object which implements a __next__
(or next
in Python 2) method and an __iter__
method.
By convention, the __iter__
method of an iterator should return the object itself (i.e. return self
). Python then calls next
on the iterator until StopIteration
is raised. All of this happens implicitly, but the following demonstration makes it visible:
import random
class DemoIterable(object):
def __iter__(self):
print('__iter__ called')
return DemoIterator()
class DemoIterator(object):
def __iter__(self):
return self
def __next__(self):
print('__next__ called')
r = random.randint(1, 10)
if r == 5:
print('raising StopIteration')
raise StopIteration
return r
Iteration over a DemoIterable
:
>>> di = DemoIterable()
>>> for x in di:
... print(x)
...
__iter__ called
__next__ called
9
__next__ called
8
__next__ called
10
__next__ called
3
__next__ called
10
__next__ called
raising StopIteration
Discussion and illustrations
On point 1 and 2: getting an iterator and unreliable checks
Consider the following class:
class BasicIterable(object):
def __getitem__(self, item):
if item == 3:
raise IndexError
return item
Calling iter
with an instance of BasicIterable
will return an iterator without any problems because BasicIterable
implements __getitem__
.
>>> b = BasicIterable()
>>> iter(b)
<iterator object at 0x7f1ab216e320>
However, it is important to note that b
does not have the __iter__
attribute and is not considered an instance of Iterable
or Sequence
:
>>> from collections import Iterable, Sequence
>>> hasattr(b, '__iter__')
False
>>> isinstance(b, Iterable)
False
>>> isinstance(b, Sequence)
False
This is why Fluent Python by Luciano Ramalho recommends calling iter
and handling the potential TypeError
as the most accurate way to check whether an object is iterable. Quoting directly from the book:
As of Python 3.4, the most accurate way to check whether an object
x
is iterable is to calliter(x)
and handle aTypeError
exception if it isn’t. This is more accurate than usingisinstance(x, abc.Iterable)
, becauseiter(x)
also considers the legacy__getitem__
method, while theIterable
ABC does not.
On point 3: Iterating over objects which only provide __getitem__
, but not __iter__
Iterating over an instance of BasicIterable
works as expected: Python
constructs an iterator that tries to fetch items by index, starting at zero, until an IndexError
is raised. The demo object's __getitem__
method simply returns the item
which was supplied as the argument to __getitem__(self, item)
by the iterator returned by iter
.
>>> b = BasicIterable()
>>> it = iter(b)
>>> next(it)
0
>>> next(it)
1
>>> next(it)
2
>>> next(it)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
Note that the iterator raises StopIteration
when it cannot return the next item and that the IndexError
which is raised for item == 3
is handled internally. This is why looping over a BasicIterable
with a for
loop works as expected:
>>> for x in b:
... print(x)
...
0
1
2
Here's another example in order to drive home the concept of how the iterator returned by iter
tries to access items by index. WrappedDict
does not inherit from dict
, which means instances won't have an __iter__
method.
class WrappedDict(object): # note: no inheritance from dict!
def __init__(self, dic):
self._dict = dic
def __getitem__(self, item):
try:
return self._dict[item] # delegate to dict.__getitem__
except KeyError:
raise IndexError
Note that calls to __getitem__
are delegated to dict.__getitem__
for which the square bracket notation is simply a shorthand.
>>> w = WrappedDict({-1: 'not printed',
... 0: 'hi', 1: 'StackOverflow', 2: '!',
... 4: 'not printed',
... 'x': 'not printed'})
>>> for x in w:
... print(x)
...
hi
StackOverflow
!
On point 4 and 5: iter
checks for an iterator when it calls __iter__
:
When iter(o)
is called for an object o
, iter
will make sure that the return value of __iter__
, if the method is present, is an iterator. This means that the returned object
must implement __next__
(or next
in Python 2) and __iter__
. iter
cannot perform any sanity checks for objects which only
provide __getitem__
, because it has no way to check whether the items of the object are accessible by integer index.
class FailIterIterable(object):
def __iter__(self):
return object() # not an iterator
class FailGetitemIterable(object):
def __getitem__(self, item):
raise Exception
Note that constructing an iterator from FailIterIterable
instances fails immediately, while constructing an iterator from FailGetItemIterable
succeeds, but will throw an Exception on the first call to __next__
.
>>> fii = FailIterIterable()
>>> iter(fii)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: iter() returned non-iterator of type 'object'
>>>
>>> fgi = FailGetitemIterable()
>>> it = iter(fgi)
>>> next(it)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/path/iterdemo.py", line 42, in __getitem__
raise Exception
Exception
On point 6: __iter__
wins
This one is straightforward. If an object implements __iter__
and __getitem__
, iter
will call __iter__
. Consider the following class
class IterWinsDemo(object):
def __iter__(self):
return iter(['__iter__', 'wins'])
def __getitem__(self, item):
return ['__getitem__', 'wins'][item]
and the output when looping over an instance:
>>> iwd = IterWinsDemo()
>>> for x in iwd:
... print(x)
...
__iter__
wins
On point 7: your iterable classes should implement __iter__
You might ask yourself why most builtin sequences like list
implement an __iter__
method when __getitem__
would be sufficient.
class WrappedList(object): # note: no inheritance from list!
def __init__(self, lst):
self._list = lst
def __getitem__(self, item):
return self._list[item]
After all, iteration over instances of the class above, which delegates calls to __getitem__
to list.__getitem__
(using the square bracket notation), will work fine:
>>> wl = WrappedList(['A', 'B', 'C'])
>>> for x in wl:
... print(x)
...
A
B
C
The reasons your custom iterables should implement __iter__
are as follows:
- If you implement
__iter__
, instances will be considered iterables, andisinstance(o, collections.abc.Iterable)
will returnTrue
. - If the object returned by
__iter__
is not an iterator,iter
will fail immediately and raise aTypeError
. - The special handling of
__getitem__
exists for backwards compatibility reasons. Quoting again from Fluent Python:
That is why any Python sequence is iterable: they all implement
__getitem__
. In fact, the standard sequences also implement__iter__
, and yours should too, because the special handling of__getitem__
exists for backward compatibility reasons and may be gone in the future (although it is not deprecated as I write this).