Overhead on looping over an iterable class
The class version spends lots of time accessing its own variables. Each self.whatever
costs cycles. If you define your __iter__
as a generator and minimize the use of instance variables, the difference between class and function versions will be negligible:
setup = """
def create_generator(num):
mylist = range(num)
for i in mylist:
yield i
class Generator(object):
def __init__(self, num):
self.start = 0
self.end = num
def __iter__(self):
return self
def next(self):
if self.start == self.end:
raise StopIteration
else:
self.start = self.start + 1
return self.start
class Generator2(object):
def __init__(self, num):
self.mylist = range(num)
def __iter__(self):
for i in self.mylist:
yield i
"""
import timeit
print timeit.timeit('for p in create_generator(1000):p', setup, number=1000)
print timeit.timeit('for p in Generator(1000):p', setup, number=1000)
print timeit.timeit('for p in Generator2(1000):p', setup, number=1000)
Results:
0.158941984177
0.696810007095
0.160784959793
so the second generator class is almost as fast as the function version.
Please do note that Generator
and Generator2
in the example are not fully equivalent, there are cases when you cannot simply replace a "plain" iterator with a generator (e.g. marshaling).