Difference between generators and functions returning generators
They will act the same. And about way to distinguish the two (without inspect
). In python? Only inspect:
import inspect
print inspect.isgeneratorfunction(g) --> False
print inspect.isgeneratorfunction(f) --> True
Of course you can also check it using dis
:
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> def f(x):
... yield x
...
>>> def g(x):
... return f(x)
...
>>> import dis
>>> dis.dis(f)
2 0 LOAD_FAST 0 (x)
3 YIELD_VALUE
4 POP_TOP
5 LOAD_CONST 0 (None)
8 RETURN_VALUE
>>> dis.dis(g)
2 0 LOAD_GLOBAL 0 (f)
3 LOAD_FAST 0 (x)
6 CALL_FUNCTION 1
9 RETURN_VALUE
but inspect
is more appropriate.
I like turkus answer, however examples shown are mostly theoretical and aren't common in day to day coding.
The main practical difference between generator function (with yield
) and function which returns generator is that the generator function is lazily evaluated.
Consider this session:
$ python
Python 3.6.0
[GCC 6.3.1 20170109] on linux
>>> def a():
... print('in a')
... yield 0
...
>>> def b():
... print('in b')
... return iter(range(1))
...
>>> aa = a() # Lazy evaluation - nothing printed after this line.
>>> next(aa)
in a
0
>>> next(aa)
Traceback ...
StopIteration
>>> bb = b() # Eager evaluation - whole function is executed after this.
in b
>>> next(bb)
0
>>> next(bb)
Traceback ...
StopIteration
None of them is a golden bullet.
To give you a real example of where this lazy evaluation makes a huge difference in your code check this example.
def get_numbers(x: int):
if x < 0:
raise ValueError("Value cannot be negative")
for i in range(x):
yield i
try:
numbers = get_numbers(-5)
except ValueError:
pass # log or something
else:
print(list(numbers)) # <== ValueError is thrown here!
Here is where lazy evaluation is actually bad for your function. It will throw exception in arguably wrong place because the intention is to make it fail just at the start, not during iteration. With this implementation you're passing responsibility of triggering the generator function and managing exception to its user which is tedious and somewhat ugly:
import itertools
try:
numbers = get_numbers(-5)
first = next(numbers)
numbers = itertools.chain([first], numbers)
except ValueError:
...
The best way to solve this is to make a function that returns a generator instead a generator function:
def get_numbers(x: int):
if x < 0:
raise ValueError("Value cannot be negative")
return (i for i in range(x)) # I know it can be just `return range(x)`, but I keep it that way to make a point.
As you can see there is no "best way" to do it, both options are viable. It all depends on how you want things to work.
The best way to check it out is using inspect.isgeneratorfunction, which is quite simple function:
def ismethod(object):
"""Return true if the object is an instance method.
Instance method objects provide these attributes:
__doc__ documentation string
__name__ name with which this method was defined
im_class class object in which this method belongs
im_func function object containing implementation of method
im_self instance to which this method is bound, or None"""
return isinstance(object, types.MethodType)
def isfunction(object):
"""Return true if the object is a user-defined function.
Function objects provide these attributes:
__doc__ documentation string
__name__ name with which this function was defined
func_code code object containing compiled function bytecode
func_defaults tuple of any default values for arguments
func_doc (same as __doc__)
func_globals global namespace in which this function was defined
func_name (same as __name__)"""
return isinstance(object, types.FunctionType)
def isgeneratorfunction(object):
"""Return true if the object is a user-defined generator function.
Generator function objects provides same attributes as functions.
See help(isfunction) for attributes listing."""
return bool((isfunction(object) or ismethod(object)) and
object.func_code.co_flags & CO_GENERATOR)
Now, if you declared your generator using a syntax like this:
my_generator = (i*i for i in range(1000000))
In that case, you could check its type quite easily, for instance, __class__
will return <type 'generator'>
.