Max recursion is not exactly what sys.getrecursionlimit() claims. How come?

The recursion limit is not the limit on recursion but the maximum depth of the python interpreter stack.There is something on the stack before your function gets executed. Spyder executes some python stuff before it calls your script, as do other interpreters like ipython.

You can inspect the stack via methods in the inspect module.

In CPython for me:

>>>print(len(inspect.stack()))
1

In Ipython for me:

>>>print(len(inspect.stack()))
10

As knbk pointed out in the comments as soon as you hit the stack limit a RecursionError is thrown and the interpreter raises the stack limit a bit to give you a possibility to handle the error gracefully. If you also exhaust that limit python will crash.


This limit is for stack, not for the function you define. There are other internal things which might push something to stack.

And of course it could depend on env in which it was executed. Some can pollute stack more, some less.


I believe the confusion originates from the difference between the stack size you see when the error occurs and the limit. The thing is that the last call, that caused the crash, likely accounts for more than 1 frame on the stack, because it itself makes some function calls. And by the time you catch the exception, the call and its internal calls will have been removed from the stack. You can actually see them in the traceback. Let's look at this one.

In [1]: import inspect

In [2]: import sys

In [3]: sys.setrecursionlimit(50)  # I'm setting this to 50 to make the traceback shorter.

In [4]: stack_log = []

In [5]: def recur():
    stack_log.append(len(inspect.stack()))
    recur()
   ...:     

In [6]: recur()

We get the traceback (note: there is no need to read it now, so just move forward to the next section).

---------------------------------------------------------------------------
RecursionError                            Traceback (most recent call last)
<ipython-input-6-45136123341b> in <module>()
----> 1 recur()

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
----> 2     stack_log.append(len(inspect.stack()))
      3     recur()
      4 

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/inspect.py in stack(context)
   1462 def stack(context=1):
   1463     """Return a list of records for the stack above the caller's frame."""
-> 1464     return getouterframes(sys._getframe(1), context)
   1465 
   1466 def trace(context=1):

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/inspect.py in getouterframes(frame, context)
   1439     framelist = []
   1440     while frame:
-> 1441         frameinfo = (frame,) + getframeinfo(frame, context)
   1442         framelist.append(FrameInfo(*frameinfo))
   1443         frame = frame.f_back

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/inspect.py in getframeinfo(frame, context)
   1412         start = lineno - 1 - context//2
   1413         try:
-> 1414             lines, lnum = findsource(frame)
   1415         except OSError:
   1416             lines = index = None

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/inspect.py in findsource(object)
    742     is raised if the source code cannot be retrieved."""
    743 
--> 744     file = getsourcefile(object)
    745     if file:
    746         # Invalidate cache if needed.

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/inspect.py in getsourcefile(object)
    670         return filename
    671     # only return a non-existent filename if the module has a PEP 302 loader
--> 672     if getattr(getmodule(object, filename), '__loader__', None) is not None:
    673         return filename
    674     # or it is in the linecache

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/inspect.py in getmodule(object, _filename)
    699     # Try the cache again with the absolute file name
    700     try:
--> 701         file = getabsfile(object, _filename)
    702     except TypeError:
    703         return None

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/inspect.py in getabsfile(object, _filename)
    683     if _filename is None:
    684         _filename = getsourcefile(object) or getfile(object)
--> 685     return os.path.normcase(os.path.abspath(_filename))
    686 
    687 modulesbyfile = {}

/Users/ilia/.venvs/py3/bin/../lib/python3.5/posixpath.py in abspath(path)
    355 def abspath(path):
    356     """Return an absolute path."""
--> 357     if not isabs(path):
    358         if isinstance(path, bytes):
    359             cwd = os.getcwdb()

/Users/ilia/.venvs/py3/bin/../lib/python3.5/posixpath.py in isabs(s)
     61 def isabs(s):
     62     """Test whether a path is absolute"""
---> 63     sep = _get_sep(s)
     64     return s.startswith(sep)
     65 

RecursionError: maximum recursion depth exceeded

What's with the stack log?

In [7]: stack_log[-1]
Out[7]: 39

Okay, we have 11 missing frames. Now, scroll down the traceback to the last recur call, i.e.

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
      2     stack_log.append(len(inspect.stack()))
----> 3     recur()
      4 

<ipython-input-5-643b16f38b2e> in recur()
      1 def recur():
----> 2     stack_log.append(len(inspect.stack()))
      3     recur()
      4 

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/inspect.py in stack(context)
   1462 def stack(context=1):
   1463     """Return a list of records for the stack above the caller's frame."""
-> 1464     return getouterframes(sys._getframe(1), context)
   1465 
   1466 def trace(context=1):

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/inspect.py in getouterframes(frame, context)
   1439     framelist = []
   1440     while frame:
-> 1441         frameinfo = (frame,) + getframeinfo(frame, context)
   1442         framelist.append(FrameInfo(*frameinfo))
   1443         frame = frame.f_back

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/inspect.py in getframeinfo(frame, context)
   1412         start = lineno - 1 - context//2
   1413         try:
-> 1414             lines, lnum = findsource(frame)
   1415         except OSError:
   1416             lines = index = None

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/inspect.py in findsource(object)
    742     is raised if the source code cannot be retrieved."""
    743 
--> 744     file = getsourcefile(object)
    745     if file:
    746         # Invalidate cache if needed.

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/inspect.py in getsourcefile(object)
    670         return filename
    671     # only return a non-existent filename if the module has a PEP 302 loader
--> 672     if getattr(getmodule(object, filename), '__loader__', None) is not None:
    673         return filename
    674     # or it is in the linecache

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/inspect.py in getmodule(object, _filename)
    699     # Try the cache again with the absolute file name
    700     try:
--> 701         file = getabsfile(object, _filename)
    702     except TypeError:
    703         return None

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/inspect.py in getabsfile(object, _filename)
    683     if _filename is None:
    684         _filename = getsourcefile(object) or getfile(object)
--> 685     return os.path.normcase(os.path.abspath(_filename))
    686 
    687 modulesbyfile = {}

/Users/ilia/.venvs/py3/bin/../lib/python3.5/posixpath.py in abspath(path)
    355 def abspath(path):
    356     """Return an absolute path."""
--> 357     if not isabs(path):
    358         if isinstance(path, bytes):
    359             cwd = os.getcwdb()

/Users/ilia/.venvs/py3/bin/../lib/python3.5/posixpath.py in isabs(s)
     61 def isabs(s):
     62     """Test whether a path is absolute"""
---> 63     sep = _get_sep(s)
     64     return s.startswith(sep)
     65 

RecursionError: maximum recursion depth exceeded

And here you are, there are exactly 11 function calls (the arrows on the left), that is 11 frames on the stack that were removed when the exception was raised.