Retry function in Python
I know this is an old question. However, I'd like to add the solution that I've cooked up. The best way is to write a retry
decorator that will retry when an exception occurs. Moreover, you can also set custom exponential delay. The docstring explains how you can use the decorator. Here you go:
import logging
import time
from functools import partial, wraps
def retry(func=None, exception=Exception, n_tries=5, delay=5, backoff=1, logger=False):
"""Retry decorator with exponential backoff.
Parameters
----------
func : typing.Callable, optional
Callable on which the decorator is applied, by default None
exception : Exception or tuple of Exceptions, optional
Exception(s) that invoke retry, by default Exception
n_tries : int, optional
Number of tries before giving up, by default 5
delay : int, optional
Initial delay between retries in seconds, by default 5
backoff : int, optional
Backoff multiplier e.g. value of 2 will double the delay, by default 1
logger : bool, optional
Option to log or print, by default False
Returns
-------
typing.Callable
Decorated callable that calls itself when exception(s) occur.
Examples
--------
>>> import random
>>> @retry(exception=Exception, n_tries=4)
... def test_random(text):
... x = random.random()
... if x < 0.5:
... raise Exception("Fail")
... else:
... print("Success: ", text)
>>> test_random("It works!")
"""
if func is None:
return partial(
retry,
exception=exception,
n_tries=n_tries,
delay=delay,
backoff=backoff,
logger=logger,
)
@wraps(func)
def wrapper(*args, **kwargs):
ntries, ndelay = n_tries, delay
while ntries > 1:
try:
return func(*args, **kwargs)
except exception as e:
msg = f"{str(e)}, Retrying in {ndelay} seconds..."
if logger:
logging.warning(msg)
else:
print(msg)
time.sleep(ndelay)
ntries -= 1
ndelay *= backoff
return func(*args, **kwargs)
return wrapper
There is another pip installable retry package: https://pypi.org/project/retry/
pip install retry
There are a couple of Python Packages:
- backoff
- tenacity
Example for backoff
import backoff
@backoff.on_exception(backoff.expo,
(MyPossibleException1,
MyPossibleException2))
def your_function(param1, param2):
# Do something
Example for tenacity
from tenacity import wait_exponential, retry, stop_after_attempt
@retry(wait=wait_exponential(multiplier=2, min=2, max=30), stop=stop_after_attempt(5))
def your_function(param1, param2):
# Do something
Apart from being able to pass functions and use them by adding ()
after the name (Python's syntax for invoking calls), you don't need to use recursion; just put it in a loop:
import time
def retry(fun, max_tries=10):
for i in range(max_tries):
try:
time.sleep(0.3)
fun()
break
except Exception:
continue
except Exception
should be changed to catch a meaningfull exception that the function might raise. Using Exception
(as I did in the example) is generally bad practice since it catches a large class of exceptions that you might not of wanted caught.
Apart from that, using a for-loop
instead of an explicit third counter and recursion (which leads to a long call-stack for large values) is better.