How to handle urllib's timeout in Python 3?
The exception is timeout from socket, so
from socket import timeout
try:
response = urllib.request.urlopen(url, timeout=10).read().decode('utf-8')
except (HTTPError, URLError) as error:
logging.error('Data of %s not retrieved because %s\nURL: %s', name, error, url)
except timeout:
logging.error('socket timed out - URL %s', url)
else:
logging.info('Access successful.')
should catch the new exception.
The previous answer does not correctly intercept timeout errors. Timeout errors are raised as URLError
, so if we want to specifically catch them, we need to write:
from urllib.error import HTTPError, URLError
import socket
try:
response = urllib.request.urlopen(url, timeout=10).read().decode('utf-8')
except HTTPError as error:
logging.error('Data not retrieved because %s\nURL: %s', error, url)
except URLError as error:
if isinstance(error.reason, socket.timeout):
logging.error('socket timed out - URL %s', url)
else:
logging.error('some other error happened)
else:
logging.info('Access successful.')
Note that ValueError
can independently be raised, i.e. if the URL is invalid. Like HTTPError
, it is not associated with a timeout.