How to handle response encoding from urllib.request.urlopen() , to avoid TypeError: can't use a string pattern on a bytes-like object
As for me, the solution is as following (python3):
resource = urllib.request.urlopen(an_url)
content = resource.read().decode(resource.headers.get_content_charset())
You just need to decode the response, using the Content-Type
header typically the last value. There is an example given in the tutorial too.
output = response.decode('utf-8')
I had the same issues for the last two days. I finally have a solution.
I'm using the info()
method of the object returned by urlopen()
:
req=urllib.request.urlopen(URL)
charset=req.info().get_content_charset()
content=req.read().decode(charset)