urllib and "SSL: CERTIFICATE_VERIFY_FAILED" Error

To expand on Craig Glennie's answer:

in Python 3.6.1 on MacOs Sierra

Entering this in the bash terminal solved the problem:

pip install certifi
/Applications/Python\ 3.6/Install\ Certificates.command

On Windows, Python does not look at the system certificate, it uses its own located at ?\lib\site-packages\certifi\cacert.pem.

The solution to your problem:

1. download the domain validation certificate as *.crt or *pem file
2. open the file in editor and copy it's content to clipboard
3. find your cacert.pem location: from requests.utils import DEFAULT_CA_BUNDLE_PATH; print(DEFAULT_CA_BUNDLE_PATH)
4. edit the cacert.pem file and paste your domain validation certificate at the end of the file.
5. Save the file and enjoy requests!

If you just want to bypass verification, you can create a new SSLContext. By default newly created contexts use CERT_NONE.

Be careful with this as stated in section 17.3.7.2.1

When calling the SSLContext constructor directly, CERT_NONE is the default. Since it does not authenticate the other peer, it can be insecure, especially in client mode where most of time you would like to ensure the authenticity of the server you’re talking to. Therefore, when in client mode, it is highly recommended to use CERT_REQUIRED.

But if you just want it to work now for some other reason you can do the following, you'll have to import ssl as well:

input = input.replace("!web ", "")      
url = "https://domainsearch.p.mashape.com/index.php?name=" + input
req = urllib2.Request(url, headers={ 'X-Mashape-Key': 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' })
gcontext = ssl.SSLContext()  # Only for gangstars
info = urllib2.urlopen(req, context=gcontext).read()
Message.Chat.SendMessage ("" + info)

This should get round your problem but you're not really solving any of the issues, but you won't see the [SSL: CERTIFICATE_VERIFY_FAILED] because you now aren't verifying the cert!

To add to the above, if you want to know more about why you are seeing these issues you will want to have a look at PEP 476.

This PEP proposes to enable verification of X509 certificate signatures, as well as hostname verification for Python's HTTP clients by default, subject to opt-out on a per-call basis. This change would be applied to Python 2.7, Python 3.4, and Python 3.5.

There is an advised opt out which isn't dissimilar to my advice above:

import ssl

# This restores the same behavior as before.
context = ssl._create_unverified_context()
urllib.urlopen("https://no-valid-cert", context=context)

It also features a highly discouraged option via monkeypatching which you don't often see in python:

import ssl

ssl._create_default_https_context = ssl._create_unverified_context

Which overrides the default function for context creation with the function to create an unverified context.

Please note with this as stated in the PEP:

This guidance is aimed primarily at system administrators that wish to adopt newer versions of Python that implement this PEP in legacy environments that do not yet support certificate verification on HTTPS connections. For example, an administrator may opt out by adding the monkeypatch above to sitecustomize.py in their Standard Operating Environment for Python. Applications and libraries SHOULD NOT be making this change process wide (except perhaps in response to a system administrator controlled configuration setting).

If you want to read a paper on why not validating certs is bad in software you can find it here!

This isn't a solution to your specific problem, but I'm putting it here because this thread is the top Google result for "SSL: CERTIFICATE_VERIFY_FAILED", and it lead me on a wild goose chase.

If you have installed Python 3.6 on OSX and are getting the "SSL: CERTIFICATE_VERIFY_FAILED" error when trying to connect to an https:// site, it's probably because Python 3.6 on OSX has no certificates at all, and can't validate any SSL connections. This is a change for 3.6 on OSX, and requires a post-install step, which installs the certifi package of certificates. This is documented in the file ReadMe.rtf, which you can find at /Applications/Python\ 3.6/ReadMe.rtf (see also the file Conclusion.rtf, and the script build-installer.py that generates the macOS installer).

The ReadMe will have you run the post-install script at

/Applications/Python\ 3.10/Install\ Certificates.command (Terminal App, this command alone should, fix the issue. Be sure to update the file path using your current subversion.)

(its source is install_certificates.command), which:

• first installs the Python package certifi, and
• then creates a symbolic link from the OpenSSL certificates file to the certificates file installed by the package certifi.

Release notes have some more info: https://www.python.org/downloads/release/python-360/

On newer versions of Python, there is more documentation about this:

• https://github.com/python/cpython/blob/e05a703848473b0365886dcc593cbddc46609f29/Mac/BuildScript/resources/ReadMe.rtf#L22-L34
• https://github.com/python/cpython/blob/e05a703848473b0365886dcc593cbddc46609f29/Mac/BuildScript/resources/Conclusion.rtf#L15-L19
• https://github.com/python/cpython/blob/e05a703848473b0365886dcc593cbddc46609f29/Mac/BuildScript/resources/Welcome.rtf#L23-L25
• https://github.com/python/cpython/blob/e05a703848473b0365886dcc593cbddc46609f29/Mac/BuildScript/resources/install_certificates.command
• https://github.com/python/cpython/blob/e05a703848473b0365886dcc593cbddc46609f29/Mac/BuildScript/README.rst
• https://github.com/python/cpython/blob/e05a703848473b0365886dcc593cbddc46609f29/Mac/BuildScript/build-installer.py#L239-L246