New submission from Christian Korneck :
Hello,
I have the impression that there's a general issue with how the Python stdlib
module `ssl` uses the Windows certificate store to read the "bundle" of trusted
Root CA certificates. At a first look, I couldn't find this issue documented
elsewhere, so I'm trying to describe it below (apologies if its a duplicate).
This issue leads to that on a standard Windows 10 installation with a standard
Python 2.x or 3.x installation TLS verification for many webservers fails out
of the box, including for common domains/webservers with a highly correct TLS
setup like https://google.de or https://www.verisign.com/ .
In short: On a vanilla Win 10 with a vanilla Python 2/3 installation, HTTPS
connections to "commonly trusted" domain names fail out of the box. Example
with Python 2.7.15:
>>> import urllib2
>>> response = urllib2.urlopen("https://google.de";)
[...]
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed
(_ssl.c:726)
Expected Behavior: TLS verify succeeds
Actual Behavior: TLS verify fails
Affected Python version/environment: I believe every Python version that uses
the Windows certificate store is affected (since 3.4 / 2.7.9). However, I've
only tested 2.7.11, 2.7.15, 3.7.2 (all 64 bit). I did test on Windows 10 1803,
1809, Windows Server 2019 1809 (all Pro x64 with latest patchlevel, i.e. the
Jan 2019 cumulative update). All tested Python versions on all tested Windows
10 versions show the same behavior.
Details:
1.) Background
- Factor1: Python's "ssl" std lib
Since Python 3.4 / 2.7.9 the ssl lib uses the Windows certificate store to get
a "bundle" of the trusted root CA certificates. (Some Python libraries like
requests bring their own ca bundle though, usually through certifi. These libs
are not affected). However, the ssl lib is not using the Windows SCHANNEL
library but instead bundles its own copy of openssl.
- Factor2: Windows 10 behavior
Windows provides a certificate store, a vendor managed and updated "bundle" of
Trusted Root CA certificates and a library for TLS operations called SCHANNEL
(the native Windows openssl equivalent).
On Windows 10, the list of pre-installed Trusted Root CA certificates is very
minimal. On Windows 10 1809 only 12 Root CAs are known by the certificate
store. In comparison certifi (Mozilla cabundle) currently lists 134 trusted
RootCAs. Many widely trusted RootCAs are missing out of the box in the Windows
certstore. Instead there's an online download mechanism used by the SCHANNEL
library to download additional trusted root CA certificates from a Microsoft
server when they are needed for the first time.
Example: The certificate currently used for https://google.de was signed by an
IntermediateCA which was signed by the RootCA "GlobalSign Root CA - R2". The
cert for this RootCA is not out of the box present in the Windows certstore and
therefore not trusted. When I make a HTTPS connection to this domain with a
client that uses the SCHANNEL library (i.e. Microsoft Edge or Internet Explorer
browser), the connection succeeds and is shown as "trusted". Afterwards the
previously missing RootCA certificate appears in the windows certstore. (The
Windows certstores can get inspected with the GUIs certml.msc (Machine store)
and certmgr.msc (User store)).
2.) Behavior
- install a vanilla Windows 10 1809 with default settings
- install a vanilla Python 2.7.15 and/or 3.7.2
In Python:
c:\python27\python.exe
Python 2.7.15 (v2.7.15:ca079a3ea3, Apr 30 2018, 16:30:26) [MSC v.1500 64 bit
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket, ssl
>>> context = ssl.SSLContext(ssl.PROTOCOL_TLS)
>>> context.verify_mode = ssl.CERT_REQUIRED
>>> context.check_hostname = True
# by default there are no cacerts in the context
>>> len(context.get_ca_certs())
0
>>> context.load_default_certs()
>>> len(context.get_ca_certs())
# after loading the cacerts from the Windows cert store "ROOT", we are seeing
some - but it's only 12 root cacerts in a vanilla Windows 10 (compared to 134
in the certifi / mozilla cabundle!)
12
>>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>> ssl_sock = context.wrap_socket(s, server_hostname='www.google.de')
>>> ssl_sock.connect(('www.google.de', 443))
Traceback (most recent call last):
File "", line 1, in
File "c:\python27\lib\ssl.py", line 882, in connect
self._real_connect(addr, False)
File "c:\python27\lib\ssl.py", line 873, in _real_connect
self.do_handshake()
File "c:\python27\lib\ssl.py", line 846, in do_handshake
self._sslobj.
