Filing here for Python3 with updated steps to reproduce and updated patch.
+++ This bug was initially created as a clone of Bug #1284916 +++
Description of problem:
The latest ssl.py file tries to validate hostnames vs certificates but includes a faulty regexp which causes any wildcard domains (e.g. *.s3.amazonaws.com) to fail validation.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
>>> import ssl
<_sre.SRE_Match object at 0x1474030> - the function should match the provided domain and hostname.
The following patch fixes the issue:
--- ssl.py.orig 2015-11-24 15:22:37.392416265 +0200
+++ ssl.py 2015-11-24 15:24:48.253470872 +0200
@@ -135,7 +135,7 @@
if frag == '*':
# When '*' is a fragment by itself, it matches a non-empty dotless
# Otherwise, '*' matches any dotless fragment.
frag = re.escape(frag)
From Python's documentation:
Used to indicate a set of characters. In a set:
Special characters lose their special meaning inside sets. For example, [(+*)] will match any of the literal characters '(', '+', '*', or ')'.
^^^^^^^^^ this is the cause of the error
The Python ssl module behaves correctly and as designed. Since recently it conforms to RFC 6125 section 6.4.3. (https://tools.ietf.org/html/rfc6125#section-6.4.3). The RFC restricts wildcards in several ways. Most importantly only one wildcard on the left-most label is supported. The suggested fix will break RFC 6125 compatibility and introduce a security bug.
By the way I'm a Python core committer, maintainer of the SSL module and the person that made the SSL module RFC 6125 conform. I closed the upstream issue https://bugs.python.org/issue25722
I've seen multiple people complaining about cert validation errors with AWS certs. AWS violates standards here. You have to roll your own custom host name verification in order to validate AWS certs.
indeed you are right, sorry for the false alarm. I've managed to track down the issue further. There's also a change in httplib.py and the calling code wasn't aware of that. I've managed to create a fix for the caller:
Probably this one should be closed.
You are welcome!
Your fix in https://github.com/s3tools/s3cmd/pull/668/files is dangerous and insecure. It allows MITM attacks on the connection. All it takes is a valid X.509 certificate that is signed by a trusted root CA. Without hostname checks your code will trust a certificate for e.g. www.example.com.
You must add additional checks, e.g. for Subject CN and X509v3 SAN field.
For the record:
There are custom hostname checks that conform to the AWS behavior, see match_hostname_aws() on line 74 in ConnMan.py. It's executed later in the code, after the connection has been established. I've only made sure that stock validation doesn't execute and blow things up on line 180.
Originally I hit the exception at line 180, you see the custom check performed on line 182:
181 if conn.ssl and cfg.check_ssl_certificate and cfg.check_ssl_hostname: