Bug 1284930

Summary: [PATCH] Latest ssl.py breaks certificate validation for wildcard domains, e.g. *.s3.amazonaws.com
Product: Red Hat Software Collections Reporter: Alexander Todorov <atodorov>
Component: python33Assignee: Python Maintainers <python-maint>
Status: CLOSED NOTABUG QA Contact: BaseOS QE - Apps <qe-baseos-apps>
Severity: low Docs Contact:
Priority: low    
Version: devassist09CC: cheimes, jblazek, lmiksik, mstuchli, python-maint, qe-baseos-apps
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1284916 Environment:
Last Closed: 2015-11-24 14:51:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1284916    
Bug Blocks:    

Description Alexander Todorov 2015-11-24 13:27:23 UTC 
Filing here for Python3 with updated steps to reproduce and updated patch.

+++ This bug was initially created as a clone of Bug #1284916 +++

Description of problem:

The latest ssl.py file tries to validate hostnames vs certificates but includes a faulty regexp which causes any wildcard domains (e.g. *.s3.amazonaws.com) to fail validation. 

Version-Release number of selected component (if applicable):
python3-libs-3.3.0-1.el7.x86_64

How reproducible:
Always

Steps to Reproduce:

>>> import ssl
>>> ssl._dnsname_to_pat("*.s3.amazonaws.com").match("planet.sofiavalley.com.s3.amazonaws.com")
>>>

Actual results:


Expected results:
<_sre.SRE_Match object at 0x1474030> - the function should match the provided domain and hostname.

Additional info:

The following patch fixes the issue:

--- ssl.py.orig	2015-11-24 15:22:37.392416265 +0200
+++ ssl.py	2015-11-24 15:24:48.253470872 +0200
@@ -135,7 +135,7 @@
         if frag == '*':
             # When '*' is a fragment by itself, it matches a non-empty dotless
             # fragment.
-            pats.append('[^.]+')
+            pats.append('^.+')
         else:
             # Otherwise, '*' matches any dotless fragment.
             frag = re.escape(frag)


From Python's documentation:

[]

    Used to indicate a set of characters. In a set:

...
        Special characters lose their special meaning inside sets. For example, [(+*)] will match any of the literal characters '(', '+', '*', or ')'.

^^^^^^^^^ this is the cause of the error
Comment 1 Christian Heimes 2015-11-24 14:02:30 UTC 
The Python ssl module behaves correctly and as designed. Since recently it conforms to RFC 6125 section 6.4.3. (https://tools.ietf.org/html/rfc6125#section-6.4.3). The RFC restricts wildcards in several ways. Most importantly only one wildcard on the left-most label is supported. The suggested fix will break RFC 6125 compatibility and introduce a security bug.

By the way I'm a Python core committer, maintainer of the SSL module and the person that made the SSL module RFC 6125 conform. I closed the upstream issue https://bugs.python.org/issue25722 

I've seen multiple people complaining about cert validation errors with AWS certs. AWS violates standards here. You have to roll your own custom host name verification in order to validate AWS certs.
Comment 2 Alexander Todorov 2015-11-24 14:47:09 UTC 
Christian,
indeed you are right, sorry for the false alarm. I've managed to track down the issue further. There's also a change in httplib.py and the calling code wasn't aware of that. I've managed to create a fix for the caller:
https://github.com/s3tools/s3cmd/pull/668

Probably this one should be closed.
Comment 3 Christian Heimes 2015-11-24 14:56:15 UTC 
You are welcome!

Your fix in https://github.com/s3tools/s3cmd/pull/668/files is dangerous and insecure. It allows MITM attacks on the connection. All it takes is a valid X.509 certificate that is signed by a trusted root CA. Without hostname checks your code will trust a certificate for e.g. www.example.com.

You must add additional checks, e.g. for Subject CN and X509v3 SAN field.
Comment 4 Alexander Todorov 2015-11-24 15:01:57 UTC 
For the record:

There are custom hostname checks that conform to the AWS behavior, see match_hostname_aws() on line 74 in ConnMan.py. It's executed later in the code, after the connection has been established. I've only made sure that stock validation doesn't execute and blow things up on line 180.


Originally I hit the exception at line 180, you see the custom check performed on line 182:

    180             conn.c.connect()
    181             if conn.ssl and cfg.check_ssl_certificate and cfg.check_ssl_hostname:
    182                 conn.match_hostname()