Bug 1173041 (CVE-2014-9365)

Created python tracking bugs for this issue:

Affects: fedora-all [bug 1173043]

Created jython tracking bugs for this issue:

Affects: epel-7 [bug 1173046]
Affects: epel-5 [bug 1173047]

Created python26 tracking bugs for this issue:

Affects: epel-5 [bug 1173045]

Created python3 tracking bugs for this issue:

Affects: fedora-all [bug 1173044]

This CVE is related to Python Enhancement Proposal PEP 476 "Enabling certificate verification by default for stdlib http clients":

https://www.python.org/dev/peps/pep-0476/

Python standard library was changed via this proposal to make http client implementation perform server SSL/TLS certificate verification by default (i.e. certificates is checked to ensure it was issued by one of the trusted certificate authorities and that connection host name matches name list in subject's CommonName or one of the subjectAlternateNames).  This change affects urllib, urllib2, http, and httplib modules, and other modules that use one of the mentioned modules (e.g. xmlrpclib).

Upstream bugs linked by PEP 476:

http://bugs.python.org/issue22366
http://bugs.python.org/issue22417

This change depends on another PEP 466 "Network Security Enhancements for Python 2.7.x":

https://www.python.org/dev/peps/pep-0466/

The PEP 466 covers a set of not directly related enhancement, one of them being update of the ssl module.  The update adds few features relevant here - SSLContext support, which defines connections parameters and verification done during handshake, name checking functions (previously packages for python 2.7 via a separate python-backports-ssl_match_hostname package), and functionality for loading system trust store.

Upstream bug for the ssl module part of PEP 466:

http://bugs.python.org/issue21308

Following LWN article provides good overview of the changes and motivations:

https://lwn.net/Articles/611243/

The key concern here is how to handle operating environments where defaulting to validation of HTTPS certificates is currently infeasible. Upstream we opted for the relatively straightforward solution of a hard break in 2.7.9 and 3.4.3, with a sitecustomize based solution for reverting to the old behaviour in a given environment: https://www.python.org/dev/peps/pep-0476/#opting-out

Further discussion of the backport in the Linux distribution case showed that that approach wasn't going to be sufficient for redistributors that offer strong stability guarantees for controlled environments: we need a way to opt-out by default that doesn't rely on sitecustomize in order to provide an "opt-in" transition period that decouples the Python upgrade decision from the activation of default HTTPS certification verification in Python based scripts and applications. 

The relevant upstream bug seeking a standardised approach to this is at https://bugs.python.org/issue23857

The two main candidates are:

* introduce a Python SSL configuration file under /etc that is read by the SSL module at import time. The default behaviour in the absence of that file would be to verify HTTPS certifications. If the file was present, there would be three options: verification forced off, verification forced on, and setting delegated to the platform vendor.

* use an environment variable with the same settings as suggested for the configuration file. The main problem with this is that the -E switch will ignore any environment variable based solution, so passing -E would also have the effect of opting back in to verifying HTTPS certificates by default.

While the latter approach was previously used for the Python hash randomisation change, the interaction with -E is likely to be far more problematic in this case.

So my recommendation is to proceed with the configuration file based approach, and write an upstream Python Enhancement Proposal seeking standardisation of the name, file format and behaviour for redistributors that opt-in to providing this behaviour. This would be similar to the existing PEP providing recommendations for management of the Python symlink: https://www.python.org/dev/peps/pep-0394/

First draft of an upstream recommendations PEP: https://hg.python.org/peps/rev/85bc7f13b295

I listed Robert as a co-author, since the suggested design is actually his, I just added all the surrounding rationale.

I referenced the draft PEP from an ongoing python-dev discussion (primarily with Marc-Andre Lemburg of eGenix): https://mail.python.org/pipermail/python-dev/2015-May/139922.html

The key technical point raised in that discussion so far is that any patch we apply should be "virtualenv aware", as it may be desirable to be able to have the certificate verification default behaviour be different in virtual environments.

We can detect that easily enough by looking for "sys.real_prefix": http://stackoverflow.com/questions/1871549/python-determine-if-running-inside-virtualenv

My initial inclination is to just not read the global config file when running inside a virtual environment, and hence always apply the upstream default of verifying HTTPS certificates.

The alternative would be to always read the configuration file, but have separate global settings for running in the system Python directly vs running in virtual environments.

The latter approach does have the virtue of offering a phased migration path that looks like:

* initial default matches upstream 2.7.5 (no HTTPS certificate verification)
* virtualenv default setting is updated to enable verification
* system Python default setting updated to enable verification

Configuration of the setting within a *specific* virtual environment would be out of scope for this BZ, but rather a discussion to be had with the upstream virtualenv project: https://virtualenv.pypa.io/en/latest/

Pre-formatted version of the PEP 493 proposal (source is linked from comment 14):

https://www.python.org/dev/peps/pep-0493/

Link to python-dev discussion thread as archived on gmane.  It's the same discussion as linked in comment 15, just easier to follow when discussion spans multiple months.

http://thread.gmane.org/gmane.comp.python.devel/152846/

(In reply to Nick Coghlan from comment #15)
> My initial inclination is to just not read the global config file when
> running inside a virtual environment, and hence always apply the upstream
> default of verifying HTTPS certificates.

It sounds like this would lead to the same problems we're trying to address with the global opt-in / opt-out switch - it forces the change of behaviour inside virtualenvs with no easy way to revert it.

AFAICS, virtualenv uses system stdlib and does not make a separate copy of it in ENV.  If this should be extra virtualenv friendly, ssl can look for the config file in ENV/etc before using /etc file.  However, this does not seem very important and may even be confusing (e.g. why should verification configuration be per virtualenv when there's no support for per virtualenv CA bundle?).  What's the argument for treating virtualenv as special?

This could be one argument.

Lets say we have system with following settings in the config file:

[https]
verify=enable
virtualenv_verify=disable

Now those of users who aren't administrators may create their virtualenv in which the python will not verify certificates by default.

Or we could just simply support users config files in ~/.python/cert-verification.conf?

The proposed patch we currently have can be extended in many ways - it can be made virtualenv aware, it can support per-user configuration, it can provide per-site configuration capabilities.  The original vision here was to give administrators an easy way to set what should be the global default.  It was not meant to provide per-application or per-invocation setting.  Global or user-specific config does not provide sufficient granularity.

Anyone building virtualenv only to disable verification for a specific application can use out-out mechanisms documented in PEP 476.  If we find it important to provide per-application or per-invocation setting that does not require any patching, I'd consider exploring mechanisms that are better suited for that - it seems other vendors shipping python are already looking at using PYTHONHTTPSVERIFY environment variable.

Right, after a couple of iterations of the discussion with MAL on python-dev, I finally understood what problem he was trying to solve, and how it differed from our problem as a platform provider.

As a result, I pushed an update which describes both approaches (a config file and an environment variable), and the contexts where they may be appropriate: https://hg.python.org/peps/rev/b395246d0af7

For our use case of providing this as an opt-in feature on a per-system basis atop 2.7.5, I think the config file recommendation is good way to go. Relative to Robert's draft patch, the changes in the current PEP draft are:

* Using .cfg as the filename extension (based on pyvenv.cfg as the venv config extension in Python 3)
* Adding "verify_in_virtualenv" as an optional override for the main "verify" setting if different behaviour is wanted in virtual environments

At the system Python level, I don't think supporting the environment variable *as well* is worth the extra complexity it introduces, especially given the availability of other tools like containers.

However, I'm not sure that's the right answer for the Python 2.7 software collection, though. The software collection potentially seems closer to MAL's PyRun use case, where the environment variable based solution might be a better fit.

(In reply to Nick Coghlan from comment #20)
> Relative to Robert's draft patch, the changes in the current PEP draft are:

* When config file can not be read, or does not contain [https] section or verify option, or verify option value is unknown, PEP 493 defaults to enabled verification rather than platform default.

This may not be obvious from the patch that went into the upstream bug as platform default there is enable, hence these error paths also use enable.  I don't view fallback to platform default as completely unreasonable, at least when config file can be opened and does not contain [https] section (e.g. it's all commented out).  Cases when config can not be opened or verify has invalid value are more problematic.

> * Adding "verify_in_virtualenv" as an optional override for the main
> "verify" setting if different behaviour is wanted in virtual environments

I do not find much explanation why this is worth it.  I believe it's coming from here:

https://mail.python.org/pipermail/python-dev/2015-May/139932.html

* Most applications use some kind of virtualenv
  Python environment to run the code. These are
  typically isolated from the system Python installation
  and so wouldn't want to use a system wide global INI
  neither.

As noted above, virtualenvs use system Python stdlib including ssl module, which is going to use system CA bundle.  So it seems unexpected that there's special handling only for the system config file.

> At the system Python level, I don't think supporting the environment
> variable *as well* is worth the extra complexity it introduces,

The environment variable seems less interesting while defaulting to disabled verification (i.e. as opt-in mechanism), but is more useful when default is enabled and it can be used to easily opt-out in specific cases.

PEP 493 suggests both mechanisms can be implemented by distributors, but does not suggest how to combine them.  I do not believe combined approach is actually complex and this is how it should be done:

def _get_https_context_factory():
    # check environment variable first
    config_setting = os.environ.get('PYTHONHTTPSVERIFY')
    if config_setting is not None:
        if config_setting == '0':
            return _create_unverified_context
        return create_default_context

    # use configured or platform default if not overridden by environment
    config_file = '/etc/python/cert-verification.conf'
    context_factories = {
    ....

> especially given the availability of other tools like containers.

This is why I view verify_in_virtualenv as not really necessary.  virtualenv is container for python application and hence container specific overrides (sitecustomize or PYTHONHTTPSVERIFY if supported) seem more appropriate than global setting affecting all containers.

OK, I like the idea of dropping the global virtualenv setting in favour of documenting how to combine the configuration file with the environment variable. I'll update the PEP accordingly.

Update pushed at https://hg.python.org/peps/rev/b5e0791f27bf

Notable changes:

* PYTHONHTTPSVERIFY is now presented first
* Separate verify_in_virtualenv setting is gone
* New section covers combining the two recommendations
* Comments about the attack surface area have been removed
* The platform default setting is now used whenever the setting can't be accessed (I originally wrote the example implementation assuming 2.7.9 as the basis for the default behaviour, rather than 2.7.5. I missed a couple of logic branches in the commit linked above, fixed here: https://hg.python.org/peps/rev/19c17ec83bcd)

Minor fixes:

* example code now uses the ".cfg" extension
* the unused "section" variable has been removed from the example code
* the default factory is now always read from the mapping rather than being separately hard coded in the final mapping lookup

Thank you, the above changes look good to me.  What are the next steps that need to happen upstream to get the PEP out of the draft stage?

Assigning a BDFL-Delegate to make the final decision, and having them say Yes :)

Marc-Andre Lemburg from eGenix was the other core def most interested in resolving this problem, so I'll ask him if he's willing to handle it, and then make the suggestion to Guido.

I posted the request for pronouncement to python-dev: https://mail.python.org/pipermail/python-dev/2015-July/140649.html

There's one slightly technical change from the previous drafts: moving the environment variable name and the configuration file name up to being stored in module globals so it becomes trivial to check for the presence of these capabilities.

Gmane archive link for the python-dev thread linked in commnent 26 above:

http://thread.gmane.org/gmane.comp.python.devel/153831

This issue has been addressed in the following products:

  Red Hat Enterprise Linux 7

Via RHSA-2015:2101 https://rhn.redhat.com/errata/RHSA-2015-2101.html

This issue has been addressed in the following products:

  Red Hat Software Collections for Red Hat Enterprise Linux 6
  Red Hat Software Collections for Red Hat Enterprise Linux 7
  Red Hat Software Collections for Red Hat Enterprise Linux 6.7 EUS
  Red Hat Software Collections for Red Hat Enterprise Linux 6.6 EUS
  Red Hat Software Collections for Red Hat Enterprise Linux 7.1 EUS
  Red Hat Software Collections for Red Hat Enterprise Linux 7.2 EUS

Via RHSA-2016:1166 https://access.redhat.com/errata/RHSA-2016:1166

This issue has been addressed in the following products:

  Red Hat Software Collections for Red Hat Enterprise Linux 6
  Red Hat Software Collections for Red Hat Enterprise Linux 7
  Red Hat Software Collections for Red Hat Enterprise Linux 6.7 EUS
  Red Hat Software Collections for Red Hat Enterprise Linux 7.3 EUS

Via RHSA-2017:1162 https://access.redhat.com/errata/RHSA-2017:1162

This issue has been addressed in the following products:

  Red Hat Enterprise Linux 7

Via RHSA-2017:1868 https://access.redhat.com/errata/RHSA-2017:1868