Bug 1206443 (CVE-2015-1843) - CVE-2015-1843 docker: regression of CVE-2014-5277
Summary: CVE-2015-1843 docker: regression of CVE-2014-5277
Keywords:
Status: CLOSED ERRATA
Alias: CVE-2015-1843
Product: Security Response
Classification: Other
Component: vulnerability
Version: unspecified
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Red Hat Product Security
QA Contact:
URL:
Whiteboard:
Depends On: 1206445 1206446 1206447
Blocks: 1206438
TreeView+ depends on / blocked
 
Reported: 2015-03-27 05:49 UTC by Trevor Jay
Modified: 2021-02-17 05:28 UTC (History)
13 users (show)

Fixed In Version: docker-1.5.0-28.el7_1
Doc Type: Bug Fix
Doc Text:
It was found that the fix for the CVE-2014-5277 issue was incomplete: the docker client could under certain circumstances erroneously fall back to HTTP when an HTTPS connection to a registry failed. This could allow a man-in-the-middle attacker to obtain authentication and image data from traffic sent from a client to the registry.
Clone Of:
Environment:
Last Closed: 2015-04-02 19:45:57 UTC


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2015:0776 0 normal SHIPPED_LIVE Moderate: docker security update 2015-04-02 23:31:23 UTC

Description Trevor Jay 2015-03-27 05:49:33 UTC
A regression of CVE-2014-5277 was found in the current version of the docker client on RHEL 7.1 (Docker version 1.4.1-dev, build d26b358/1.4.1).

Acknowledgements:

Red Hat would like to thank Eric Windisch of Docker Inc. for reporting this issue.

Comment 2 Trevor Jay 2015-03-27 06:22:25 UTC
Created docker tracking bugs for this issue:

Affects: fedora-all [bug 1206447]

Comment 3 Daniel Walsh 2015-03-27 12:58:19 UTC
Since we just shipped or are about to ship docker-1.5, are we all set?

Comment 4 Trevor Jay 2015-03-27 13:10:38 UTC
(In reply to Daniel Walsh from comment #3)
> Since we just shipped or are about to ship docker-1.5, are we all set?
>

Unfortunately, I don't think so.

Docker Inc. reported it to us as a RHEL specific problem. However, I built a fresh build of 1.5 straight from github that has the same issue, so it's looking more and more like it's their regression and we just consumed it.

I CC'ed you on an e-mail to upstream, but they've yet to comment.

Comment 5 Daniel Walsh 2015-03-27 13:22:36 UTC
Trevor, so the basic problem is the failover from https to http correct?

Comment 6 Trevor Jay 2015-03-27 13:28:01 UTC
(In reply to Daniel Walsh from comment #5)
> Trevor, so the basic problem is the failover from https to http correct?

Correct. If I "docker pull registry.access.redhat.com/rhel" and 443 doesn't respond, it'll try on 80.

Comment 7 Daniel Walsh 2015-03-27 13:29:48 UTC
And docker claims they turned this off?

Comment 8 Daniel Walsh 2015-03-27 13:35:19 UTC
Michal can  you take a look at the code around this to make sure we can turn off the fall back between https to http when pulling containers.

Comment 9 Michal Minar 2015-03-27 14:38:42 UTC
I've just reproduced this. Tested on RHEL7 with a private repository accessible on port 80.

Upstream registry:

    $ docker pull miminarnb.virbr0:5004/miminar/hwdata-check
    FATA[0004] Error response from daemon: invalid registry endpoint https://miminarnb.virbr0:5004/v0/: unable to ping registry endpoint https://miminarnb.virbr0:5004/v0/
    v2 ping attempt failed with error: Get https://miminarnb.virbr0:5004/v2/: EOF
    v1 ping attempt failed with error: Get https://miminarnb.virbr0:5004/v1/_ping: EOF. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add `--insecure-registry miminarnb.virbr0:5004` to the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag; simply place the CA certificate at /etc/docker/certs.d/miminarnb.virbr0:5004/ca.crt

Current rhatdan/1.6 rhel branch and recent docker builds.
- without any additional registry specified, it behaves the same as upstream registry
- with --add-registry=miminarnb.virbr0:5004 it pulls the given registry

So I don't think it's an upstream issue but a regression caused by registry
patches. I'll investigate further.

Comment 10 Trevor Jay 2015-03-27 14:46:55 UTC
Here's what I did with a fresh upstream build (Docker version 1.5.0-dev, build da5c863):

I set modified my /etc/hosts:

    127.0.0.1   registry.access.redhat.com

and started a netcat on port 80:

    sudo nc -l 80

I then:

    sudo ./docker-1.5.0-dev -d

and:

    sudo ./docker-1.5.0-dev pull registry.access.redhat.com/rhel

The result was fallback to HTTP (from netcat's output):

    ...
    GET /v2/ HTTP/1.1
    Host: registry.access.redhat.com
    User-Agent: docker/1.5.0-dev go/go1.4.2 git-commit/da5c863 kernel/3.17.4-301.fc21.x86_64 os/linux arch/amd64
    Accept-Encoding: gzip
    ...

Given my *and* Michal's case. Perhaps upstream does the right thing initially (with a private repo) but still fails on fallback?

Comment 13 Michal Minar 2015-03-27 15:27:50 UTC
Trevor,
I've just found out why in your case the registry is accessed on port 80:
  
  https://github.com/docker/docker/blob/master/registry/config.go#L93

Localhost is considered insecure by default. I don't think that's a security issue.

Comment 15 Trevor Jay 2015-03-28 00:30:15 UTC
(In reply to Michal Minar from comment #13)
> Trevor,
> I've just found out why in your case the registry is accessed on port 80:
>   
>   https://github.com/docker/docker/blob/master/registry/config.go#L93
> 
> Localhost is considered insecure by default. I don't think that's a security
> issue.

You are correct. I just retried using a second machine as the pull target and there was no incorrect fallback.

Comment 19 Daniel Walsh 2015-03-31 13:21:58 UTC
Well the fix will go into docker-1.6, so we should have all of the pull requests there.

Comment 20 Michal Minar 2015-03-31 14:41:32 UTC
I don't quite follow. This fix is RH only. Upstream is unaffected by this because it doesn't have our registry patches merged and it won't have.

Currently I'm working on rebasing all our work that happened since the last update of PR 10411 on top of it. And I plan to just update the PR with it. I don't see a reason for splitting that into new pull requests - all of them containing also PR 10411 - only to see them being closed later. Am I missing something?

Comment 21 Daniel Walsh 2015-03-31 15:42:54 UTC
Michal the question is whether or not the login process sends the username/passwords intended for  docker.io to registry.access.redhat.com.

Comment 22 Trevor Jay 2015-04-01 00:14:57 UTC
(In reply to Daniel Walsh from comment #21)
> Michal the question is whether or not the login process sends the
> username/passwords intended for  docker.io to registry.access.redhat.com.
>

No version I have tested after Michal's patch mid-february sends the credentials. I have confirmed with Docker Inc. that they were testing with docker-1.4.1-37.el7.

Comment 25 Trevor Jay 2015-04-02 13:51:01 UTC
An expanded explaination of my methodology for testing the HTTP fallback issue:

  1) On a second machine or VM with port 80 accessible and *nothing* running on port 443, I run:

    sudo nc -l 80

  2) On the machine hosting a fresh docker install, add an entry to /etc/hosts:

    X.X.X.X registry.access.redhat.com

  where X.X.X.X is the IP address of the VM or machine running netcat.

  3) I then start docker:

    sudo service docker restart

  4) And attempt:

    sudo docker pull registry.access.redhat.com/rhel

The expected behavior is that the netcat recieves *no* traffic and the pull attempt fails. The problem is still present if any request are received on the running netcat instance.

Comment 26 Trevor Jay 2015-04-02 13:52:24 UTC
I will now confirm this does not regress the authorization header issue. Once I've done that, I'll provide a quick write-up of the method that worked for me.

Comment 27 Michal Minar 2015-04-02 14:10:59 UTC
See *Doc Text* field for errata release notes.

Comment 28 Michal Minar 2015-04-02 14:21:59 UTC
Trevo(In reply to Trevor Jay from comment #25)
> An expanded explaination of my methodology for testing the HTTP fallback
> issue:
> 
>   1) On a second machine or VM with port 80 accessible and *nothing* running
> on port 443, I run:
> 

Important note:
test with --add-registry=registry.access.redhat.com passed to daemon and then without. It should behave the same.

However with --insecure-registry=registry.access.redhat.com it should fall back to port 80 regardless of --add-registry options.

Comment 29 Michal Minar 2015-04-02 14:35:29 UTC
And one more thing - test also with unqualified repository:
  $ docker pull registry.access.redhat.com/rhel

So there's actually 8 combinations:

  A I FQ  fall back to port 80?
  N N N   N
  N N Y   N
  N Y N   Y      (docker.io shall redirect)
  N Y Y   Y
  Y N N   N
  Y N Y   N
  Y Y N   Y      (docker.io shall redirect)
  Y Y Y   Y

A - with --add-registry=registry.access.redhat.com/rhel
I - with --insecure-registry=registry.access.redhat.com/rhel
FQ - with repository fully qualified

Comment 30 Michal Minar 2015-04-02 14:38:05 UTC
> A - with --add-registry=registry.access.redhat.com/rhel
> I - with --insecure-registry=registry.access.redhat.com/rhel
> FQ - with repository fully qualified
Should obviously be:
A - with --add-registry=registry.access.redhat.com
I - with --insecure-registry=registry.access.redhat.com
FQ - with repository fully qualified

Now I know, what I like about github and trello - *editable comments*!

Comment 31 Michal Minar 2015-04-02 14:55:04 UTC
Another update - we don't follow redirects from official registry.

  A I FQ  fall back to port 80?
  N N N   N
  N N Y   N
  N Y N   N      (docker.io shall redirect but daemon won't follow)
  N Y Y   Y
  Y N N   N
  Y N Y   N
  Y Y N   Y
  Y Y Y   Y

A - with --add-registry=registry.access.redhat.com
I - with --insecure-registry=registry.access.redhat.com
FQ - with repository fully qualified

Comment 32 Trevor Jay 2015-04-02 14:58:10 UTC
To test Authorization header regression:

  On Machine a:

  1) Install python pip.
  2) Install mitmproxy:
      
    pip install mitmproxy

  3) Download headers.py (attached to this comment)
  4) Generate SSL certs:

    openssl genrsa -out cert.key 8192
    openssl req -new -x509 -key cert.key -out cert.crt
    cat cert.key cert.crt > cert.pem

  Remember to make the Common Name registry.access.redhat.com

  5) Copy cert.crt to /etc/docker/certs.d/registry.access.redhat.com/ca.crt on machine b (the machine that will host docker). Make sure it is world readable.
  6) Run the headers.py script as a mitmdump module:
  
    sudo mitmdump --cert=cert.pem -p 443 -s headers.py --http-form-in relative --destation-server https://registry.access.redhat.com

  On machine b:

  1) start docker:

    sudo service docker start

  2) Login to the Docker hub

    sudo docker login

  You will need to sign up for an account on website.

  2) Attempt to pull rhel:

    sudo docker pull registry.access.redhat.com/rhel

This will fail because we're failing to spoof access.redhat.com, but should get far enough that Authorization headers are exchanged. They should contain "Og==" and *not* the base64 information contained in the .dockercfg file.

Comment 33 Trevor Jay 2015-04-02 14:59:44 UTC
headers.py can be found here:

    http://pastebin.test.redhat.com/273954

Comment 34 Trevor Jay 2015-04-02 15:53:35 UTC
(In reply to Michal Minar from comment #31)
> Another update - we don't follow redirects from official registry.
> 
>   A I FQ  fall back to port 80?
>   N N N   N
>   N N Y   N
>   N Y N   N      (docker.io shall redirect but daemon won't follow)
>   N Y Y   Y
>   Y N N   N
>   Y N Y   N
>   Y Y N   Y
>   Y Y Y   Y
> 
> A - with --add-registry=registry.access.redhat.com
> I - with --insecure-registry=registry.access.redhat.com
> FQ - with repository fully qualified

I have confirmed this matrix.

Comment 35 Trevor Jay 2015-04-02 16:33:14 UTC
To use mitmdump on Fed 21:

    sudo yum install libxml2-devel libxslt-devel python-devel libffi-devel openssl-devel openssl
    sudo pip install mitmproxy

you also need to change --destination-server to -U

Comment 36 Trevor Jay 2015-04-02 16:36:50 UTC
To test with --insecure-registry=registry.access.redhat.com and mitmdump, the port will need to be swapped to 80 and the protocol to HTTP. For example:

    sudo mitmdump --cert=cert.pem -p 80 -s headers.py --http-form-in relative -U http://registry.access.redhat.com

Remember, you only need to use this form when applying the --insecure option.

You will also need to append the cert.crt from before the the global cert.pem:

    sudo cat cert.crt >> /etc/pki/tls/cert.pem

Comment 38 errata-xmlrpc 2015-04-02 19:31:41 UTC
This issue has been addressed in the following products:

  RHEL Extras for RHEL-7

Via RHSA-2015:0776 https://rhn.redhat.com/errata/RHSA-2015-0776.html

Comment 39 Trevor Jay 2015-04-02 19:45:57 UTC
In rush must not have been moved to verified by QE (that is moved to the verified state, QE work was marked as completed on the errata). Manually closing.


Note You need to log in before you can comment on or make changes to this bug.