Bug 1252422 - docker.py in new package docker-python does not work with client certificate authentication
docker.py in new package docker-python does not work with client certificate ...
Status: ASSIGNED
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: python-docker-py (Show other bugs)
7.1
x86_64 Linux
unspecified Severity high
: rc
: ---
Assigned To: Tomas Tomecek
atomic-bugs@redhat.com
: Extras
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-08-11 07:20 EDT by Jarle Bjørgeengen
Modified: 2017-08-02 02:54 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
jhonce: needinfo-


Attachments (Terms of Use)

  None (edit)
Description Jarle Bjørgeengen 2015-08-11 07:20:44 EDT
Description of problem:

We use ansible's docker module for orchestrating containers in an environment with local registry authenticated by x509 client certificates (nginx).

After introducing the package docker-python in RHEL7 (previously python-docker-py in EPEL. ) all operations with the docker.py library fails. 

Version-Release number of selected component (if applicable):


How reproducible:

Consistent. 

Steps to Reproduce:
1. Configure docker to authenticate to local docker-registry using X509 client-certificate authentication with nginx in front of docker-registry. 
2. Check that docker pull command works against the local registry 
3. Try to use docker.py library from docker-python-1.4.0-108 to the same thing. 


from docker import Client
c = Client(base_url='unix://var/run/docker.sock')
for line in c.pull('docker-registry.uio.no:8088/uio-logstash','7d81ad4', stream=True):
      print(json.dumps(json.loads(line), indent=4))

Actual results:
Traceback (most recent call last):
  File "docker-pull.py", line 4, in <module>
    for line in c.pull('docker-registry.uio.no:8088/uio-logstash','7d81ad4', stream=True):
  File "/usr/lib/python2.7/site-packages/docker/client.py", line 592, in pull
    repository, insecure=insecure_registry
  File "/usr/lib/python2.7/site-packages/docker/auth/auth.py", line 60, in resolve_repository_name
    return expand_registry_url(parts[0], insecure), parts[1]
  File "/usr/lib/python2.7/site-packages/docker/auth/auth.py", line 39, in expand_registry_url
    "HTTPS endpoint unresponsive and insecure mode isn't enabled."
docker.errors.DockerException: HTTPS endpoint unresponsive and insecure mode isn't enabled.


Expected results:

That it does the same thing as the docker pull command. 

Additional info:

The problem is not apparent in the EPEL package python-docker-py-0.4.0-3 which I believe the new docker-python from RHEL7 replaces. 

It seems likely that the reason is that docker.py from docker-python is not correctly picking up client certificates from /etc/docker/certs.d
Comment 1 Jarle Bjørgeengen 2015-08-11 07:25:18 EDT
nginx access log says (when running the above python snippet):

129.240.0.118 - - [11/Aug/2015:13:22:10 +0200] "GET /v2/ HTTP/1.1" 400 252 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.4.2.el7.x86_64" "-"
129.240.0.118 - - [11/Aug/2015:13:22:10 +0200] "GET /v1/_ping HTTP/1.1" 400 252 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.4.2.el7.x86_64" "-"


When running docker pull it says:



129.240.0.118 - - [11/Aug/2015:13:23:50 +0200] "GET /v2/ HTTP/1.1" 404 233 "-" "docker/1.7.1 go/go1.4.2 kernel/3.10.0-229.4.2.el7.x86_64 os/linux arch/amd64" "-"
129.240.0.118 - - [11/Aug/2015:13:23:50 +0200] "GET /v1/_ping HTTP/1.1" 200 2 "-" "docker/1.7.1 go/go1.4.2 kernel/3.10.0-229.4.2.el7.x86_64 os/linux arch/amd64" "-"
129.240.0.118 - - [11/Aug/2015:13:23:50 +0200] "GET /v1/_ping HTTP/1.1" 200 2 "-" "docker/1.7.1 go/go1.4.2 kernel/3.10.0-229.4.2.el7.x86_64 os/linux arch/amd64" "-"
129.240.0.118 - - [11/Aug/2015:13:23:50 +0200] "GET /v1/repositories/uio-logstash/images HTTP/1.1" 200 760 "-" "docker/1.7.1 go/go1.4.2 kernel/3.10.0-229.4.2.el7.x86_64 os/linux arch/amd64" "-"
129.240.0.118 - - [11/Aug/2015:13:23:51 +0200] "GET /v1/repositories/uio-logstash/tags HTTP/1.1" 200 158 "-" "docker/1.7.1 go/go1.4.2 kernel/3.10.0-229.4.2.el7.x86_64 os/linux arch/amd64" "-"
129.240.0.118 - - [11/Aug/2015:13:23:51 +0200] "GET /v1/images/90a9503d3cb1c249f1ec04de4f70904bcdc096a7ba76bc31de8b4ebb9405d633/ancestry HTTP/1.1" 200 476 "-" "docker/1.7.1 go/go1.4.2 kernel/3.10.0-229.4.2.el7.x86_64 os/linux
Comment 3 Jarle Bjørgeengen 2015-08-11 08:22:39 EDT
So it seems that the reason is indeed that the ping_registry()

in ../auth/auth.py 

tries to access the registry without providing the client certificates in /etc/docker/certs.d, hence it fails.

When commenting out the ping check I bangs the head against the next error: 

==========================================
Traceback (most recent call last):
  File "/root/.ansible/tmp/ansible-tmp-1439293825.39-44102419682611/docker", line 3132, in <module>
    main()
  File "/root/.ansible/tmp/ansible-tmp-1439293825.39-44102419682611/docker", line 1494, in main
    started(manager, containers, count, name)
  File "/root/.ansible/tmp/ansible-tmp-1439293825.39-44102419682611/docker", line 1354, in started
    created = manager.create_containers(delta)
  File "/root/.ansible/tmp/ansible-tmp-1439293825.39-44102419682611/docker", line 1227, in create_containers
    containers = do_create(count, params)
  File "/root/.ansible/tmp/ansible-tmp-1439293825.39-44102419682611/docker", line 1217, in do_create
    result = self.client.create_container(**params)
  File "/usr/lib/python2.7/site-packages/docker/client.py", line 237, in create_container
    volume_driver
  File "/usr/lib/python2.7/site-packages/docker/utils/utils.py", line 546, in create_container_config
    'mem_limit has been moved to host_config in API version 1.19'
docker.errors.InvalidVersion: mem_limit has been moved to host_config in API version 1.19
====================================

This seems to be https://github.com/ansible/ansible-modules-core/issues/1707, which in turn is triggered by the upstream change https://github.com/docker/docker-py/pull/644

Sigh... 

Is it possible to rollback the version of docker-py to one closer to the one that what was in EPEL until compability problems is sorted out? Another upshot is that  

https://bugzilla.redhat.com/show_bug.cgi?id=1251392

also (at least for now) disappears.
Comment 4 Jarle Bjørgeengen 2015-08-11 09:19:42 EDT
FYI: It works with.

yum remove docker-python
yum install python-pip
pip install docker.py==1.2.3
Comment 5 Daniel Walsh 2015-08-21 00:21:47 EDT
Lokesh can you look into this?
Comment 9 Daniel Walsh 2016-02-22 14:31:07 EST
Jon update please.
Comment 10 Jhon Honce 2016-03-30 12:30:39 EDT
It appears the dependencies have all been corrected now.
Comment 13 Tomas Tomecek 2016-04-20 02:52:28 EDT
We have updated python-docker-py to "python-docker-py-1.7.2-1.el7". Can you please check if the update resolves your problem? I know there was a lot of upstream development related to connecting to a secure registry.

Note You need to log in before you can comment on or make changes to this bug.