Bug 1230710

Summary: Can't create container from external registry
Product: Red Hat Satellite Reporter: Elyézer Rezende <erezende>
Component: Container ManagementAssignee: Daniel Lobato Garcia <dlobatog>
Status: CLOSED ERRATA QA Contact: Lukas Pramuk <lpramuk>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.1.0CC: abradshaw, dherrman, ehelms, lpramuk, mmccune, oshtaier
Target Milestone: UnspecifiedKeywords: Regression, Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
URL: http://projects.theforeman.org/issues/14181
Whiteboard:
Fixed In Version: rubygem-foreman_docker-2.0.1.4-1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-07-27 09:15:57 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Elyézer Rezende 2015-06-11 12:37:31 UTC
Description of problem:
Is not possible to create a new container using an external registry (registry.access.redhat.com). The "New Container" wizard does not completes and neither shows an error message.

Version-Release number of selected component (if applicable):
S8C1

How reproducible:
Aways

Steps to Reproduce:
1. Create an external registry using registry.access.redhat.com as URL
2. Go to New Container wizard and select an already created docker compute resouce
3. In the next step select the "External Registry" tab, search for rhel and select one rhel image.
4. Proceed until the end of the wizard and try to finish it.

Actual results:
The wizard does not complete and neither shows an error message

Expected results:
The wizard should complete without any issue

Additional info:
Relevant log grabbed when trying to complete the wizard

==> /var/log/foreman/production.log <==
2015-06-11 08:36:08 [I] Processing by Containers::StepsController#update as HTML
2015-06-11 08:36:08 [I]   Parameters: {"utf8"=>"✓", "authenticity_token"=>"ISa+4eV4eHAuMQOVyc021ctGcsguhtfcpptdYVZvvpw=", "docker_container_wizard_states_environment"=>{"tty"=>"1", "attach_stdin"=>"1", "attach_stdout"=>"1", "attach_stderr"=>"1"}, "wizard_state_id"=>"3", "id"=>"environment"}

==> /var/log/messages <==
Jun 11 08:36:08 qe-sat6-rhel71 docker: time="2015-06-11T08:36:08-04:00" level=info msg="POST /v1.15/images/create?fromImage=%3A%2Frhel7%3A7.1-6"
Jun 11 08:36:08 qe-sat6-rhel71 docker: time="2015-06-11T08:36:08-04:00" level=info msg="+job pull(:/rhel7, 7.1-6)"
Jun 11 08:36:08 qe-sat6-rhel71 docker: time="2015-06-11T08:36:08-04:00" level=info msg="+job resolve_repository(:/rhel7)"
Jun 11 08:36:08 qe-sat6-rhel71 docker: time="2015-06-11T08:36:08-04:00" level=info msg="-job resolve_repository(:/rhel7) = OK (0)"
Jun 11 08:36:08 qe-sat6-rhel71 docker: invalid registry endpoint https://:/v0/: unable to ping registry endpoint https://:/v0/
Jun 11 08:36:08 qe-sat6-rhel71 docker: v2 ping attempt failed with error: Get https://:/v2/: dial tcp :0: connection refused
Jun 11 08:36:08 qe-sat6-rhel71 docker: v1 ping attempt failed with error: Get https://:/v1/_ping: dial tcp :0: connection refused. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add `--insecure-registry :` to the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag; simply place the CA certificate at /etc/docker/certs.d/:/ca.crt
Jun 11 08:36:08 qe-sat6-rhel71 docker: time="2015-06-11T08:36:08-04:00" level=info msg="-job pull(:/rhel7, 7.1-6) = ERR (1)"
Jun 11 08:36:08 qe-sat6-rhel71 docker: time="2015-06-11T08:36:08-04:00" level=error msg="Handler for POST /images/create returned error:  v1 ping attempt failed with error: Get https://:/v1/_ping: dial tcp :0: connection refused. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add `--insecure-registry :` to the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag; simply place the CA certificate at /etc/docker/certs.d/:/ca.crt"
Jun 11 08:36:08 qe-sat6-rhel71 docker: time="2015-06-11T08:36:08-04:00" level=error msg="HTTP Error: statusCode=500  v1 ping attempt failed with error: Get https://:/v1/_ping: dial tcp :0: connection refused. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add `--insecure-registry :` to the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag; simply place the CA certificate at /etc/docker/certs.d/:/ca.crt"

==> /var/log/foreman/production.log <==
2015-06-11 08:36:08 [I] Failed to save:
2015-06-11 08:36:08 [I]   Rendered /opt/rh/ruby193/root/usr/share/gems/gems/foreman_docker-1.2.0.14/app/views/foreman_docker/common_parameters/_environment_variable.html.erb (1.2ms)
2015-06-11 08:36:08 [I]   Rendered /opt/rh/ruby193/root/usr/share/gems/gems/foreman_docker-1.2.0.14/app/views/containers/steps/_form_buttons.html.erb (0.8ms)
2015-06-11 08:36:08 [I]   Rendered /opt/rh/ruby193/root/usr/share/gems/gems/foreman_docker-1.2.0.14/app/views/containers/steps/_title.html.erb (192.5ms)
2015-06-11 08:36:08 [I]   Rendered /opt/rh/ruby193/root/usr/share/gems/gems/foreman_docker-1.2.0.14/app/views/containers/steps/environment.html.erb within layouts/application (192.9ms)
2015-06-11 08:36:08 [I]   Rendered home/_submenu.html.erb (2.0ms)
2015-06-11 08:36:08 [I]   Rendered home/_user_dropdown.html.erb (6.1ms)
2015-06-11 08:36:08 [I] Read fragment views/tabs_and_title_records-3 (0.1ms)
2015-06-11 08:36:09 [I]   Rendered home/_organization_dropdown.html.erb (373.2ms)
2015-06-11 08:36:09 [I]   Rendered home/_location_dropdown.html.erb (36.4ms)
2015-06-11 08:36:09 [I]   Rendered home/_org_switcher.html.erb (410.5ms)
2015-06-11 08:36:09 [I]   Rendered home/_submenu.html.erb (4.3ms)
2015-06-11 08:36:09 [I]   Rendered home/_submenu.html.erb (4.7ms)
2015-06-11 08:36:09 [I]   Rendered home/_submenu.html.erb (2.2ms)
2015-06-11 08:36:09 [I]   Rendered home/_submenu.html.erb (4.4ms)
2015-06-11 08:36:09 [I]   Rendered home/_submenu.html.erb (3.3ms)
2015-06-11 08:36:09 [I]   Rendered home/_submenu.html.erb (2.7ms)
2015-06-11 08:36:09 [I]   Rendered home/_submenu.html.erb (1.8ms)
2015-06-11 08:36:09 [I]   Rendered home/_submenu.html.erb (3.9ms)
2015-06-11 08:36:09 [I] Write fragment views/tabs_and_title_records-3 (1.1ms)
2015-06-11 08:36:09 [I]   Rendered home/_topbar.html.erb (454.0ms)
2015-06-11 08:36:09 [I]   Rendered layouts/base.html.erb (455.5ms)
2015-06-11 08:36:09 [I] Completed 200 OK in 711ms (Views: 634.9ms | ActiveRecord: 27.8ms)

Comment 2 Mike McCune 2015-06-12 17:02:47 UTC
The error here was that the registry was specified as:

registry.access.redhat.com

vs:

http://registry.access.redhat.com

the protocol is required.

We should validate the field input to verify that it is a URL

Comment 3 Ade Bradshaw 2015-08-16 13:17:46 UTC
I also came across this issue, so I added the http://registry.access.redhat.com but this also failed, then I added 

https://registry.access.redhat.com  

This seemed to work much better (from looking at the logs) but then I ran into a different issue

Maybe we should add a verification step on the field, the one that requires a protocol

Comment 4 Dirk Herrmann 2015-12-09 19:37:11 UTC
Tested it today using Satellite 6.1.4 and different docker client versions. It works using https with docker 1.7.1 and in contrast to katello repos also using current docker version (1.8.2). Using http fails with both versions. Tested both WebUI and hammer CLI.

Comment 5 Elyézer Rezende 2016-01-21 18:12:11 UTC
A registry may be available only under https. I think registry.access.redhat.com is the case, this can explain why it is failing for http.

Comment 6 Daniel Lobato Garcia 2016-03-14 13:04:05 UTC
Fixed under - https://github.com/theforeman/foreman-docker/pull/142 to be merged

Comment 7 Daniel Lobato Garcia 2016-03-17 11:03:39 UTC
Fix merged upstream

Comment 10 Lukas Pramuk 2016-03-24 10:25:02 UTC
FailedQA.
@Sat6.2.0-Beta-Snap5

This is a showstopper, cannot create external registry anymore
While in Snap4 I was able to create and even search external registries
such as https://registry.hub.docker.com or https://registry.access.redhat.com

Comment 11 Lukas Pramuk 2016-03-24 10:30:48 UTC
2016-03-24 05:18:50 [app] [I] Failed to save: Unable to log in to this Docker Registry - Expected([200, 201, 202, 203, 204, 304]) <=> Actual(404 Not Found)

Similar message is also shown at UI.
(What about 301 and 302? these should also be expected...)

Comment 13 Daniel Lobato Garcia 2016-03-29 11:30:53 UTC
Lukas,

301 shouldn't really be expected. Satellite 6 only supports Docker Registry API v1 for external registries (https://registry.hub.docker.com/ changed to v2 very recently).

If you want to add registries, ensure they're v1 first. The patch I'm working on uses basic authentication with v1 registries to $REGISTRYURL/v1/users. The current call to '/auth' is wrong as it authenticates to the docker host, not the registry. 

Thanks for spotting that.

Comment 14 Daniel Lobato Garcia 2016-03-29 13:09:25 UTC
Now under review at https://github.com/theforeman/foreman-docker/pull/148

Comment 15 Lukas Pramuk 2016-04-06 18:48:28 UTC
FailedQA.

@Sat6.2.0-Beta-Snap6.2
tfm-rubygem-foreman_docker-2.0.1.3-1.el7sat.noarch

2016-04-06 14:25:35 [app] [I] Started POST "/registries" for <CLIENT_IP> at 2016-04-06 14:25:35 -0400
2016-04-06 14:25:35 [app] [I] Processing by RegistriesController#create as HTML
2016-04-06 14:25:35 [app] [I]   Parameters: {"utf8"=>"✓", "authenticity_token"=>"RLUwHJsw1ACjnBKnNMtHvjbPxc88aApYjhoxI4uhN54=", "docker_registry"=>{"name"=>"hub.docker.com", "url"=>"https://registry.hub.docker.com/", "description"=>"", "username"=>"", "password"=>"[FILTERED]", "location_ids"=>[""], "organization_ids"=>["", "1"]}, "commit"=>"Submit"}
2016-04-06 14:25:35 [app] [I] Failed to save: Unable to log in to this Docker Registry - Expected([200, 201, 202, 203, 204, 304]) <=> Actual(503 Service Unavailable)


https://registry.hub.docker.com/ >>> Actual(503 Service Unavailable)

https://registry.access.redhat.com/ >>> Actual(404 Not Found)

Comment 16 Daniel Lobato Garcia 2016-04-07 06:45:18 UTC
I'm sorry but how have you even tested this? The second cherry-pick that fixed that wasn't even made yet. 

-----------------------

Notice:

https://github.com/theforeman/foreman-docker/pull/148 

is not in here:

https://gitlab.sat.lab.tlv.redhat.com/satellite6/foreman_docker/blob/SATELLITE-6.2.0/app/models/docker_registry.rb

-----------------------

I don't know why this was set to on_qa w/o the second cherry-pick, but it should be definitely on POST.

Comment 17 Lukas Pramuk 2016-04-18 11:07:10 UTC
VERIFIED.

@Sat6.2.0-Beta-Snap8
tfm-rubygem-foreman_docker-2.0.1.4-1.el7sat.noarch

I was able to create an external registry. (Step 1)

I was able to search for rhel images using external registry (Step 3) only if username(password) didnt contain special char
>>> I guess username/password is not urlencoded - lets have another bz for this (not regression)

I was able to create a container based on rhel image using external registry (Step 4) <<< SUCCESS
>>> Though I wasn't able to power it on (another bz)

Comment 19 errata-xmlrpc 2016-07-27 09:15:57 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501