1230710 – Can't create container from external registry

Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1230710 - Can't create container from external registry

Summary: Can't create container from external registry

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Satellite
Classification:	Red Hat
Component:	Container Management
Sub Component:
Version:	6.1.0
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	high
Target Milestone:	Unspecified
Assignee:	Daniel Lobato Garcia
QA Contact:	Lukas Pramuk
Docs Contact:
URL:	http://projects.theforeman.org/issues...
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2015-06-11 12:37 UTC by Elyézer Rezende
Modified:	2019-09-26 14:46 UTC (History)
CC List:	6 users (show)
Fixed In Version:	rubygem-foreman_docker-2.0.1.4-1
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:	2016-07-27 09:15:57 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Foreman Issue Tracker	14181	0	None	None	None	2016-04-01 14:46:09 UTC
Red Hat Product Errata	RHBA-2016:1501	0	normal	SHIPPED_LIVE	Red Hat Satellite 6.2 Capsule and Server	2016-07-27 12:28:58 UTC

Description Elyézer Rezende 2015-06-11 12:37:31 UTC

Description of problem:
Is not possible to create a new container using an external registry (registry.access.redhat.com). The "New Container" wizard does not completes and neither shows an error message.

Version-Release number of selected component (if applicable):
S8C1

How reproducible:
Aways

Steps to Reproduce:
1. Create an external registry using registry.access.redhat.com as URL
2. Go to New Container wizard and select an already created docker compute resouce
3. In the next step select the "External Registry" tab, search for rhel and select one rhel image.
4. Proceed until the end of the wizard and try to finish it.

Actual results:
The wizard does not complete and neither shows an error message

Expected results:
The wizard should complete without any issue

Additional info:
Relevant log grabbed when trying to complete the wizard

==> /var/log/foreman/production.log <==
2015-06-11 08:36:08 [I] Processing by Containers::StepsController#update as HTML
2015-06-11 08:36:08 [I]   Parameters: {"utf8"=>"✓", "authenticity_token"=>"ISa+4eV4eHAuMQOVyc021ctGcsguhtfcpptdYVZvvpw=", "docker_container_wizard_states_environment"=>{"tty"=>"1", "attach_stdin"=>"1", "attach_stdout"=>"1", "attach_stderr"=>"1"}, "wizard_state_id"=>"3", "id"=>"environment"}

==> /var/log/messages <==
Jun 11 08:36:08 qe-sat6-rhel71 docker: time="2015-06-11T08:36:08-04:00" level=info msg="POST /v1.15/images/create?fromImage=%3A%2Frhel7%3A7.1-6"
Jun 11 08:36:08 qe-sat6-rhel71 docker: time="2015-06-11T08:36:08-04:00" level=info msg="+job pull(:/rhel7, 7.1-6)"
Jun 11 08:36:08 qe-sat6-rhel71 docker: time="2015-06-11T08:36:08-04:00" level=info msg="+job resolve_repository(:/rhel7)"
Jun 11 08:36:08 qe-sat6-rhel71 docker: time="2015-06-11T08:36:08-04:00" level=info msg="-job resolve_repository(:/rhel7) = OK (0)"
Jun 11 08:36:08 qe-sat6-rhel71 docker: invalid registry endpoint https://:/v0/: unable to ping registry endpoint https://:/v0/
Jun 11 08:36:08 qe-sat6-rhel71 docker: v2 ping attempt failed with error: Get https://:/v2/: dial tcp :0: connection refused
Jun 11 08:36:08 qe-sat6-rhel71 docker: v1 ping attempt failed with error: Get https://:/v1/_ping: dial tcp :0: connection refused. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add `--insecure-registry :` to the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag; simply place the CA certificate at /etc/docker/certs.d/:/ca.crt
Jun 11 08:36:08 qe-sat6-rhel71 docker: time="2015-06-11T08:36:08-04:00" level=info msg="-job pull(:/rhel7, 7.1-6) = ERR (1)"
Jun 11 08:36:08 qe-sat6-rhel71 docker: time="2015-06-11T08:36:08-04:00" level=error msg="Handler for POST /images/create returned error:  v1 ping attempt failed with error: Get https://:/v1/_ping: dial tcp :0: connection refused. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add `--insecure-registry :` to the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag; simply place the CA certificate at /etc/docker/certs.d/:/ca.crt"
Jun 11 08:36:08 qe-sat6-rhel71 docker: time="2015-06-11T08:36:08-04:00" level=error msg="HTTP Error: statusCode=500  v1 ping attempt failed with error: Get https://:/v1/_ping: dial tcp :0: connection refused. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add `--insecure-registry :` to the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag; simply place the CA certificate at /etc/docker/certs.d/:/ca.crt"

==> /var/log/foreman/production.log <==
2015-06-11 08:36:08 [I] Failed to save:
2015-06-11 08:36:08 [I]   Rendered /opt/rh/ruby193/root/usr/share/gems/gems/foreman_docker-1.2.0.14/app/views/foreman_docker/common_parameters/_environment_variable.html.erb (1.2ms)
2015-06-11 08:36:08 [I]   Rendered /opt/rh/ruby193/root/usr/share/gems/gems/foreman_docker-1.2.0.14/app/views/containers/steps/_form_buttons.html.erb (0.8ms)
2015-06-11 08:36:08 [I]   Rendered /opt/rh/ruby193/root/usr/share/gems/gems/foreman_docker-1.2.0.14/app/views/containers/steps/_title.html.erb (192.5ms)
2015-06-11 08:36:08 [I]   Rendered /opt/rh/ruby193/root/usr/share/gems/gems/foreman_docker-1.2.0.14/app/views/containers/steps/environment.html.erb within layouts/application (192.9ms)
2015-06-11 08:36:08 [I]   Rendered home/_submenu.html.erb (2.0ms)
2015-06-11 08:36:08 [I]   Rendered home/_user_dropdown.html.erb (6.1ms)
2015-06-11 08:36:08 [I] Read fragment views/tabs_and_title_records-3 (0.1ms)
2015-06-11 08:36:09 [I]   Rendered home/_organization_dropdown.html.erb (373.2ms)
2015-06-11 08:36:09 [I]   Rendered home/_location_dropdown.html.erb (36.4ms)
2015-06-11 08:36:09 [I]   Rendered home/_org_switcher.html.erb (410.5ms)
2015-06-11 08:36:09 [I]   Rendered home/_submenu.html.erb (4.3ms)
2015-06-11 08:36:09 [I]   Rendered home/_submenu.html.erb (4.7ms)
2015-06-11 08:36:09 [I]   Rendered home/_submenu.html.erb (2.2ms)
2015-06-11 08:36:09 [I]   Rendered home/_submenu.html.erb (4.4ms)
2015-06-11 08:36:09 [I]   Rendered home/_submenu.html.erb (3.3ms)
2015-06-11 08:36:09 [I]   Rendered home/_submenu.html.erb (2.7ms)
2015-06-11 08:36:09 [I]   Rendered home/_submenu.html.erb (1.8ms)
2015-06-11 08:36:09 [I]   Rendered home/_submenu.html.erb (3.9ms)
2015-06-11 08:36:09 [I] Write fragment views/tabs_and_title_records-3 (1.1ms)
2015-06-11 08:36:09 [I]   Rendered home/_topbar.html.erb (454.0ms)
2015-06-11 08:36:09 [I]   Rendered layouts/base.html.erb (455.5ms)
2015-06-11 08:36:09 [I] Completed 200 OK in 711ms (Views: 634.9ms | ActiveRecord: 27.8ms)

Comment 2 Mike McCune 2015-06-12 17:02:47 UTC

The error here was that the registry was specified as:

registry.access.redhat.com

vs:

http://registry.access.redhat.com

the protocol is required.

We should validate the field input to verify that it is a URL

Comment 3 Ade Bradshaw 2015-08-16 13:17:46 UTC

I also came across this issue, so I added the http://registry.access.redhat.com but this also failed, then I added 

https://registry.access.redhat.com  

This seemed to work much better (from looking at the logs) but then I ran into a different issue

Maybe we should add a verification step on the field, the one that requires a protocol

Comment 4 Dirk Herrmann 2015-12-09 19:37:11 UTC

Tested it today using Satellite 6.1.4 and different docker client versions. It works using https with docker 1.7.1 and in contrast to katello repos also using current docker version (1.8.2). Using http fails with both versions. Tested both WebUI and hammer CLI.

Comment 5 Elyézer Rezende 2016-01-21 18:12:11 UTC

A registry may be available only under https. I think registry.access.redhat.com is the case, this can explain why it is failing for http.

Comment 6 Daniel Lobato Garcia 2016-03-14 13:04:05 UTC

Fixed under - https://github.com/theforeman/foreman-docker/pull/142 to be merged

Comment 7 Daniel Lobato Garcia 2016-03-17 11:03:39 UTC

Fix merged upstream

Comment 10 Lukas Pramuk 2016-03-24 10:25:02 UTC

FailedQA.
@Sat6.2.0-Beta-Snap5

This is a showstopper, cannot create external registry anymore
While in Snap4 I was able to create and even search external registries
such as https://registry.hub.docker.com or https://registry.access.redhat.com

Comment 11 Lukas Pramuk 2016-03-24 10:30:48 UTC

2016-03-24 05:18:50 [app] [I] Failed to save: Unable to log in to this Docker Registry - Expected([200, 201, 202, 203, 204, 304]) <=> Actual(404 Not Found)

Similar message is also shown at UI.
(What about 301 and 302? these should also be expected...)

Comment 13 Daniel Lobato Garcia 2016-03-29 11:30:53 UTC

Lukas,

301 shouldn't really be expected. Satellite 6 only supports Docker Registry API v1 for external registries (https://registry.hub.docker.com/ changed to v2 very recently).

If you want to add registries, ensure they're v1 first. The patch I'm working on uses basic authentication with v1 registries to $REGISTRYURL/v1/users. The current call to '/auth' is wrong as it authenticates to the docker host, not the registry. 

Thanks for spotting that.

Comment 14 Daniel Lobato Garcia 2016-03-29 13:09:25 UTC

Now under review at https://github.com/theforeman/foreman-docker/pull/148

Comment 15 Lukas Pramuk 2016-04-06 18:48:28 UTC

FailedQA.

@Sat6.2.0-Beta-Snap6.2
tfm-rubygem-foreman_docker-2.0.1.3-1.el7sat.noarch

2016-04-06 14:25:35 [app] [I] Started POST "/registries" for <CLIENT_IP> at 2016-04-06 14:25:35 -0400
2016-04-06 14:25:35 [app] [I] Processing by RegistriesController#create as HTML
2016-04-06 14:25:35 [app] [I]   Parameters: {"utf8"=>"✓", "authenticity_token"=>"RLUwHJsw1ACjnBKnNMtHvjbPxc88aApYjhoxI4uhN54=", "docker_registry"=>{"name"=>"hub.docker.com", "url"=>"https://registry.hub.docker.com/", "description"=>"", "username"=>"", "password"=>"[FILTERED]", "location_ids"=>[""], "organization_ids"=>["", "1"]}, "commit"=>"Submit"}
2016-04-06 14:25:35 [app] [I] Failed to save: Unable to log in to this Docker Registry - Expected([200, 201, 202, 203, 204, 304]) <=> Actual(503 Service Unavailable)


https://registry.hub.docker.com/ >>> Actual(503 Service Unavailable)

https://registry.access.redhat.com/ >>> Actual(404 Not Found)

Comment 16 Daniel Lobato Garcia 2016-04-07 06:45:18 UTC

I'm sorry but how have you even tested this? The second cherry-pick that fixed that wasn't even made yet. 

-----------------------

Notice:

https://github.com/theforeman/foreman-docker/pull/148 

is not in here:

https://gitlab.sat.lab.tlv.redhat.com/satellite6/foreman_docker/blob/SATELLITE-6.2.0/app/models/docker_registry.rb

-----------------------

I don't know why this was set to on_qa w/o the second cherry-pick, but it should be definitely on POST.

Comment 17 Lukas Pramuk 2016-04-18 11:07:10 UTC

VERIFIED.

@Sat6.2.0-Beta-Snap8
tfm-rubygem-foreman_docker-2.0.1.4-1.el7sat.noarch

I was able to create an external registry. (Step 1)

I was able to search for rhel images using external registry (Step 3) only if username(password) didnt contain special char
>>> I guess username/password is not urlencoded - lets have another bz for this (not regression)

I was able to create a container based on rhel image using external registry (Step 4) <<< SUCCESS
>>> Though I wasn't able to power it on (another bz)

Comment 19 errata-xmlrpc 2016-07-27 09:15:57 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501

Note You need to log in before you can comment on or make changes to this bug.