Bug 1684278 - Run tripleo-container-image-prepare logged to /var/log/tripleo-container-image-prepare.log task fails when using insecure registry
Summary: Run tripleo-container-image-prepare logged to /var/log/tripleo-container-imag...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-common
Version: 15.0 (Stein)
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: beta
: 15.0 (Stein)
Assignee: Cédric Jeanneret
QA Contact: Sasha Smolyak
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-02-28 21:28 UTC by Marius Cornea
Modified: 2020-01-31 08:09 UTC (History)
10 users (show)

Fixed In Version: openstack-tripleo-common-10.4.1-0.20190313060345.1377727.el8ost
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-09-21 11:20:27 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Gerrithub.io 447338 0 None None None 2019-03-08 06:36:40 UTC
OpenStack gerrit 639037 0 None MERGED Allow PythonImageUploader to accept unknown CA 2020-06-19 01:31:58 UTC
OpenStack gerrit 639405 0 None MERGED container-image-prepare: redirect all output to logfile 2020-06-19 01:31:58 UTC
OpenStack gerrit 640941 0 None MERGED Small corrections in image_uploader 2020-06-19 01:31:57 UTC
Red Hat Product Errata RHEA-2019:2811 0 None None None 2019-09-21 11:20:55 UTC

Description Marius Cornea 2019-02-28 21:28:44 UTC
Description of problem:
Run tripleo-container-image-prepare logged to /var/log/tripleo-container-image-prepare.log task fails when using insecure registry without providing a comprehensible reason of the error:

Starting new HTTPS connection (1): 192.168.24.1:8787
Using config files: ['/tmp/tmphg533y4n']
container_images JSON: [{'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-cron:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-glance-api:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-haproxy:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-heat-api-cfn:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-heat-api:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-heat-engine:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-ironic-api:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-ironic-conductor:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-ironic-inspector:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-ironic-pxe:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-ironic-neutron-agent:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-iscsid:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-keepalived:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-keystone:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-mariadb:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-memcached:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-mistral-api:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-mistral-engine:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-mistral-executor:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-mistral-event-engine:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-neutron-dhcp-agent:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-neutron-l3-agent:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-neutron-openvswitch-agent:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-neutron-server:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-nova-api:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-nova-compute-ironic:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-nova-conductor:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-nova-placement-api:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-nova-scheduler:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-qdrouterd:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-rabbitmq:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-swift-account:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-swift-container:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-swift-object:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-swift-proxy-server:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-tripleo-ui:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-zaqar-wsgi:20190226.1', 'push_destination': '192.168.24.1:8787'}, {'image_source': 'kolla', 'imagename': 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-tempest:20190226.1', 'push_destination': '192.168.24.1:8787'}]
Starting new HTTPS connection (1): brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888
imagename: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-tempest:20190226.1
Starting new HTTP connection (1): 192.168.24.1:8787
http://192.168.24.1:8787 "GET /v2/ HTTP/1.1" 200 2
http://192.168.24.1:8787/v2/ status code 200
http://192.168.24.1:8787 "POST /v2/rhosp15/openstack-tempest/blobs/uploads/ HTTP/1.1" 404 241
Starting new HTTP connection (1): brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888
http://brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888 "GET /v2/ HTTP/1.1" 200 2
http://brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/v2/ status code 200
http://brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888 "GET /v2/rhosp15/openstack-tempest/manifests/20190226.1 HTTP/1.1" 302 581
Starting new HTTPS connection (1): brew-pulp-docker01.web.prod.ext.phx2.redhat.com:443
http://brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888 "GET /v2/rhosp15/openstack-tempest/manifests/20190226.1 HTTP/1.1" 302 581
Starting new HTTPS connection (2): brew-pulp-docker01.web.prod.ext.phx2.redhat.com:443
http://brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888 "GET /v2/rhosp15/openstack-tempest/manifests/20190226.1 HTTP/1.1" 302 581
Starting new HTTPS connection (3): brew-pulp-docker01.web.prod.ext.phx2.redhat.com:443
http://brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888 "GET /v2/rhosp15/openstack-tempest/manifests/20190226.1 HTTP/1.1" 302 581
Starting new HTTPS connection (4): brew-pulp-docker01.web.prod.ext.phx2.redhat.com:443
Resetting dropped connection: brew-pulp-docker01.web.prod.ext.phx2.redhat.com
http://brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888 "GET /v2/rhosp15/openstack-tempest/manifests/20190226.1 HTTP/1.1" 302 581
Starting new HTTPS connection (5): brew-pulp-docker01.web.prod.ext.phx2.redhat.com:443

Version-Release number of selected component (if applicable):
openstack-tripleo-heat-templates-10.3.1-0.20190227020336.7adc623.el8ost.noarch
python3-tripleo-common-10.4.1-0.20190227000340.1d415e6.el8ost.noarch
openstack-tripleo-common-10.4.1-0.20190227000340.1d415e6.el8ost.noarch
openstack-tripleo-common-containers-10.4.1-0.20190227000340.1d415e6.el8ost.noarc

Comment 1 Cédric Jeanneret 2019-03-04 14:51:17 UTC
Hey,

So for the TLS issue, the mentioned patch will be enough, since a downstream patch will add the internal registry in the NO_VERIFY_REGISTRIES set. So it's "work in progress", patch is in upstream gate right now.

Comment 2 Cédric Jeanneret 2019-03-05 11:36:07 UTC
Linked internal gerrit for the addition of the internal registry.

Comment 3 Cédric Jeanneret 2019-03-05 13:18:31 UTC
So apparently there were some more work to do in the image_uploader.py ... the scoping is this file is terrible :(.

Comment 4 Cédric Jeanneret 2019-03-07 11:20:43 UTC
Hello Marius,

Small update on this:
- this patch is needed as it addresses the last issues: https://review.openstack.org/640941
- as discussed, the particular case of the internal registry needs a particular solution - in this case, infrared should install the IT CA on the undercloud in order to avoid this issue. Care to point me where to push a patch for that?

Cheers,

C.

Comment 5 Cédric Jeanneret 2019-03-08 06:36:41 UTC
Hello Marius,

All is merged - do we have anything to do for downstream?

Cheers,

C.

Comment 16 errata-xmlrpc 2019-09-21 11:20:27 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:2811


Note You need to log in before you can comment on or make changes to this bug.