Bug 1582149
| Summary: | "Failed to pull image: registry.access.redhat.com ..." during OC deployment | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Filip Hubík <fhubik> | ||||
| Component: | openstack-tripleo-heat-templates | Assignee: | Emilien Macchi <emacchi> | ||||
| Status: | CLOSED DUPLICATE | QA Contact: | Gurenko Alex <agurenko> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | 14.0 (Rocky) | CC: | aschultz, mburns | ||||
| Target Milestone: | --- | Keywords: | Automation, AutomationBlocker, Triaged | ||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2018-05-30 13:31:00 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
Note this might be caused by https://bugzilla.redhat.com/show_bug.cgi?id=1581756 but I don't have that verified yet. After extensive tinkering and multiple deployments with various options for "image prepare" and "deploy" commands I verified manualy that this issue is direct consequence of https://bugzilla.redhat.com/show_bug.cgi?id=1581756 so this BZ might be considered as its duplicate. I was able to workaround it passing -e /usr/share/openstack-tripleo-heat-templates/environments/docker.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml to $ openstack overcloud container image prepare ... # but openstack overcloud deploy left intact without these included (likely since https://bugzilla.redhat.com/show_bug.cgi?id=1579361 fix is inclulded in current deployments) commands during deployment. Infrared workaround: https://review.gerrithub.io/c/redhat-openstack/infrared/+/412461 is proved working. Marking this BZ as duplicate. *** This bug has been marked as a duplicate of bug 1581756 *** |
Created attachment 1441037 [details] overcloud_install.log Description of problem: Overcloud deployment of OSP14 (puddle 2018-05-23.1) is unable to pull container images from registry: $ cat overcloud_install.log | grep "ERROR" ... "2018-05-24 10:22:26,740 ERROR: 22465 -- Failed to pull image: registry.access.redhat.com/rhosp14/openstack-cron:latest", "2018-05-24 10:22:28,386 ERROR: 22465 -- Failed running docker-puppet.py for crond", "2018-05-24 10:22:28,386 ERROR: 22465 -- Unable to find image 'registry.access.redhat.com/rhosp14/openstack-cron:latest' locally", "2018-05-24 10:22:30,422 ERROR: 22464 -- Failed to pull image: registry.access.redhat.com/rhosp14/openstack-nova-compute:latest", "2018-05-24 10:22:32,080 ERROR: 22464 -- Failed running docker-puppet.py for nova_libvirt", "2018-05-24 10:22:32,081 ERROR: 22464 -- Unable to find image 'registry.access.redhat.com/rhosp14/openstack-nova-compute:latest' locally", "2018-05-24 10:22:41,078 ERROR: 22463 -- Failed to pull image: registry.access.redhat.com/rhosp14/openstack-ceilometer-central:latest", "2018-05-24 10:22:42,730 ERROR: 22463 -- Failed running docker-puppet.py for ceilometer", "2018-05-24 10:22:42,730 ERROR: 22463 -- Unable to find image 'registry.access.redhat.com/rhosp14/openstack-ceilometer-central:latest' locally", "2018-05-24 10:22:51,801 ERROR: 22465 -- Failed to pull image: registry.access.redhat.com/rhosp14/openstack-neutron-server:latest", "2018-05-24 10:22:53,454 ERROR: 22465 -- Failed running docker-puppet.py for neutron", "2018-05-24 10:22:53,454 ERROR: 22465 -- Unable to find image 'registry.access.redhat.com/rhosp14/openstack-neutron-server:latest' locally", "2018-05-24 10:22:55,511 ERROR: 22464 -- Failed to pull image: registry.access.redhat.com/rhosp14/openstack-iscsid:latest", "2018-05-24 10:22:57,282 ERROR: 22464 -- Failed running docker-puppet.py for iscsid", "2018-05-24 10:22:57,282 ERROR: 22464 -- Unable to find image 'registry.access.redhat.com/rhosp14/openstack-iscsid:latest' locally", "2018-05-24 10:22:57,283 ERROR: 22462 -- ERROR configuring ceilometer", "2018-05-24 10:22:57,283 ERROR: 22462 -- ERROR configuring nova_libvirt", "2018-05-24 10:22:57,283 ERROR: 22462 -- ERROR configuring crond", "2018-05-24 10:22:57,283 ERROR: 22462 -- ERROR configuring neutron", "2018-05-24 10:22:57,283 ERROR: 22462 -- ERROR configuring iscsid", .. How reproducible: always Steps to Reproduce: 1. Deploy OSP14 topology 1:1:1:1 2a. Deploy script cat overcloud_deploy.sh #!/bin/bash openstack overcloud deploy \ --timeout 100 \ --templates /usr/share/openstack-tripleo-heat-templates \ --stack overcloud \ --libvirt-type kvm \ --ntp-server clock.redhat.com \ -e /home/stack/virt/internal.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \ -e /home/stack/virt/network/network-environment.yaml \ -e /home/stack/virt/hostnames.yml \ -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml \ -e /home/stack/virt/debug.yaml \ -e /home/stack/virt/nodes_data.yaml \ -e /home/stack/virt/docker-images.yaml \ --log-file overcloud_deployment_3.log cat container_images.yaml container_images: - imagename: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhceph:3-6 2b. Do not use any additional params for openstack overcloud container * than these in documentation (no docker*.yml files added explicitely) Actual results: heat stack-list WARNING (shell) "heat stack-list" is deprecated, please use "openstack stack list" instead +--------------------------------------+------------+-----------------+----------------------+--------------+----------------------------------+ | id | stack_name | stack_status | creation_time | updated_time | project | +--------------------------------------+------------+-----------------+----------------------+--------------+----------------------------------+ | bbadb9e0-e05f-4a4d-891f-b5311d90bd57 | overcloud | CREATE_COMPLETE | 2018-05-24T10:03:23Z | None | 6dbbbe95ae0d4a31aa03986af7cfb778 | +--------------------------------------+------------+-----------------+----------------------+--------------+----------------------------------+ # but overcloudrc is not created since oc deployment failed most likely on post-mistral steps. Expected results: OC deploy is able to pass and create overclourc* files. Additional info: openstack-tripleo-heat-templates-9.0.0-0.20180509231813.el7ost.noarch puppet-tripleo-9.0.1-0.20180508175253.9de8a36.el7ost.noarch openstack-tripleo-image-elements-9.0.0-0.20180426101324.08096ad.el7ost.noarch python-tripleoclient-10.0.1-0.20180514014754.cea9730.el7ost.noarch openstack-tripleo-common-9.0.2-0.20180514100531.el7ost.noarch python-tripleoclient-heat-installer-10.0.1-0.20180514014754.cea9730.el7ost.noarch openstack-tripleo-common-containers-9.0.2-0.20180514100531.el7ost.noarch openstack-tripleo-validations-9.0.1-0.20180508205842.15e0d70.el7ost.noarch openstack-tripleo-puppet-elements-9.0.0-0.20180427082845.fae8944.el7ost.noarc