Bug 1664705
| Summary: | Undercloud upgrade 13->14 fails because of Error response from daemon: No such container: ironic_api | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Sasha Smolyak <ssmolyak> | ||||||
| Component: | openstack-ironic | Assignee: | Jose Luis Franco <jfrancoa> | ||||||
| Status: | CLOSED DUPLICATE | QA Contact: | mlammon | ||||||
| Severity: | high | Docs Contact: | |||||||
| Priority: | high | ||||||||
| Version: | 14.0 (Rocky) | CC: | bfournie, mburns, ohochman, sasha, sgolovat, ssmolyak | ||||||
| Target Milestone: | --- | ||||||||
| Target Release: | --- | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | eco | Doc Type: | If docs needed, set a value | ||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2019-01-14 14:53:10 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
Created attachment 1519508 [details]
/home/stack from undercloud
From install-undercloud.log, you are having problems just downloading images. This isn't Ironic specific. WARNING:tripleo_common.image.image_uploader:docker pull failed: Get https://rhos-qe-mirror-tlv.usersys.redhat.com:5000/v1/_ping: http: server gave HTTP response to HTTPS client Traceback (most recent call last): File "/usr/bin/tripleo-container-image-prepare", line 100, in <module> env, roles_data, cleanup=args.cleanup) File "/usr/lib/python2.7/site-packages/tripleo_common/image/kolla_builder.py", line 193, in container_images_prepare_multi uploader.upload() File "/usr/lib/python2.7/site-packages/tripleo_common/image/image_uploader.py", line 132, in upload uploader.run_tasks() File "/usr/lib/python2.7/site-packages/tripleo_common/image/image_uploader.py", line 514, in run_tasks result = self.upload_image(*first) File "/usr/lib/python2.7/site-packages/tripleo_common/image/image_uploader.py", line 257, in upload_image DockerImageUploader._pull(dockerc, repo, tag=source_tag) File "/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 241, in wrapped_f return self.call(f, *args, **kw) File "/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 330, in call start_time=start_time) File "/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 297, in iter raise retry_exc.reraise() File "/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 136, in reraise raise self.last_attempt.result() File "/usr/lib/python2.7/site-packages/concurrent/futures/_base.py", line 422, in result return self.__get_result() File "/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 333, in call result = fn(*args, **kwargs) File "/usr/lib/python2.7/site-packages/tripleo_common/image/image_uploader.py", line 295, in _pull raise ImageUploaderException('Could not pull image %s' % image) tripleo_common.image.exception.ImageUploaderException: Could not pull image rhos-qe-mirror-tlv.usersys.redhat.com:5000/rhosp14/openstack-tempest Created attachment 1520468 [details]
undercloud files after failed upgrade
This bug is a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1647956, the fix for it is already merged waiting to be submitted: https://code.engineering.redhat.com/gerrit/#/c/158245/5 Feel free to re-open if needed. *** This bug has been marked as a duplicate of bug 1647956 *** *** Bug 1658375 has been marked as a duplicate of this bug. *** |
Description of problem: While upgrading from RHOS13 z2 to RHOS14 with infrared, undercloud upgrade fails: TASK [Online data migration for Ironic] *************************************************************************************************************************************************************************** fatal: [undercloud-0 -> 192.168.24.1]: FAILED! => {"changed": true, "cmd": ["docker", "exec", "ironic_api", "ironic-dbsync", "--config-file", "/etc/ironic/ironic.conf", "online_data_migrations"], "delta": "0:00:00.066630", "end": "2019-01-09 08:00:26.470229", "msg": "non-zero return code", "rc": 1, "start": "2019-01-09 08:00:26.403599", "stderr": "Error response from daemon: No such container: ironic_api", "stderr_lines": ["Error response from daemon: No such container: ironic_api"], "stdout": "", "stdout_lines": []} Version-Release number of selected component (if applicable): RHOS13 puddle: 2018-08-22.2 RHOS14: 2019-01-08.1 How reproducible: 100% Steps to Reproduce: 1. Install RHOS13 with enable_telemetry=true in undercloud.conf and custom_templates: parameter_defaults: GnocchiBackend: file 2. Follow the upgrade scenario for infrared: https://gitlab.cee.redhat.com/yprokule/RHOS-13-14-upgrade : export MIRROR=tlv export PUDDLE=2019-01-08.1 infrared tripleo-undercloud \ --upgrade yes \ --mirror ${MIRROR} \ --build ${PUDDLE} \ --ansible-args="tags=upgrade_repos,undercloud_version_discovery,undercloud_containers" \ --registry-mirror rhos-qe-mirror-$MIRROR.usersys.redhat.com:5000 It passes. 3. Add container_images_file = /home/stack/containers-prepare-parameter.yaml under [DEFAULT] section of undercloud.conf Add docker_insecure_registries = rhos-qe-mirror-qeos.usersys.redhat.com:5000,brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888,registry.access.redhat.com 4. Run openstack undercloud upgrade -y 2>&1 Actual results: TASK [Online data migration for Nova] ***************************************************************************************************************************************************************************** skipping: [undercloud-0] TASK [Online data migration for Ironic] *************************************************************************************************************************************************************************** fatal: [undercloud-0 -> 192.168.24.1]: FAILED! => {"changed": true, "cmd": ["docker", "exec", "ironic_api", "ironic-dbsync", "--config-file", "/etc/ironic/ironic.conf", "online_data_migrations"], "delta": "0:00:00.066630", "end": "2019-01-09 08:00:26.470229", "msg": "non-zero return code", "rc": 1, "start": "2019-01-09 08:00:26.403599", "stderr": "Error response from daemon: No such container: ironic_api", "stderr_lines": ["Error response from daemon: No such container: ironic_api"], "stdout": "", "stdout_lines": []} Expected results: Upgrade passes Additional info: attached