Bug 1741349
| Summary: | Overcloud Controller minor update fails pulling ovn-northd which does not exist | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Alistair Tonner <atonner> | ||||||
| Component: | openstack-tripleo-heat-templates | Assignee: | RHOS Maint <rhos-maint> | ||||||
| Status: | CLOSED NOTABUG | QA Contact: | Sasha Smolyak <ssmolyak> | ||||||
| Severity: | unspecified | Docs Contact: | |||||||
| Priority: | unspecified | ||||||||
| Version: | 15.0 (Stein) | CC: | aschultz, jlibosva, mburns, mlammon, njohnston | ||||||
| Target Milestone: | --- | ||||||||
| Target Release: | --- | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2019-08-20 13:44:36 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Bug Depends On: | |||||||||
| Bug Blocks: | 1727808 | ||||||||
| Attachments: |
|
||||||||
|
Description
Alistair Tonner
2019-08-14 20:53:18 UTC
Description of problem:
minor update of openstack 15 deployment fails in overcloud update when updating Controller attempts to pull ovn-northd
Version-Release number of selected component (if applicable):
RHOS_TRUNK-15.0-RHEL-8-20190813.n.0
openstack-heat-agents.noarch 1.8.1-0.20190523210450.1e15344.el8ost @rhelosp-15.0
openstack-heat-api.noarch 1:12.0.1-0.20190805120452.3476f1d.el8ost @rhelosp-15.0
openstack-heat-common.noarch 1:12.0.1-0.20190805120452.3476f1d.el8ost @rhelosp-15.0
openstack-heat-engine.noarch 1:12.0.1-0.20190805120452.3476f1d.el8ost @rhelosp-15.0
openstack-heat-monolith.noarch 1:12.0.1-0.20190805120452.3476f1d.el8ost @rhelosp-15.0
openstack-selinux.noarch 0.8.19-0.20190606150404.06faac7.el8ost @rhelosp-15.0
openstack-tripleo-common.noarch 10.8.1-0.20190808130448.b942458.el8ost @rhelosp-15.0
openstack-tripleo-common-containers.noarch 10.8.1-0.20190808130448.b942458.el8ost @rhelosp-15.0
openstack-tripleo-heat-templates.noarch 10.6.1-0.20190812140519.2a684c0.el8ost @rhelosp-15.0
openstack-tripleo-image-elements.noarch 10.4.1-0.20190705161217.2c8a6a5.el8ost @rhelosp-15.0
openstack-tripleo-puppet-elements.noarch 10.3.2-0.20190710165331.c89fe3c.el8ost @rhelosp-15.0
openstack-tripleo-validations.noarch 10.5.1-0.20190812110443.80a9f7b.el8ost @rhelosp-15.0
puppet-openstack_extras.noarch 14.4.1-0.20190420090934.6b1b687.el8ost @rhelosp-15.0
puppet-openstacklib.noarch 14.4.1-0.20190420125152.3719ca1.el8ost @rhelosp-15.0
python-openstackclient-lang.noarch 3.18.0-0.20190312140834.6868499.el8ost @rhelosp-15.0
python3-openstackclient.noarch 3.18.0-0.20190312140834.6868499.el8ost @rhelosp-15.0
python3-openstacksdk.noarch
How reproducible:
Steps to Reproduce:
1. Deploy OSP 15 and run minor update process
2.
3.
Actual results:
2019-08-14 18:10:30 | fatal: [controller-0]: FAILED! => {"changed": true, "cmd": ["podman", "pull", "192.168.24.1:8787/rhosp15/openstack-ovn-northd:20190812.1"], "delta": "0:00:00.070167", "end": "2019-08-14 18:10:29.867197", "msg": "non-zero return code", "rc": 125, "start": "2019-08-14 18:10:29.797030", "stderr": "Trying to pull 192.168.24.1:8787/rhosp15/openstack-ovn-northd:20190812.1...Failed\nerror pulling image \"192.168.24.1:8787/rhosp15/openstack-ovn-northd:20190812.1\": unable to pull 192.168.24.1:8787/rhosp15/openstack-ovn-northd:20190812.1: unable to pull image: Error determining manifest MIME type for docker://192.168.24.1:8787/rhosp15/openstack-ovn-northd:20190812.1: Error reading manifest 20190812.1 in 192.168.24.1:8787/rhosp15/openstack-ovn-northd: error parsing HTTP 404 response body: invalid character '<' looking for beginning of value: \"<!DOCTYPE HTML PUBLIC \\\"-//IETF//DTD HTML 2.0//EN\\\">\\n<html><head>\\n<title>404 Not Found</title>\\n</head><body>\\n<h1>Not Found</h1>\\n<p>The requested URL /v2/rhosp15/openstack-ovn-northd/manifests/20190812.1 was not found on this server.</p>\\n</body></html>\\n\"", "stderr_lines": ["Trying to pull 192.168.24.1:8787/rhosp15/openstack-ovn-northd:20190812.1...Faile
-> overall update fails some time later.
Expected results:
Openstack overcloud update command completes successfully.
Additional info:
https://rhos-ci-staging-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/hardware_provisioning/view/rqci/job/DFG-hardware_provisioning-rqci-15_director-8.0-virthost-3cont_3ceph_1comp_2ironic-ipv4-geneve-minor_update-RHELOSP-48859/16/
Created attachment 1603896 [details]
controller update log
Created attachment 1603897 [details]
overcloud minor update stage log
I am also hitting this issue 019-08-15 21:13:14.383786", "msg": "non-zero return code", "rc": 125, "start": "2019-08-15 21:13:13.921758", "stderr": "Trying to pull registry.access.redhat.com/rhosp15/openstack-ovn-northd:15.0...Failed\nerror pulling image \"registry.access.redhat.com/rhosp15/openstack-ovn-northd:15.0\": unable to pull registry.access.redhat.com/rhosp15/openstack-ovn-northd:15.0: unable to pull image: Error determining manifest MIME type for docker://registry.access.redhat.com/rhosp15/openstack-ovn-northd:15.0: Error reading manifest 15.0 in registry.access.redhat.com/rhosp15/openstack-ovn-northd: name unknown: Repo not found", "stderr_lines": ["Trying to pull registry.access.redhat.com/rhosp15/openstack-ovn-northd:15.0...Failed", "error pulling image \"registry.access.redhat.com/rhosp15/openstack-ovn-northd:15.0\": unable to pull registry.access.redhat.com/rhosp15/openstack-ovn-northd:15.0: unable to pull image: Error determining manifest MIME type for docker://registry.access.redhat.com/rhosp15/openstack-ovn-northd:15.0: Error reading manifest 15.0 in registry.access.redhat.com/rhosp15/openstack-ovn-northd: name unknown: Repo not found"], "stdout": "", "stdout_lines": []} From comment #5 it would look like the job might not be picking up the right parameters in the container image prepare step or whatever it has become now. It is defaulting to general pullspec for the container images registry.access.redhat.com/rhosp15/XXXX which is not available yet as OSP 15 is not GA, so you would either need to be pointing to beta content or internal registry for the content. By providing the right parameters to the prepare stage of things for registry to fetch images from. |