Bug 1607453

Summary:	Deployment fails with: Object GET failed: https://.../overcloud/plan-environment.yaml 404 Not Found
Product:	Red Hat OpenStack	Reporter:	grozov
Component:	openstack-tripleo	Assignee:	Luke Short <lshort>
Status:	CLOSED WONTFIX	QA Contact:	Arik Chernetsky <achernet>
Severity:	high	Docs Contact:
Priority:	high
Version:	13.0 (Queens)	CC:	akaris, dpeacock, emacchi, ggrimaux, grozov, lshort, mburns, nchandek, vcojot
Target Milestone:	---	Keywords:	Reopened, Triaged, ZStream
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-08-29 19:01:46 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description grozov 2018-07-23 15:00:11 UTC

Description of problem:
I deleted a deployment and tried to deploy again. This is the command I used:
openstack overcloud deploy \
--timeout 100 \
--templates /usr/share/openstack-tripleo-heat-templates \
--stack overcloud \
--libvirt-type kvm \
--ntp-server clock.redhat.com \
-e /home/stack/virt/config_lvm.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
-e /home/stack/virt/network/network-environment.yaml \
-e /home/stack/virt/inject-trust-anchor.yaml \
-e /home/stack/virt/hostnames.yml \
-e /home/stack/virt/nodes_data.yaml \
-e /home/stack/virt/extra_templates.yaml \
-e /home/stack/virt/docker-images.yaml \
--log-file overcloud_deployment_27.log

Deployment failed after a few seconds:

Object GET failed: https://192.168.24.2:13808/v1/AUTH_6da9489fe04349188addc7c15ab0fa14/overcloud/plan-environment.yaml 404 Not Found  [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<


Version-Release number of selected component (if applicable):
openstack-tripleo-ui-9.1.1-0.20180702224622.d3d7221.el7ost.noarch

How reproducible:
100%

Steps to Reproduce:
1. I had an IR setup
2. I deleted the stack
3. I re-ran the deploy command


Additional info:
The deployment should not be trying to pull this file from swift. It really doesn't exist in the container.

Comment 1 James Slagle 2018-09-19 19:01:20 UTC

please try the deployment command again with --debug, capture that output and attach it to the bz. that will give us more info as to what to check next.

Comment 2 James Slagle 2018-10-17 21:13:58 UTC

this is a common operation, and we've not seen any other reports of the problem you've reported.

please let us know if you're able to reproduce the problem with the latest osp14 puddle, and if so, please reopen the bug and provide the requested data.

Comment 3 Vincent S. Cojot 2018-10-21 03:18:11 UTC

Just ran into this on OSP13 as well:

+ openstack overcloud deploy --templates -e /home/stack/OSP/osp13/node_info_micro.yaml -e /home/stack/OSP/osp13/overcloud-images.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/cinder-backup.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-rgw.yaml -e /home/stack/OSP/osp13/net-bond-with-vlans-with-nic4.yaml -e /home/stack/OSP/osp13/krynn-environment.yaml -e /home/stack/OSP/osp13/extraconfig-environment.yaml -e /home/stack/OSP/osp13/local-environment.yaml -e /home/stack/OSP/osp13/inject-trust-anchor.yaml -e /home/stack/OSP/osp13/ceph-custom-config.yaml
Started Mistral Workflow tripleo.validations.v1.check_pre_deployment_validations. Execution ID: b95242e5-13f1-4849-96a9-f86fe0aadd51
Waiting for messages on queue 'tripleo' with no timeout.
Object GET failed: https://10.162.200.113:13808/v1/AUTH_4b662aaa5cd94491bba39238200f9914/overcloud/plan-environment.yaml 404 Not Found  [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<

real    0m20.987s
user    0m1.119s
sys     0m0.385s
+ exit 127

Comment 4 Vincent S. Cojot 2018-10-21 03:28:42 UTC

Found a quick workaround:
(undercloud) [stack@osp8h ~]$ openstack overcloud plan list
+-----------+
| Plan Name |
+-----------+
| overcloud |
+-----------+
(undercloud) [stack@osp8h ~]$ openstack overcloud plan delete overcloud
Deleting plan overcloud...
(undercloud) [stack@osp8h ~]$ openstack overcloud plan list

I didn't have a stack but I had a plan (probably because I ctrl-c'ed out of the first deploy attempt today).

Comment 13 Luke Short 2019-09-06 10:53:51 UTC

As mentioned earlier, removing the "overcloud" container can help.

Another common problem is that the `--templates` argument is not given a directory with the correct template files. This should have all of the rendered templates such as "plan-environment.yaml". These get uploaded into the Swift container. If the important files are not in that directory then they will not be uploaded to Swift and it will result in a failed deployment.