rhel-osp-director: overcloud deployment exists with "Resource CREATE failed: resources.ControllerOvercloudServicesDeployment_Step4": puppet reports /usr/bin/nova-manage db sync returned 1 instead of one of
DescriptionAlexander Chuzhoy
2015-11-11 19:54:30 UTC
rhel-osp-director: overcloud deployment exists with "Resource CREATE failed: resources.ControllerOvercloudServicesDeployment_Step4": puppet reports /usr/bin/nova-manage db sync returned 1 instead of one of
Environment:
openstack-heat-api-cloudwatch-5.0.0-1.el7ost.noarch
openstack-heat-api-5.0.0-1.el7ost.noarch
heat-cfntools-1.3.0-2.el7.noarch
python-heatclient-0.8.0-1.el7ost.noarch
openstack-heat-api-cfn-5.0.0-1.el7ost.noarch
openstack-heat-common-5.0.0-1.el7ost.noarch
openstack-heat-engine-5.0.0-1.el7ost.noarch
instack-undercloud-2.1.3-1.el7ost.noarch
Steps to reproduce:
Attempt to deploy overcloud.
openstack overcloud deploy --templates --ntp-server x.x.x.x --timeout90 --control-scale 3 --compute-scale 1
Result:
The deployment fails with:
Stack failed with status: Resource CREATE failed: resources.ControllerOvercloudServicesDeployment_Step4: resources.ControllerNodesPostDeployment.Error: resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6
Heat Stack create failed.
Checking the os-collect-config on a controller:
Nov 11 14:29:13 overcloud-controller-0.localdomain os-collect-config[4095]: Error: /Stage[main]/Nova::Db::Sync/Exec[nova-db-sync]: Failed to call refresh: /usr/bin/nova-manage db sync returned 1 instead of one of
Nov 11 14:29:13 overcloud-controller-0.localdomain os-collect-config[4095]: Error: /Stage[main]/Nova::Db::Sync/Exec[nova-db-sync]: /usr/bin/nova-manage db sync returned 1 instead of one of [0]
Nov 11 14:29:13 overcloud-controller-0.localdomain os-collect-config[4095]: Error: /Stage[main]/Heat::Db::Sync/Exec[heat-dbsync]: Failed to call refresh: heat-manage --config-file /etc/heat/heat.conf db_sync retur
Nov 11 14:29:13 overcloud-controller-0.localdomain os-collect-config[4095]: Error: /Stage[main]/Heat::Db::Sync/Exec[heat-dbsync]: heat-manage --config-file /etc/heat/heat.conf db_sync returned 1 instead of one of
Nov 11 14:29:13 overcloud-controller-0.localdomain os-collect-config[4095]: [2015-11-11 14:29:13,158] (heat-config) [ERROR] Error running /var/lib/heat-config/heat-config-puppet/fe7a1ad6-d0ce-467a-b81c-880ea46c783
Nov 11 14:29:13 overcloud-controller-0.localdomain os-collect-config[4095]: [2015-11-11 14:29:13,165] (heat-config) [INFO] Completed /var/lib/heat-config/hooks/puppet
Comment 2Alexander Chuzhoy
2015-11-11 19:56:48 UTC
Gael has hit this issue consistently on about 10 deployments.
I tried running the failing command directly from the command line and it appeared to work:
[heat-admin@overcloud-controller-0 ~]$ sudo /usr/bin/nova-manage db sync
No handlers could be found for logger "oslo_config.cfg"
[heat-admin@overcloud-controller-0 ~]$ echo $?
we can't deploy multiple controllers without pacemaker yet, this seems to be failing because we don't default to use pacemaker for 8 yet, see bz #1281613
rhel-osp-director: overcloud deployment exists with "Resource CREATE failed: resources.ControllerOvercloudServicesDeployment_Step4": puppet reports /usr/bin/nova-manage db sync returned 1 instead of one of Environment: openstack-heat-api-cloudwatch-5.0.0-1.el7ost.noarch openstack-heat-api-5.0.0-1.el7ost.noarch heat-cfntools-1.3.0-2.el7.noarch python-heatclient-0.8.0-1.el7ost.noarch openstack-heat-api-cfn-5.0.0-1.el7ost.noarch openstack-heat-common-5.0.0-1.el7ost.noarch openstack-heat-engine-5.0.0-1.el7ost.noarch instack-undercloud-2.1.3-1.el7ost.noarch Steps to reproduce: Attempt to deploy overcloud. openstack overcloud deploy --templates --ntp-server x.x.x.x --timeout90 --control-scale 3 --compute-scale 1 Result: The deployment fails with: Stack failed with status: Resource CREATE failed: resources.ControllerOvercloudServicesDeployment_Step4: resources.ControllerNodesPostDeployment.Error: resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6 Heat Stack create failed. Checking the os-collect-config on a controller: Nov 11 14:29:13 overcloud-controller-0.localdomain os-collect-config[4095]: Error: /Stage[main]/Nova::Db::Sync/Exec[nova-db-sync]: Failed to call refresh: /usr/bin/nova-manage db sync returned 1 instead of one of Nov 11 14:29:13 overcloud-controller-0.localdomain os-collect-config[4095]: Error: /Stage[main]/Nova::Db::Sync/Exec[nova-db-sync]: /usr/bin/nova-manage db sync returned 1 instead of one of [0] Nov 11 14:29:13 overcloud-controller-0.localdomain os-collect-config[4095]: Error: /Stage[main]/Heat::Db::Sync/Exec[heat-dbsync]: Failed to call refresh: heat-manage --config-file /etc/heat/heat.conf db_sync retur Nov 11 14:29:13 overcloud-controller-0.localdomain os-collect-config[4095]: Error: /Stage[main]/Heat::Db::Sync/Exec[heat-dbsync]: heat-manage --config-file /etc/heat/heat.conf db_sync returned 1 instead of one of Nov 11 14:29:13 overcloud-controller-0.localdomain os-collect-config[4095]: [2015-11-11 14:29:13,158] (heat-config) [ERROR] Error running /var/lib/heat-config/heat-config-puppet/fe7a1ad6-d0ce-467a-b81c-880ea46c783 Nov 11 14:29:13 overcloud-controller-0.localdomain os-collect-config[4095]: [2015-11-11 14:29:13,165] (heat-config) [INFO] Completed /var/lib/heat-config/hooks/puppet