rhosp-director: Failed to minor update overcloud - fails before running yum update. Environment: openstack-puppet-modules-9.3.0-1.el7ost.noarch instack-undercloud-5.2.0-1.el7ost.noarch openstack-tripleo-heat-templates-5.2.0-3.el7ost.noarch Steps to reproduce: 1) Deployed overcloud with older osp10 openstack overcloud deploy --debug --templates --libvirt-type kvm --ntp-server clock.redhat.com --neutron-network-type vxlan --neutron-tunnel-types vxlan --control-scale 3 --control-flavor controller-d75f3dec-c770-5f88-9d4c-3fea1bf9c484 --compute-scale 10 --compute-flavor compute-b634c10a-570f-59ba-bdbf-0c313d745a10 --ceph-storage-scale 3 --ceph-storage-flavor ceph-cf1f074b-dadb-5eb8-9eb0-55828273fab7 -e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e virt/ceph.yaml -e virt/hostnames.yml -e virt/network/network-environment.yaml --log-file overcloud_deployment_48.log 2) Minor updated the undercloud 3) attempted to update the overcloud with: yes ""| openstack overcloud update stack -i overcloud Result: 23:25:43 stdout: starting package update on stack overcloud 23:25:43 WAITING 23:25:43 on_breakpoint: [u'compute-3', u'compute-6', u'compute-4', u'ceph-2', u'compute-5', u'compute-2', u'ceph-0', u'compute-8', u'compute-9', u'controller-0', u'compute-7', u'controller-1', u'ceph-1', u'controller-2', u'compute-0', u'compute-1'] 23:25:43 Breakpoint reached, continue? Regexp or Enter=proceed (will clear 56f0a527-1894-4319-977b-9ced788a1b09), no=cancel update, C-c=quit interactive mode: IN_PROGRESS 23:25:43 FAILED 23:25:43 update finished with status FAILED [stack@undercloud-0 ~]$ openstack stack failures list overcloud overcloud.CephStorage.1: resource_type: OS::TripleO::CephStorage physical_resource_id: c6e4229b-e8c6-4bcd-bff7-5eebc7aa03fa status: UPDATE_FAILED status_reason: | UPDATE aborted overcloud.CephStorage.0: resource_type: OS::TripleO::CephStorage physical_resource_id: e2ae4df6-56d5-4806-8a57-382b1d6c9f8c status: UPDATE_FAILED status_reason: | UPDATE aborted overcloud.CephStorage.2: resource_type: OS::TripleO::CephStorage physical_resource_id: a07e2312-ddb2-46cd-a0bb-5956365376a7 status: UPDATE_FAILED status_reason: | UPDATE aborted overcloud.Controller.1: resource_type: OS::TripleO::Controller physical_resource_id: f0e17e3b-a16f-43c9-a4ba-7433d04be742 status: UPDATE_FAILED status_reason: | UPDATE aborted overcloud.Controller.0: resource_type: OS::TripleO::Controller physical_resource_id: a4623f1b-269b-4b25-b5d9-c4b9df065af1 status: UPDATE_FAILED status_reason: | UPDATE aborted overcloud.Controller.2: resource_type: OS::TripleO::Controller physical_resource_id: 69df9a6e-e63a-4d56-8e31-823c8d0efc7e status: UPDATE_FAILED status_reason: | UPDATE aborted overcloud.Compute.1.UpdateDeployment: resource_type: OS::Heat::SoftwareDeployment physical_resource_id: 56f0a527-1894-4319-977b-9ced788a1b09 status: UPDATE_FAILED status_reason: | Error: resources.UpdateDeployment: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 3 deploy_stdout: | Started yum_update.sh on server 406f8d92-e1ce-4e99-bd69-4a52161c8824 at Tue Jan 24 23:24:22 UTC 2017 deploy_stderr: | overcloud.Compute.0: resource_type: OS::TripleO::Compute physical_resource_id: ca03031a-721e-4f6d-b4e3-82e7edfe8003 status: UPDATE_FAILED status_reason: | UPDATE aborted overcloud.Compute.3: resource_type: OS::TripleO::Compute physical_resource_id: 77a8f33d-4d75-44b7-9a3f-3a3777811f12 status: UPDATE_FAILED status_reason: | UPDATE aborted overcloud.Compute.2: resource_type: OS::TripleO::Compute physical_resource_id: 0d5c17d3-9dbc-48a1-b819-248eb4e34051 status: UPDATE_FAILED status_reason: | UPDATE aborted overcloud.Compute.5: resource_type: OS::TripleO::Compute physical_resource_id: d6f03709-6e2d-484f-b938-5c934e49ba89 status: UPDATE_FAILED status_reason: | UPDATE aborted overcloud.Compute.4: resource_type: OS::TripleO::Compute physical_resource_id: 04e64e48-0bc8-4d5a-9dca-8f55937aac72 status: UPDATE_FAILED status_reason: | UPDATE aborted overcloud.Compute.7: resource_type: OS::TripleO::Compute physical_resource_id: c7f794a4-7daa-4249-8b84-bd05a0cf32bf status: UPDATE_FAILED status_reason: | UPDATE aborted overcloud.Compute.6: resource_type: OS::TripleO::Compute physical_resource_id: 06f40157-86c5-4cc2-a7ca-d07287a12baa status: UPDATE_FAILED status_reason: | UPDATE aborted overcloud.Compute.9: resource_type: OS::TripleO::Compute physical_resource_id: 1e6d1a48-b570-414c-a3c7-b22c2abf8cc1 status: UPDATE_FAILED status_reason: | UPDATE aborted overcloud.Compute.8: resource_type: OS::TripleO::Compute physical_resource_id: eae7c25d-00ab-4b1c-a39c-91e0be6e4cab status: UPDATE_FAILED status_reason: | UPDATE aborted ########################################################################## nova list|grep 406f8d92-e1ce-4e99-bd69-4a52161c8824 | 406f8d92-e1ce-4e99-bd69-4a52161c8824 | compute-1 | ACTIVE | - | Running | ctlplane=192.0.2.10 | ########################################################################## Checking the errors in os-collect-config on the node with error: Jan 24 23:24:24 compute-1.localdomain os-collect-config[4216]: [2017-01-24 23:24:24,770] (heat-config) [ERROR] Error running /var/lib/heat-config/heat-config-script/6f673a67-bfa6-4fce-8032-03b9c909c278. [3] The file 6f673a67-bfa6-4fce-8032-03b9c909c278 is attached.
Created attachment 1244107 [details] 6f673a67-bfa6-4fce-8032-03b9c909c278
systemctl is-active pacemaker => unknown issue
Related to bug #1414779
*** Bug 1426261 has been marked as a duplicate of this bug. ***
Ack from my side.
Verified with openstack-tripleo-heat-templates-5.2.0-15.el7ost.noarch openstack stack list +--------------------------------------+------------+-----------------+----------------------+----------------------+ | ID | Stack Name | Stack Status | Creation Time | Updated Time | +--------------------------------------+------------+-----------------+----------------------+----------------------+ | 62648ef1-70d2-41e0-ad56-de77616988a2 | overcloud | UPDATE_COMPLETE | 2017-05-09T10:43:20Z | 2017-05-09T13:32:15Z | +--------------------------------------+------------+-----------------+----------------------+----------------------+
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:1242
Hitting the same on OpenStack 10 and openstack-tripleo-heat-templates-5.3.10-17.el7ost.noarch. Can anybody reproduce that? Opened support case 02233196 for a customer.