rhel-osp-director: 7.1GA->7.3 async fails "resources.Controller: Property error: resources[1].properties: Property MysqlClustercheckPassword not assigned " Environment: openstack-puppet-modules-2015.1.8-51.el7ost.noarch instack-undercloud-2.1.2-39.el7ost.noarch openstack-tripleo-heat-templates-0.8.6-126.el7ost.noarch Steps to reproduce: 1. Deploy 7.1GA with: openstack overcloud deploy --templates --control-scale 3 --compute-scale 1 --neutron-network-type vxlan --neutron-tunnel-types vxlan --ntp-server clock.redhat.com --timeout 90 -e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e network-environment.yaml --ceph-storage-scale 1 2.Populate the OC with some objects. 3. Update the undercloud and attempt to update the overcloud. Result: starting package update on stack overcloud IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS FAILED update finished with status FAILED [stack@instack ~]$ heat resource-list -n5 overcloud |grep -v COMPLE +---------------------------------------------+-----------------------------------------------+---------------------------------------------------+-----------------+----------------------+------------------------- --------------------+ | resource_name | physical_resource_id | resource_type | resource_status | updated_time | parent_resource | +---------------------------------------------+-----------------------------------------------+---------------------------------------------------+-----------------+----------------------+------------------------- --------------------+ | CephStorage | dfc62ec2-cda1-4fa2-a200-17fb1071c1b6 | OS::Heat::ResourceGroup | UPDATE_FAILED | 2016-06-10T17:33:53Z | | | 0 | 6295702f-8795-4ffb-9f2c-b0b80452f47e | OS::TripleO::CephStorage | UPDATE_FAILED | 2016-06-10T17:33:55Z | CephStorage | | Compute | 0bb1608a-ef35-4f0c-967e-f9750bf856ad | OS::Heat::ResourceGroup | UPDATE_FAILED | 2016-06-10T17:34:13Z | | | 0 | 5bcbf1f6-a44f-4f20-bff0-1e8058606fd4 | OS::TripleO::Compute | UPDATE_FAILED | 2016-06-10T17:34:16Z | Compute | | Controller | 8d4e30a8-7d85-452c-85ef-78a96027e366 | OS::Heat::ResourceGroup | UPDATE_FAILED | 2016-06-10T17:34:19Z | | | CephStorage | 765f5c23-c141-4a89-bd75-21affac9c6e0 | OS::Nova::Server | CREATE_FAILED | 2016-06-10T17:34:28Z | 0 | | NovaCompute | 935d65b9-b061-4ee0-aa4c-58ba73449e1c | OS::Nova::Server | CREATE_FAILED | 2016-06-10T17:34:49Z | 0 | +---------------------------------------------+-----------------------------------------------+---------------------------------------------------+-----------------+----------------------+---------------------------------------------+ heat resource-show overcloud Controller resources.Controller: Property error: resources[1].properties: Property MysqlClustercheckPassword not assigned heat resource-show overcloud Compute (same message for CephStorage) resource_status_reason | resources.Compute: resources[0]: ResourceInError: resources.NovaCompute: Went to status ERROR due to "Message: No valid host was found. There are not enough hosts available., Code: 500 Nova list shows duplicated entries: [stack@instack ~]$ nova list +--------------------------------------+-------------------------+--------+------------+-------------+---------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+-------------------------+--------+------------+-------------+---------------------+ | 68588bb2-6fb2-4b3f-ba6b-60ede4c0382c | overcloud-cephstorage-0 | ACTIVE | - | Running | ctlplane=192.0.2.7 | | 765f5c23-c141-4a89-bd75-21affac9c6e0 | overcloud-cephstorage-0 | ERROR | - | NOSTATE | | | 4f2c137a-71d2-42ff-9f34-60ed22d1d28a | overcloud-compute-0 | ACTIVE | - | Running | ctlplane=192.0.2.9 | | 935d65b9-b061-4ee0-aa4c-58ba73449e1c | overcloud-compute-0 | ERROR | - | NOSTATE | | | f9b7506d-3d25-4a1e-b968-262b6fcd2345 | overcloud-controller-0 | ACTIVE | - | Running | ctlplane=192.0.2.10 | | a1d9831a-65bb-4c2f-a843-155452c0e3f1 | overcloud-controller-1 | ACTIVE | - | Running | ctlplane=192.0.2.8 | | c54eabf8-dcd1-4798-ab2f-1944a34583f2 | overcloud-controller-2 | ACTIVE | - | Running | ctlplane=192.0.2.11 | +--------------------------------------+-------------------------+--------+------------+-------------+---------------------+ Expected result: Successful update.
Reproduced on attemot to update 7.2GA->7.3 async. FAILED update finished with status FAILED [stack@instack ~]$ heat stack-list +--------------------------------------+------------+---------------+----------------------+ | id | stack_name | stack_status | creation_time | +--------------------------------------+------------+---------------+----------------------+ | ea0b8806-806a-4f0a-a8e4-c5e5c191c4c0 | overcloud | UPDATE_FAILED | 2016-06-28T23:42:20Z | +--------------------------------------+------------+---------------+----------------------+ [stack@instack ~]$ heat resource-list -n5 overcloud ^C... terminating heat client [stack@instack ~]$ heat resource-list -n5 overcloud|grep -v COMPLE +-----------------------------------------------+-----------------------------------------------+---------------------------------------------------+-----------------+----------------------+-----------------------------------------------+ | resource_name | physical_resource_id | resource_type | resource_status | updated_time | parent_resource | +-----------------------------------------------+-----------------------------------------------+---------------------------------------------------+-----------------+----------------------+-----------------------------------------------+ | Controller | 7b933db1-dd35-4473-a498-f359358a56da | OS::Heat::ResourceGroup | UPDATE_FAILED | 2016-06-29T01:19:02Z | | +-----------------------------------------------+-----------------------------------------------+---------------------------------------------------+-----------------+----------------------+-----------------------------------------------+ [stack@instack ~]$ heat resource-show overcloud Controller +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+ | Property | Value | +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+ | attributes | { | | | "attributes": null, | | | "refs": null | | | } | | description | | | links | http://192.0.2.1:8004/v1/bc250ff1287c4cfea7d1e852e9b129d3/stacks/overcloud/ea0b8806-806a-4f0a-a8e4-c5e5c191c4c0/resources/Controller (self) | | | http://192.0.2.1:8004/v1/bc250ff1287c4cfea7d1e852e9b129d3/stacks/overcloud/ea0b8806-806a-4f0a-a8e4-c5e5c191c4c0 (stack) | | | http://192.0.2.1:8004/v1/bc250ff1287c4cfea7d1e852e9b129d3/stacks/overcloud-Controller-dz6bhlucp7aw/7b933db1-dd35-4473-a498-f359358a56da (nested) | | logical_resource_id | Controller | | physical_resource_id | 7b933db1-dd35-4473-a498-f359358a56da | | required_by | ControllerClusterConfig | | | ControllerClusterDeployment | | | ControllerIpListMap | | | CephClusterConfig | | | AllNodesExtraConfig | | | ControllerCephDeployment | | | ControllerBootstrapNodeConfig | | | ControllerSwiftDeployment | | | ControllerAllNodesDeployment | | | VipDeployment | | | SwiftDevicesAndProxyConfig | | | AllNodesValidationConfig | | | ControllerNodesPostDeployment | | | ControllerBootstrapNodeDeployment | | | ControllerAllNodesValidationDeployment | | | allNodesConfig | | resource_name | Controller | | resource_status | UPDATE_FAILED | | resource_status_reason | resources.Controller: Property error: resources[1].properties: Property MysqlClustercheckPassword not assigned | | resource_type | OS::Heat::ResourceGroup | | updated_time | 2016-06-29T01:19:02Z | +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+ Environment: openstack-tripleo-heat-templates-0.8.6-127.el7ost.noarch openstack-puppet-modules-2015.1.8-51.el7ost.noarch instack-undercloud-2.1.2-39.el7ost.noarch
Hi Sasha, would you be able to add the following one-liner patch to one of your update jobs and report back? https://review.openstack.org/337304 Thanks, Michele
Hi Michele, So after editing the file /usr/share/openstack-tripleo-heat-templates/puppet/controller-puppet.yaml to have this: MysqlClustercheckPassword: type: string hidden: true default: '' I encountered this BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1353019
Also reproduced the same error when tried to update 8.0GA with ipv6 to 8.0Async
*** Bug 1353019 has been marked as a duplicate of this bug. ***
Verified: Environment: openstack-tripleo-heat-templates-0.8.14-16.el7ost.noarch The reported issue doesn't reproduce.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-1768.html