Created attachment 1295817 [details] openstack stack failures list Description of problem: Cannot scale compute on HA environment overcloud.AllNodesDeploySteps.ControllerDeployment_Step1.1: resource_type: OS::Heat::StructuredDeployment physical_resource_id: 56472e57-9af7-4033-8e66-a94f86efeaf8 status: CREATE_FAILED status_reason: | Error: resources[1]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2 (undercloud) [stack@undercloud-0 ~]$ nova list /usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for 192.168.24.2 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.) SubjectAltNameWarning /usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for 192.168.24.2 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.) SubjectAltNameWarning +--------------------------------------+--------------+--------+------------+-------------+------------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+--------------+--------+------------+-------------+------------------------+ | 307275b6-987a-4b84-9797-e118a6d56e90 | compute-0 | ACTIVE | - | Running | ctlplane=192.168.24.12 | | 8f4f87d6-0db6-4cc9-87d3-796125a07446 | compute-1 | ACTIVE | - | Running | ctlplane=192.168.24.21 | | e012ae3f-9123-444b-8500-3be25a401444 | controller-0 | ACTIVE | - | Running | ctlplane=192.168.24.9 | | 9197ec9e-a9c8-46bc-81b5-90eda7116c02 | controller-1 | ACTIVE | - | Running | ctlplane=192.168.24.18 | | 85514472-036d-47e2-944c-cadb96852100 | controller-2 | ACTIVE | - | Running | ctlplane=192.168.24.14 | +--------------------------------------+--------------+--------+------------+-------------+------------------------+ | AllNodesDeploySteps | e9cf1c3c-e477-4813-a20e-43c1c626b30a | OS::TripleO::PostDeploySteps | UPDATE_FAILED | 2017-07-10T12:11:08Z | Version-Release number of selected component (if applicable): OSP12 openstack-nova-compute-16.0.0-0.20170627215041.el7ost.noarch openstack-neutron-openvswitch-11.0.0-0.20170628060509.9a72bfe.el7ost.noarch openstack-ironic-inspector-5.1.1-0.20170627095652.c172b2e.el7ost.noarch openstack-swift-object-2.14.1-0.20170622024006.2d18ecd.el7ost.noarch openstack-ironic-api-8.0.1-0.20170628015314.3521203.el7ost.noarch openstack-tripleo-puppet-elements-7.0.0-0.20170614005502.9285877.el7ost.noarch openstack-mistral-executor-5.0.0-0.20170627151830.fe1a1e1.el7ost.noarch openstack-nova-api-16.0.0-0.20170627215041.el7ost.noarch openstack-nova-conductor-16.0.0-0.20170627215041.el7ost.noarch openstack-glance-15.0.0-0.20170623215940.8188eca.el7ost.noarch openstack-nova-common-16.0.0-0.20170627215041.el7ost.noarch puppet-openstacklib-11.2.0-0.20170626053453.8264861.el7ost.noarch openstack-tripleo-image-elements-7.0.0-0.20170607161959.401d861.el7ost.noarch python-openstacksdk-0.9.17-0.20170621195806.7946243.el7ost.noarch openstack-swift-container-2.14.1-0.20170622024006.2d18ecd.el7ost.noarch openstack-keystone-12.0.0-0.20170628062203.e4c0c8d.el7ost.noarch openstack-heat-api-9.0.0-0.20170628061551.30c87ba.el7ost.noarch openstack-ironic-conductor-8.0.1-0.20170628015314.3521203.el7ost.noarch openstack-tempest-16.0.1-0.20170628065146.f6288d7.el7ost.noarch openstack-mistral-common-5.0.0-0.20170627151830.fe1a1e1.el7ost.noarch openstack-tripleo-ui-7.1.1-0.20170627161345.68e8684.el7ost.noarch openstack-nova-scheduler-16.0.0-0.20170627215041.el7ost.noarch openstack-puppet-modules-10.0.0-0.20170315222135.0333c73.el7.1.noarch openstack-tripleo-heat-templates-7.0.0-0.20170628002128.el7ost.noarch openstack-tripleo-common-7.2.1-0.20170628001837.5a6423b.el7ost.noarch openstack-swift-account-2.14.1-0.20170622024006.2d18ecd.el7ost.noarch openstack-neutron-ml2-11.0.0-0.20170628060509.9a72bfe.el7ost.noarch openstack-swift-proxy-2.14.1-0.20170622024006.2d18ecd.el7ost.noarch openstack-heat-api-cfn-9.0.0-0.20170628061551.30c87ba.el7ost.noarch openstack-mistral-api-5.0.0-0.20170627151830.fe1a1e1.el7ost.noarch openstack-zaqar-5.0.0-0.20170626084003.b1cd63c.el7ost.noarch openstack-neutron-11.0.0-0.20170628060509.9a72bfe.el7ost.noarch openstack-heat-engine-9.0.0-0.20170628061551.30c87ba.el7ost.noarch openstack-ironic-common-8.0.1-0.20170628015314.3521203.el7ost.noarch openstack-mistral-engine-5.0.0-0.20170627151830.fe1a1e1.el7ost.noarch python-openstack-mistral-5.0.0-0.20170627151830.fe1a1e1.el7ost.noarch openstack-selinux-0.8.8-0.20170622195307.74ddc0e.el7ost.noarch openstack-nova-placement-api-16.0.0-0.20170627215041.el7ost.noarch puppet-openstack_extras-11.2.0-0.20170626060428.de67a39.el7ost.noarch openstack-neutron-common-11.0.0-0.20170628060509.9a72bfe.el7ost.noarch openstack-heat-common-9.0.0-0.20170628061551.30c87ba.el7ost.noarch python-openstackclient-3.11.0-0.20170613232431.c69304e.el7ost.noarch openstack-tripleo-validations-7.1.1-0.20170627224206.591a65f.el7ost.noarch openstack-tripleo-common-containers-7.2.1-0.20170628001837.5a6423b.el7ost.noarch How reproducible: Steps to Reproduce: 1.Deploy undercloud OSP12 with SSL 2. Deploy 3ctrl+1compute overcloud using http://etherpad.corp.redhat.com/testing-osp12-containers-ha from Line 343 but with using 1 compute node http://pastebin.test.redhat.com/501524 http://pastebin.test.redhat.com/501525 3.Scale up with 2 compute node Actual results: stack update failed Expected results: stack updated with 1 compute node Additional info: openstack stack failures list in attachment
So the error seems to be: rsync: rename failed for \"/var/lib/config-data/haproxy/etc/hostname\" (from etc/.~tmp~/hostname): Device or resource busy (16)", "rsync: rename failed for \"/va r/lib/config-data/haproxy/etc/hosts\" (from etc/.~tmp~/hosts): Device or resource busy (16)", "rsync: rename failed for \"/var/lib/config-data/haproxy/etc/resolv.conf\" (from etc/.~tmp~/ resolv.conf): Device or resource busy (16)", "rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1052) [sender=3.0.9] I think https://review.openstack.org/#/c/477535/ should fix this
Moving to POST since the all the patches merged upstream.
Verified openstack-tripleo-heat-templates-7.0.3-0.20171024200823.el7ost.noarch [stack@undercloud-0 ~]$ source stackrc (undercloud) [stack@undercloud-0 ~]$ nova list +--------------------------------------+--------------+--------+------------+-------------+------------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+--------------+--------+------------+-------------+------------------------+ | 6fb04d2a-fa32-413a-b8ff-a3416a5a285b | compute-0 | ACTIVE | - | Running | ctlplane=192.168.24.11 | | 10e51128-2226-4857-b2bb-200c948e8c12 | compute-1 | ACTIVE | - | Running | ctlplane=192.168.24.18 | | ef8ee19e-aed1-4785-a600-833da5b0d802 | controller-0 | ACTIVE | - | Running | ctlplane=192.168.24.10 | | 5c8bb574-4a3e-4dbb-a4d8-8bfb34f7c5d3 | controller-1 | ACTIVE | - | Running | ctlplane=192.168.24.6 | | f67299d1-b970-454c-afc2-07a363d431e6 | controller-2 | ACTIVE | - | Running | ctlplane=192.168.24.16 | +--------------------------------------+--------------+--------+------------+-------------+------------------------+ (undercloud) [stack@undercloud-0 ~]$ heat stack-list WARNING (shell) "heat stack-list" is deprecated, please use "openstack stack list" instead +--------------------------------------+------------+-----------------+----------------------+----------------------+----------------------------------+ | id | stack_name | stack_status | creation_time | updated_time | project | +--------------------------------------+------------+-----------------+----------------------+----------------------+----------------------------------+ | 1a683c36-523a-4c03-9990-0e06d80e940a | overcloud | UPDATE_COMPLETE | 2017-11-06T13:37:44Z | 2017-11-06T14:26:17Z | cefc730dd8464a5581ce95f5a8685740 | +--------------------------------------+------------+-----------------+----------------------+----------------------+----------------------------------+
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:3462