Bug 1469132 - [osp12]Cannot scale compute node on HA environment
[osp12]Cannot scale compute node on HA environment
Status: ON_QA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates (Show other bugs)
12.0 (Pike)
x86_64 Linux
high Severity high
: beta
: 12.0 (Pike)
Assigned To: Emilien Macchi
Gurenko Alex
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-07-10 08:54 EDT by Artem Hrechanychenko
Modified: 2017-10-23 13:03 EDT (History)
7 users (show)

See Also:
Fixed In Version: openstack-tripleo-heat-templates-7.0.0-0.20170718190543.el7ost puppet-tripleo-7.1.1-0.20170719150224.cb66aeb.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
openstack stack failures list (2.63 MB, text/plain)
2017-07-10 08:54 EDT, Artem Hrechanychenko
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
OpenStack gerrit 476153 None None None 2017-07-11 10:19 EDT
OpenStack gerrit 477535 None None None 2017-07-11 10:20 EDT

  None (edit)
Description Artem Hrechanychenko 2017-07-10 08:54:39 EDT
Created attachment 1295817 [details]
openstack stack failures list

Description of problem:
Cannot scale compute on HA environment

overcloud.AllNodesDeploySteps.ControllerDeployment_Step1.1:
  resource_type: OS::Heat::StructuredDeployment
  physical_resource_id: 56472e57-9af7-4033-8e66-a94f86efeaf8
  status: CREATE_FAILED
  status_reason: |
    Error: resources[1]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2

(undercloud) [stack@undercloud-0 ~]$ nova list
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for 192.168.24.2 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for 192.168.24.2 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
+--------------------------------------+--------------+--------+------------+-------------+------------------------+
| ID                                   | Name         | Status | Task State | Power State | Networks               |
+--------------------------------------+--------------+--------+------------+-------------+------------------------+
| 307275b6-987a-4b84-9797-e118a6d56e90 | compute-0    | ACTIVE | -          | Running     | ctlplane=192.168.24.12 |
| 8f4f87d6-0db6-4cc9-87d3-796125a07446 | compute-1    | ACTIVE | -          | Running     | ctlplane=192.168.24.21 |
| e012ae3f-9123-444b-8500-3be25a401444 | controller-0 | ACTIVE | -          | Running     | ctlplane=192.168.24.9  |
| 9197ec9e-a9c8-46bc-81b5-90eda7116c02 | controller-1 | ACTIVE | -          | Running     | ctlplane=192.168.24.18 |
| 85514472-036d-47e2-944c-cadb96852100 | controller-2 | ACTIVE | -          | Running     | ctlplane=192.168.24.14 |
+--------------------------------------+--------------+--------+------------+-------------+------------------------+

| AllNodesDeploySteps                       | e9cf1c3c-e477-4813-a20e-43c1c626b30a                     | OS::TripleO::PostDeploySteps                     | UPDATE_FAILED   | 2017-07-10T12:11:08Z |




Version-Release number of selected component (if applicable):
OSP12
openstack-nova-compute-16.0.0-0.20170627215041.el7ost.noarch
openstack-neutron-openvswitch-11.0.0-0.20170628060509.9a72bfe.el7ost.noarch
openstack-ironic-inspector-5.1.1-0.20170627095652.c172b2e.el7ost.noarch
openstack-swift-object-2.14.1-0.20170622024006.2d18ecd.el7ost.noarch
openstack-ironic-api-8.0.1-0.20170628015314.3521203.el7ost.noarch
openstack-tripleo-puppet-elements-7.0.0-0.20170614005502.9285877.el7ost.noarch
openstack-mistral-executor-5.0.0-0.20170627151830.fe1a1e1.el7ost.noarch
openstack-nova-api-16.0.0-0.20170627215041.el7ost.noarch
openstack-nova-conductor-16.0.0-0.20170627215041.el7ost.noarch
openstack-glance-15.0.0-0.20170623215940.8188eca.el7ost.noarch
openstack-nova-common-16.0.0-0.20170627215041.el7ost.noarch
puppet-openstacklib-11.2.0-0.20170626053453.8264861.el7ost.noarch
openstack-tripleo-image-elements-7.0.0-0.20170607161959.401d861.el7ost.noarch
python-openstacksdk-0.9.17-0.20170621195806.7946243.el7ost.noarch
openstack-swift-container-2.14.1-0.20170622024006.2d18ecd.el7ost.noarch
openstack-keystone-12.0.0-0.20170628062203.e4c0c8d.el7ost.noarch
openstack-heat-api-9.0.0-0.20170628061551.30c87ba.el7ost.noarch
openstack-ironic-conductor-8.0.1-0.20170628015314.3521203.el7ost.noarch
openstack-tempest-16.0.1-0.20170628065146.f6288d7.el7ost.noarch
openstack-mistral-common-5.0.0-0.20170627151830.fe1a1e1.el7ost.noarch
openstack-tripleo-ui-7.1.1-0.20170627161345.68e8684.el7ost.noarch
openstack-nova-scheduler-16.0.0-0.20170627215041.el7ost.noarch
openstack-puppet-modules-10.0.0-0.20170315222135.0333c73.el7.1.noarch
openstack-tripleo-heat-templates-7.0.0-0.20170628002128.el7ost.noarch
openstack-tripleo-common-7.2.1-0.20170628001837.5a6423b.el7ost.noarch
openstack-swift-account-2.14.1-0.20170622024006.2d18ecd.el7ost.noarch
openstack-neutron-ml2-11.0.0-0.20170628060509.9a72bfe.el7ost.noarch
openstack-swift-proxy-2.14.1-0.20170622024006.2d18ecd.el7ost.noarch
openstack-heat-api-cfn-9.0.0-0.20170628061551.30c87ba.el7ost.noarch
openstack-mistral-api-5.0.0-0.20170627151830.fe1a1e1.el7ost.noarch
openstack-zaqar-5.0.0-0.20170626084003.b1cd63c.el7ost.noarch
openstack-neutron-11.0.0-0.20170628060509.9a72bfe.el7ost.noarch
openstack-heat-engine-9.0.0-0.20170628061551.30c87ba.el7ost.noarch
openstack-ironic-common-8.0.1-0.20170628015314.3521203.el7ost.noarch
openstack-mistral-engine-5.0.0-0.20170627151830.fe1a1e1.el7ost.noarch
python-openstack-mistral-5.0.0-0.20170627151830.fe1a1e1.el7ost.noarch
openstack-selinux-0.8.8-0.20170622195307.74ddc0e.el7ost.noarch
openstack-nova-placement-api-16.0.0-0.20170627215041.el7ost.noarch
puppet-openstack_extras-11.2.0-0.20170626060428.de67a39.el7ost.noarch
openstack-neutron-common-11.0.0-0.20170628060509.9a72bfe.el7ost.noarch
openstack-heat-common-9.0.0-0.20170628061551.30c87ba.el7ost.noarch
python-openstackclient-3.11.0-0.20170613232431.c69304e.el7ost.noarch
openstack-tripleo-validations-7.1.1-0.20170627224206.591a65f.el7ost.noarch
openstack-tripleo-common-containers-7.2.1-0.20170628001837.5a6423b.el7ost.noarch


How reproducible:


Steps to Reproduce:
1.Deploy undercloud OSP12 with SSL
2. Deploy 3ctrl+1compute overcloud using http://etherpad.corp.redhat.com/testing-osp12-containers-ha from Line 343 but with using 1 compute node
http://pastebin.test.redhat.com/501524
http://pastebin.test.redhat.com/501525


3.Scale up with 2 compute node 


Actual results:
stack update failed

Expected results:
stack updated with 1 compute node

Additional info:
openstack stack failures list in attachment
Comment 3 Michele Baldessari 2017-07-11 04:49:29 EDT
So the error seems to be:
rsync: rename failed for \"/var/lib/config-data/haproxy/etc/hostname\" (from etc/.~tmp~/hostname): Device or resource busy (16)", "rsync: rename failed for \"/va    r/lib/config-data/haproxy/etc/hosts\" (from etc/.~tmp~/hosts): Device or resource busy (16)", "rsync: rename failed for \"/var/lib/config-data/haproxy/etc/resolv.conf\" (from etc/.~tmp~/    resolv.conf): Device or resource busy (16)", "rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1052) [sender=3.0.9]

I think https://review.openstack.org/#/c/477535/ should fix this
Comment 4 Martin André 2017-07-17 10:43:30 EDT
Moving to POST since the all the patches merged upstream.

Note You need to log in before you can comment on or make changes to this bug.