Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1290574

Summary: Scaling out an update overcloud (7.1 -> 7.2) doesn't provision a new node
Product: Red Hat OpenStack Reporter: Marius Cornea <mcornea>
Component: rhosp-directorAssignee: Giulio Fidente <gfidente>
Status: CLOSED ERRATA QA Contact: Marius Cornea <mcornea>
Severity: unspecified Docs Contact:
Priority: urgent    
Version: 7.0 (Kilo)CC: dmacpher, gfidente, jcoufal, jslagle, mburns, mcornea, rhel-osp-director-maint, sbaker
Target Milestone: y2   
Target Release: 7.0 (Kilo)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: python-rdomanager-oscplugin-0.0.10-22.el7ost Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-12-21 16:54:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
deploy debug none

Description Marius Cornea 2015-12-10 20:35:53 UTC 
Description of problem:
I'm trying to add an additional compute node to an updated overcloud but no new node gets provisioned. After running the deploy command with +1 compute nodes for the 2nd time(BZ#1290572) the deploy finishes successfully but there's still only one compute node provisioned.  

Version-Release number of selected component (if applicable):
python-rdomanager-oscplugin-0.0.10-21.el7ost.noarch
penstack-heat-api-2015.1.2-4.el7ost.noarch
openstack-heat-api-cloudwatch-2015.1.2-4.el7ost.noarch
openstack-tripleo-heat-templates-0.8.6-91.el7ost.noarch
heat-cfntools-1.2.8-2.el7.noarch
openstack-heat-common-2015.1.2-4.el7ost.noarch
openstack-heat-api-cfn-2015.1.2-4.el7ost.noarch
python-heatclient-0.6.0-1.el7ost.noarch
openstack-heat-engine-2015.1.2-4.el7ost.noarch
openstack-heat-templates-0-0.8.20150605git.el7ost.noarch

1. Deploy 7.1 by using 7.1 templates:
openstack overcloud deploy \
    --templates ~/templates/my-overcloud \
    --control-scale 3 --compute-scale 1 --ceph-storage-scale 3 \
    --ntp-server clock.redhat.com \
    --libvirt-type qemu \
    -e ~/templates/my-overcloud/environments/network-isolation.yaml \
    -e ~/templates/network-environment.yaml \
    -e ~/templates/firstboot-environment.yaml \
    -e ~/templates/ceph.yaml 

2. Update the undercloud to 7.2 and run the update procedure to 7.2 with 7.2 templates:
/usr/bin/yes '' | openstack overcloud update stack overcloud -i \
         --templates ~/templates/my-overcloud \
         -e ~/templates/my-overcloud/overcloud-resource-registry-puppet.yaml \
         -e ~/templates/my-overcloud/environments/network-isolation.yaml \
         -e ~/templates/network-environment.yaml \
         -e ~/templates/firstboot-environment.yaml \
         -e ~/templates/ceph.yaml \
         -e ~/templates/my-overcloud/environments/updates/update-from-vip.yaml \
         -e ~/templates/ctrlport.yaml

Wait for the update to complete

3. Try to scale out with an additional node:

openstack overcloud deploy \
    --templates ~/templates/my-overcloud \
    --control-scale 3 --compute-scale 2 --ceph-storage-scale 3 \
    --ntp-server clock.redhat.com \
    --libvirt-type qemu \
    -e ~/templates/my-overcloud/overcloud-resource-registry-puppet.yaml \
    -e ~/templates/my-overcloud/environments/network-isolation.yaml \
    -e ~/templates/network-environment.yaml \
    -e ~/templates/firstboot-environment.yaml \
    -e ~/templates/ceph.yaml \
    -e ~/templates/my-overcloud/environments/updates/update-from-vip.yaml \
    -e ~/templates/ctrlport.yaml

Actual results:
stack@instack:~>>> nova list
+--------------------------------------+-------------------------+--------+------------+-------------+---------------------+
| ID                                   | Name                    | Status | Task State | Power State | Networks            |
+--------------------------------------+-------------------------+--------+------------+-------------+---------------------+
| 4254a40d-22e7-4166-b6ec-9704d6683e7e | overcloud-cephstorage-0 | ACTIVE | -          | Running     | ctlplane=192.0.2.15 |
| 8f3f8394-e31d-4460-9ac0-6aea16eb9cd1 | overcloud-cephstorage-1 | ACTIVE | -          | Running     | ctlplane=192.0.2.16 |
| 57c61042-a99a-40f7-83b1-61e9deab5d4d | overcloud-cephstorage-2 | ACTIVE | -          | Running     | ctlplane=192.0.2.17 |
| c271fd10-dded-4726-8ec4-03cbb0ba9a6e | overcloud-compute-0     | ACTIVE | -          | Running     | ctlplane=192.0.2.19 |
| 9f9fbcc1-7fd7-4e1a-afa5-a76909e989da | overcloud-controller-0  | ACTIVE | -          | Running     | ctlplane=192.0.2.20 |
| d94e597e-b01a-4804-ac63-9a0ac07cedfd | overcloud-controller-1  | ACTIVE | -          | Running     | ctlplane=192.0.2.18 |
| 98bf473a-d440-4caa-9978-cee51f1882ea | overcloud-controller-2  | ACTIVE | -          | Running     | ctlplane=192.0.2.21 |
+--------------------------------------+-------------------------+--------+------------+-------------+---------------------+

stack@instack:~>>> heat stack-show overcloud  | grep ComputeCount
|                       |   "ComputeCount": "1",                                   

Expected results:


Additional info:
A number of 2 compute nodes get provisioned.
Comment 1 James Slagle 2015-12-10 21:43:57 UTC 
can you attach /var/log/heat/heat-api.log and /var/log/heat/heat-engine.log here?
Comment 3 Marius Cornea 2015-12-11 13:47:24 UTC 
Created attachment 1104693 [details]
deploy debug

Attaching the deploy command debug log. Command that I ran:

openstack overcloud deploy \
    --templates ~/templates/my-overcloud \
    --control-scale 3 --compute-scale 2 --ceph-storage-scale 3 \
    --ntp-server clock.redhat.com \
    --libvirt-type qemu \
    -e ~/templates/my-overcloud/overcloud-resource-registry-puppet.yaml \
    -e ~/templates/my-overcloud/environments/network-isolation.yaml \
    -e ~/templates/network-environment.yaml \
    -e ~/templates/firstboot-environment.yaml \
    -e ~/templates/ceph.yaml \
    -e ~/templates/my-overcloud/environments/updates/update-from-vip.yaml \
    -e ~/templates/ctrlport.yaml \
    --debug \
    --log-file scaleout-debug.log
Comment 4 Marius Cornea 2015-12-11 14:50:54 UTC 
FWIW, I tested scale out with a fresh deployed 7.2 and it completed fine. The difference here is that I didn't pass the update environment files so maybe that's causing the failure during the update flow.
Comment 11 errata-xmlrpc 2015-12-21 16:54:42 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2015:2651