Bug 1425472

Summary: OSP10 -> OSP11 upgrade: unable to live migrate instances after upgrading controller services(major-upgrade-composable-steps.yaml)
Product: Red Hat OpenStack Reporter: Marius Cornea <mcornea>
Component: rhosp-directorAssignee: Angus Thomas <athomas>
Status: CLOSED DUPLICATE QA Contact: Amit Ugol <augol>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 11.0 (Ocata)CC: dbecker, mandreou, mburns, morazi, rhel-osp-director-maint
Target Milestone: ga   
Target Release: 11.0 (Ocata)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-02-23 14:39:14 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
controller-2
none
controller-1
none
controller-0
none
compute-1
none
compute-0
none
nova logs with debug on on controller-0 none

Description Marius Cornea 2017-02-21 14:06:24 UTC 
Description of problem:
After upgrading the control plane services(major-upgrade-composable-steps.yaml) I am unable to live migrate instances from compute nodes in order to proceed with upgrading the computes.

Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:

1. Upgrade control plane to OSP11
source ~/stackrc
export THT=/usr/share/openstack-tripleo-heat-templates/

openstack overcloud deploy --templates $THT \
-e $THT/environments/network-isolation.yaml \
-e $THT/environments/network-management.yaml \
-e $THT/environments/storage-environment.yaml \
-e ~/openstack_deployment/environments/nodes.yaml \
-e ~/openstack_deployment/environments/network-environment.yaml \
-e ~/openstack_deployment/environments/disk-layout.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/major-upgrade-composable-steps.yaml \
-e ~/repo.yaml \
-e ~/openstack_deployment/environments/neutron-settings.yaml

2. Check compute node where instance is running:

openstack server show st--b5-instance-aw6cnn724t3s-my_instance-243tyxisa7su | grep hypervisor_hostname
| OS-EXT-SRV-ATTR:hypervisor_hostname | overcloud-compute-0.localdomain 

3. Migrate instance to different node
nova live-migration st--b5-instance-aw6cnn724t3s-my_instance-243tyxisa7su overcloud-compute-1.localdomain

4. Check instance status
openstack server list | grep st--b5-instance-aw6cnn724t3s-my_instance-243tyxisa7su
| ba9a4eb3-b3c7-42dd-9ddc-038ffbfe3623 | st--b5-instance-aw6cnn724t3s-my_instance-243tyxisa7su | ACTIVE | stack-b5-tenant_net_ext_tagged-4i7mw5fxjdox-private_network-fudvkibznbj3=10.10.10.8, 172.16.18.141 | Fedora     |

5. Check hypervisor where instance is running:
openstack server show st--b5-instance-aw6cnn724t3s-my_instance-243tyxisa7su | grep hypervisor_hostname
| OS-EXT-SRV-ATTR:hypervisor_hostname | overcloud-compute-0.localdomain              
 
Actual results:
Instance is running on the initial compute node(overcloud-compute-0.localdomain  )

Expected results:
Instance should run on the node where I requested live-migration to move(overcloud-compute-1.localdomain) 

Additional info:
Attaching and config files from compute nodes and controllers.
Comment 1 Marius Cornea 2017-02-21 14:13:48 UTC 
Created attachment 1256151 [details]
controller-2
Comment 2 Marius Cornea 2017-02-21 14:14:21 UTC 
Created attachment 1256152 [details]
controller-1
Comment 3 Marius Cornea 2017-02-21 14:14:53 UTC 
Created attachment 1256153 [details]
controller-0
Comment 4 Marius Cornea 2017-02-21 14:19:46 UTC 
Created attachment 1256158 [details]
compute-1
Comment 5 Marius Cornea 2017-02-21 14:20:11 UTC 
Created attachment 1256159 [details]
compute-0
Comment 6 Marius Cornea 2017-02-21 14:58:24 UTC 
Created attachment 1256164 [details]
nova logs with debug on on controller-0

nova logs with debug on on controller-0
Comment 7 Marios Andreou 2017-02-23 10:57:07 UTC 
Hey Marius, had a quick look at the logs but can't see something obvious... I am wondering (as we just discussed on irc) if this may be related to BZ 1406826 (the compute nova names will have changed to overcloud-novacompute-X after the composable upgade step is completed)
Comment 8 Marius Cornea 2017-02-23 14:39:14 UTC 

*** This bug has been marked as a duplicate of bug 1406826 ***