Bug 1321364

Summary: Live migration from compute running juno to compute running kilo fails
Product: Red Hat OpenStack Reporter: Jeremy <jmelvin>
Component: openstack-novaAssignee: Lee Yarwood <lyarwood>
Status: CLOSED INSUFFICIENT_DATA QA Contact: nlevinki <nlevinki>
Severity: high Docs Contact:
Priority: high    
Version: 7.0 (Kilo)CC: berrange, dasmith, eglynn, jmelvin, kchamart, lyarwood, sbauza, sferdjao, sgordon, srevivo, vromanso
Target Milestone: ---Keywords: Unconfirmed, ZStream
Target Release: 7.0 (Kilo)Flags: lyarwood: needinfo? (jmelvin)
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-06-27 13:42:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Jeremy 2016-03-25 19:44:29 UTC 
Description of problem:
Customer is attempting live migrations of 2 instances  from compute 104 and 106. He has already migrated all the other instances and there are just 2 remaining that do not work. Also note that these 2 compute nodes 104 and 106 are juno while the rest including 107 are kilo, which is why he is trying to migrate the instances off for the upgrades. 
Migration fails with the little errors:
[instance: 31c80d84-ec38-4ede-aa9a-7d9afcd596f8
compute106 source compute
2016-03-24 14:12:24.686 20909 DEBUG nova.virt.libvirt.driver [req-0a8ec242-6e65-4769-8eca-fc386821361b None] [instance: 31c80d84-ec38-4ede-aa9a-7d9afcd596f8] Could not determine fibre channel world wide port names get_volume_connector /usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:1297
2016-03-24 14:12:24.687 20909 DEBUG nova.volume.cinder [req-0a8ec242-6e65-4769-8eca-fc386821361b None] Cinderclient connection created using URL: http://10.47.129.24:8776/v2/3244c329ec6444828cf82e46d927ff3a get_cinder_client_version /usr/lib/python2.7/site-packages/nova/volume/cinder.py:255
2016-03-24 14:12:25.448 20909 WARNING nova.context [-] Arguments dropped when creating context: {u'read_only': False, u'domain': None, u'show_deleted': False, u'user_identity': u'b334764d47cb4ce6bf5ef86eebfcc54e 3244c329ec6444828cf82e46d927ff3a - - -', u'project_domain': None, u'resource_uuid': None, u'user_domain': None}
2016-03-24 14:12:27.097 20909 ERROR nova.compute.manager [req-0a8ec242-6e65-4769-8eca-fc386821361b None] [instance: 31c80d84-ec38-4ede-aa9a-7d9afcd596f8] Pre live migration failed at wdckvm107a

compute107 destination compute
2016-03-24 14:12:25.625 1454 DEBUG oslo_concurrency.processutils [req-0a8ec242-6e65-4769-8eca-fc386821361b b334764d47cb4ce6bf5ef86eebfcc54e 3244c329ec6444828cf82e46d927ff3a - - -] u'sudo nova-rootwrap /etc/nova/rootwrap.conf systool -c fc_host -v' failed. Not Retrying. execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:291
2016-03-24 14:12:25.626 1454 DEBUG nova.virt.libvirt.driver [req-0a8ec242-6e65-4769-8eca-fc386821361b b334764d47cb4ce6bf5ef86eebfcc54e 3244c329ec6444828cf82e46d927ff3a - - -] [instance: 31c80d84-ec38-4ede-aa9a-7d9afcd596f8] Could not determine fibre channel world wide port names get_volume_connector /usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:1008


Version-Release number of selected component (if applicable):
openstack-nova-compute-2014.2.3-9.el7ost.noarch (compute 106 juno)
openstack-nova-compute-2015.1.2-13.el7ost.noarch (compute 107 kilo)

How reproducible:
100%

Steps to Reproduce:
1. Live migrate from the commmand line form compute 106 to compute 107
2.note failure
3.

Actual results:
fails

Expected results:
live migrates

Additional info:

Cinder backend is ceph rbd, so fibre channel should not be used. Not sure of the messages regarding fibre channel