Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1613449

Summary: [12][IPv6]Live migration failed: Unable to parse URI qemu+ssh://nova_migration@fd00:
Product: Red Hat OpenStack Reporter: Yurii Prokulevych <yprokule>
Component: openstack-novaAssignee: Lee Yarwood <lyarwood>
Status: CLOSED WONTFIX QA Contact: OSP DFG:Compute <osp-dfg-compute>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 12.0 (Pike)CC: augol, berrange, ccamacho, dasmith, eglynn, jhakimra, kchamart, lyarwood, owalsh, sbauza, sgolovat, sgordon, srevivo, vromanso
Target Milestone: ---Keywords: Triaged, ZStream
Target Release: 12.0 (Pike)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1614447 (view as bug list) Environment:
Last Closed: 2018-11-15 11:23:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1614447, 1614451    

Description Yurii Prokulevych 2018-08-07 14:51:34 UTC 
Description of problem:
-----------------------
Attempt to migrate workload to upgraded compute failed:
...
2018-08-07 13:52:37.234 34324 ERROR nova.virt.libvirt.driver [req-7751982a-1f31-4dbd-a01e-7ec63e783489 6177334513c94044857220f0bfc78b22 87ab28106b594ec083a8a701d2741714 - - -] [instance: 6f3925d4-2d6c-4d64-8644-
a48bfe26c694] Live Migration failure: internal error: Unable to parse URI qemu+ssh://nova_migration@fd00:fd00:fd00:2000::1e/system?keyfile=/etc/nova/migration/identity
2018-08-07 13:52:37.234 34324 DEBUG nova.virt.libvirt.driver [req-7751982a-1f31-4dbd-a01e-7ec63e783489 6177334513c94044857220f0bfc78b22 87ab28106b594ec083a8a701d2741714 - - -] [instance: 6f3925d4-2d6c-4d64-8644-
a48bfe26c694] Migration operation thread notification thread_finished /usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:6612
2018-08-07 13:52:37.279 34324 DEBUG oslo_messaging._drivers.amqpdriver [-] received reply msg_id: 82b5c77aa5eb4733b8df322821e86e4f __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:
419
2018-08-07 13:52:37.289 34324 DEBUG oslo_messaging._drivers.amqpdriver [-] received message with unique_id: e11018ec4c9f43559d7366b48900e552 __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqp
driver.py:269
2018-08-07 13:52:37.291 34324 DEBUG nova.virt.libvirt.migration [req-7751982a-1f31-4dbd-a01e-7ec63e783489 6177334513c94044857220f0bfc78b22 87ab28106b594ec083a8a701d2741714 - - -] [instance: 6f3925d4-2d6c-4d64-86
44-a48bfe26c694] VM running on src, migration failed find_job_type /usr/lib/python2.7/site-packages/nova/virt/libvirt/migration.py:228
2018-08-07 13:52:37.292 34324 DEBUG nova.virt.libvirt.driver [req-7751982a-1f31-4dbd-a01e-7ec63e783489 6177334513c94044857220f0bfc78b22 87ab28106b594ec083a8a701d2741714 - - -] [instance: 6f3925d4-2d6c-4d64-8644-
a48bfe26c694] Fixed incorrect job type to be 4 _live_migration_monitor /usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:6428
2018-08-07 13:52:37.292 34324 ERROR nova.virt.libvirt.driver [req-7751982a-1f31-4dbd-a01e-7ec63e783489 6177334513c94044857220f0bfc78b22 87ab28106b594ec083a8a701d2741714 - - -] [instance: 6f3925d4-2d6c-4d64-8644-
a48bfe26c694] Migration operation has aborted


Version-Release number of selected component (if applicable):
-------------------------------------------------------------
openstack-nova-placement-api-15.0.8-5.el7ost.noarch
openstack-nova-api-15.0.8-5.el7ost.noarch
python-nova-15.0.8-5.el7ost.noarch
openstack-nova-compute-15.0.8-5.el7ost.noarch
openstack-nova-conductor-15.0.8-5.el7ost.noarch
puppet-nova-10.4.1-5.el7ost.noarch
openstack-nova-migration-15.0.8-5.el7ost.noarch
openstack-nova-scheduler-15.0.8-5.el7ost.noarch
openstack-nova-common-15.0.8-5.el7ost.noarch
openstack-nova-console-15.0.8-5.el7ost.noarch
openstack-nova-cert-15.0.8-5.el7ost.noarch
openstack-nova-novncproxy-15.0.8-5.el7ost.noarch
python-novaclient-7.1.2-1.el7ost.noarch

Steps to Reproduce:
-------------------
1. Upgrade UC to RHOS-12
2. Follow upgrade procedure for OC
3. Before upgrading compute migrate VM(s) running there to another compute
4. Follow upgrade procedure to upgrade compute
5. Try live-migrate instance back to upgraded compute

Additional info:
----------------
Virtual setup: 3controllers + 2computes + 3ceph
Comment 2 Daniel Berrangé 2018-08-07 14:59:29 UTC 
The key problem here this:

Unable to parse URI qemu+ssh://nova_migration@fd00:fd00:fd00:2000::1e/system?keyfile=/etc/nova/migration/identity

That is not a valid URI, because IPv6 addresses need to be wrapped in [] when used in a URI, to avoid ambiguity wrt port number.

The problem is nova/virt/libvirt/driver.py file _live_migration_uri method, which simply does a direct string substitution of the target machine. This needs to be enhanced to check whether the target machine is a raw IPv6 address and wrap it in []. The code in question looks like it has always been broken in respect of raw IPv6 addresses so bug probably needs cloning to multiple nova versions.
Comment 3 Ollie Walsh 2018-08-07 16:33:36 UTC 
I assume this isn't restricted to upgrades, all live-migration on OSP12 with IPv6 would be affected.
Comment 6 Lee Yarwood 2018-11-15 11:23:11 UTC 
Closing this out as WONTFIX due to the upcoming EOL of 12.