Bug 1613449 - [12][IPv6]Live migration failed: Unable to parse URI qemu+ssh://nova_migration@fd00:
Summary: [12][IPv6]Live migration failed: Unable to parse URI qemu+ssh://nova_migrati...
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova
Version: 12.0 (Pike)
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 12.0 (Pike)
Assignee: Lee Yarwood
QA Contact: OSP DFG:Compute
URL:
Whiteboard:
Depends On:
Blocks: 1614447 1614451
TreeView+ depends on / blocked
 
Reported: 2018-08-07 14:51 UTC by Yurii Prokulevych
Modified: 2023-03-21 18:58 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1614447 (view as bug list)
Environment:
Last Closed: 2018-11-15 11:23:11 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1786058 0 None None None 2018-08-08 15:35:32 UTC
OpenStack gerrit 589548 0 None None None 2018-08-08 11:55:45 UTC
OpenStack gerrit 603738 0 None None None 2018-09-19 12:45:06 UTC

Description Yurii Prokulevych 2018-08-07 14:51:34 UTC
Description of problem:
-----------------------
Attempt to migrate workload to upgraded compute failed:
...
2018-08-07 13:52:37.234 34324 ERROR nova.virt.libvirt.driver [req-7751982a-1f31-4dbd-a01e-7ec63e783489 6177334513c94044857220f0bfc78b22 87ab28106b594ec083a8a701d2741714 - - -] [instance: 6f3925d4-2d6c-4d64-8644-
a48bfe26c694] Live Migration failure: internal error: Unable to parse URI qemu+ssh://nova_migration@fd00:fd00:fd00:2000::1e/system?keyfile=/etc/nova/migration/identity
2018-08-07 13:52:37.234 34324 DEBUG nova.virt.libvirt.driver [req-7751982a-1f31-4dbd-a01e-7ec63e783489 6177334513c94044857220f0bfc78b22 87ab28106b594ec083a8a701d2741714 - - -] [instance: 6f3925d4-2d6c-4d64-8644-
a48bfe26c694] Migration operation thread notification thread_finished /usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:6612
2018-08-07 13:52:37.279 34324 DEBUG oslo_messaging._drivers.amqpdriver [-] received reply msg_id: 82b5c77aa5eb4733b8df322821e86e4f __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:
419
2018-08-07 13:52:37.289 34324 DEBUG oslo_messaging._drivers.amqpdriver [-] received message with unique_id: e11018ec4c9f43559d7366b48900e552 __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqp
driver.py:269
2018-08-07 13:52:37.291 34324 DEBUG nova.virt.libvirt.migration [req-7751982a-1f31-4dbd-a01e-7ec63e783489 6177334513c94044857220f0bfc78b22 87ab28106b594ec083a8a701d2741714 - - -] [instance: 6f3925d4-2d6c-4d64-86
44-a48bfe26c694] VM running on src, migration failed find_job_type /usr/lib/python2.7/site-packages/nova/virt/libvirt/migration.py:228
2018-08-07 13:52:37.292 34324 DEBUG nova.virt.libvirt.driver [req-7751982a-1f31-4dbd-a01e-7ec63e783489 6177334513c94044857220f0bfc78b22 87ab28106b594ec083a8a701d2741714 - - -] [instance: 6f3925d4-2d6c-4d64-8644-
a48bfe26c694] Fixed incorrect job type to be 4 _live_migration_monitor /usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:6428
2018-08-07 13:52:37.292 34324 ERROR nova.virt.libvirt.driver [req-7751982a-1f31-4dbd-a01e-7ec63e783489 6177334513c94044857220f0bfc78b22 87ab28106b594ec083a8a701d2741714 - - -] [instance: 6f3925d4-2d6c-4d64-8644-
a48bfe26c694] Migration operation has aborted


Version-Release number of selected component (if applicable):
-------------------------------------------------------------
openstack-nova-placement-api-15.0.8-5.el7ost.noarch
openstack-nova-api-15.0.8-5.el7ost.noarch
python-nova-15.0.8-5.el7ost.noarch
openstack-nova-compute-15.0.8-5.el7ost.noarch
openstack-nova-conductor-15.0.8-5.el7ost.noarch
puppet-nova-10.4.1-5.el7ost.noarch
openstack-nova-migration-15.0.8-5.el7ost.noarch
openstack-nova-scheduler-15.0.8-5.el7ost.noarch
openstack-nova-common-15.0.8-5.el7ost.noarch
openstack-nova-console-15.0.8-5.el7ost.noarch
openstack-nova-cert-15.0.8-5.el7ost.noarch
openstack-nova-novncproxy-15.0.8-5.el7ost.noarch
python-novaclient-7.1.2-1.el7ost.noarch

Steps to Reproduce:
-------------------
1. Upgrade UC to RHOS-12
2. Follow upgrade procedure for OC
3. Before upgrading compute migrate VM(s) running there to another compute
4. Follow upgrade procedure to upgrade compute
5. Try live-migrate instance back to upgraded compute

Additional info:
----------------
Virtual setup: 3controllers + 2computes + 3ceph

Comment 2 Daniel Berrangé 2018-08-07 14:59:29 UTC
The key problem here this:

Unable to parse URI qemu+ssh://nova_migration@fd00:fd00:fd00:2000::1e/system?keyfile=/etc/nova/migration/identity

That is not a valid URI, because IPv6 addresses need to be wrapped in [] when used in a URI, to avoid ambiguity wrt port number.

The problem is nova/virt/libvirt/driver.py file _live_migration_uri method, which simply does a direct string substitution of the target machine. This needs to be enhanced to check whether the target machine is a raw IPv6 address and wrap it in []. The code in question looks like it has always been broken in respect of raw IPv6 addresses so bug probably needs cloning to multiple nova versions.

Comment 3 Ollie Walsh 2018-08-07 16:33:36 UTC
I assume this isn't restricted to upgrades, all live-migration on OSP12 with IPv6 would be affected.

Comment 6 Lee Yarwood 2018-11-15 11:23:11 UTC
Closing this out as WONTFIX due to the upcoming EOL of 12.


Note You need to log in before you can comment on or make changes to this bug.