Bug 971394 - RFE: The ability to specify source address when migrating VMs
RFE: The ability to specify source address when migrating VMs
Status: CLOSED CANTFIX
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt (Show other bugs)
6.6
Unspecified Unspecified
low Severity low
: rc
: ---
Assigned To: Libvirt Maintainers
Virtualization Bugs
network
: FutureFeature
Depends On: 971395
Blocks: 969054
  Show dependency treegraph
 
Reported: 2013-06-06 08:18 EDT by Assaf Muller
Modified: 2013-06-24 04:55 EDT (History)
14 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: 969054
: 971395 (view as bug list)
Environment:
Last Closed: 2013-06-24 04:55:25 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Assaf Muller 2013-06-06 08:18:50 EDT
+++ This bug was initially created as a clone of Bug #969054 +++

Description of problem:
When migrating a VM and opting to use the new migration network, if the source host has multiple NICs and the destination host is on a different subnet, then the socket will be created with a random source address / NIC. The source host will see that the destination host is on a different subnet and forward towards the default gateway. If the host has multiple gateways, we wish to specify the source address to that of the migration network, so the correct NIC will be chosen.

If the hosts are on the same subnet there is no issue.

Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:
1. Setup 2 hosts on different subnets
2. Configure the source host with 2 NICs
3. One NIC with ovirtmgmt, connected to 1 gateway
4. Second NIC with migration network, connected to a different gateway
5. Migrate the VM

Actual results:
The migration will occur over either the migration network, or the ovirtmgmt network. The network used is random.

Expected results:
The migration network should always be used.

Additional info:

--- Additional comment from Assaf Muller on 2013-06-02 03:15:04 EDT ---

Proposed Solution:
We need the ability to specify the source address when setting up the migration socket. Currently QEMU and Libvirt don't support this feature.

--- Additional comment from Dan Kenigsberg on 2013-06-04 08:10:16 EDT ---

The cleanest implementation for this issue requires some work on upstream qemu and libvirt, as qemu does expose the ability to set the source address of migration traffic (from qmp-commands.hx):

 migrate
 -------

 Migrate to URI.

 Arguments:

 - "blk": block migration, full disk copy (json-bool, optional)
 - "inc": incremental disk copy (json-bool, optional)
 - "uri": Destination URI (json-string)


Another alternative would be to set a special destination routing before migration starts. I believe that this alternative would prove fragile, and that it should not be taken before we see a proven need for cross-subnet migration.
Comment 1 Daniel Berrange 2013-06-06 08:23:46 EDT
As discussed on IRC, I don't believe this should be done within libvirt or QEMU. If traffic between 2 specific subnets is supposed to use a specific NIC, then the kernel routing tables should be configured to direct traffic accordingly. This does not need to be done at time of migration, routing is something that should be configured when creating subnets and/or deploying hosts.
Comment 3 Daniel Berrange 2013-06-06 09:06:22 EDT
It also isn't clear if it is even possible todo what it requested via the sockets API. To set the source address you need to call 'bind' on the socket before 'connect'. This can be demonstrated using the 'nc' tool and its '-s' option. 

I have a host with 2 nics on different subnets, both subnets are able to reach a third subnet 'gsslab.fab.redhat.com'. The default route favours eth1 on the 192.168.122.130.0/24 subnet:

# ip -4 addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    inet 127.0.0.1/8 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    inet 192.168.122.66/24 brd 192.168.122.255 scope global eth0
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    inet 192.168.130.68/24 brd 192.168.130.255 scope global eth1
# ip -4 route
192.168.130.0/24 dev eth1  proto kernel  scope link  src 192.168.130.68 
192.168.122.0/24 dev eth0  proto kernel  scope link  src 192.168.122.66 
169.254.0.0/16 dev eth0  scope link  metric 1002 
169.254.0.0/16 dev eth1  scope link  metric 1003 
default via 192.168.130.1 dev eth1 

Connecting with no source address works

# nc avocado.gsslab.fab.redhat.com 22
SSH-2.0-OpenSSH_6.1

Connecting with an explicit source address on the subnet corresponding to eth1 / 192.168.130.0/24 works:

# nc -s 192.168.130.68 avocado.gsslab.fab.redhat.com 22
SSH-2.0-OpenSSH_6.1

Connecting with an explicit source address on the subnet which is *not* corresponding to the default route does not work:

# nc -s 192.168.122.66 avocado.gsslab.fab.redhat.com 22
...no connection established..


Setting source addresses on sockets mostly just seems to be useful in receiving broadcast/multicast traffic on UDP sockets. It can't override the kernel's routing table decisions for outbound traffic.

So AFAICT, there is no technical way to satisfy this request.
Comment 4 Jiri Denemark 2013-06-24 04:55:25 EDT
I'm closing this bug based on comments 1 and 3. The hosts' routing
tables is the place to configure which way packets between hosts
should go.

Note You need to log in before you can comment on or make changes to this bug.