Red Hat Bugzilla – Bug 971394
RFE: The ability to specify source address when migrating VMs
Last modified: 2013-06-24 04:55:25 EDT
+++ This bug was initially created as a clone of Bug #969054 +++
Description of problem:
When migrating a VM and opting to use the new migration network, if the source host has multiple NICs and the destination host is on a different subnet, then the socket will be created with a random source address / NIC. The source host will see that the destination host is on a different subnet and forward towards the default gateway. If the host has multiple gateways, we wish to specify the source address to that of the migration network, so the correct NIC will be chosen.
If the hosts are on the same subnet there is no issue.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. Setup 2 hosts on different subnets
2. Configure the source host with 2 NICs
3. One NIC with ovirtmgmt, connected to 1 gateway
4. Second NIC with migration network, connected to a different gateway
5. Migrate the VM
The migration will occur over either the migration network, or the ovirtmgmt network. The network used is random.
The migration network should always be used.
--- Additional comment from Assaf Muller on 2013-06-02 03:15:04 EDT ---
We need the ability to specify the source address when setting up the migration socket. Currently QEMU and Libvirt don't support this feature.
--- Additional comment from Dan Kenigsberg on 2013-06-04 08:10:16 EDT ---
The cleanest implementation for this issue requires some work on upstream qemu and libvirt, as qemu does expose the ability to set the source address of migration traffic (from qmp-commands.hx):
Migrate to URI.
- "blk": block migration, full disk copy (json-bool, optional)
- "inc": incremental disk copy (json-bool, optional)
- "uri": Destination URI (json-string)
Another alternative would be to set a special destination routing before migration starts. I believe that this alternative would prove fragile, and that it should not be taken before we see a proven need for cross-subnet migration.
As discussed on IRC, I don't believe this should be done within libvirt or QEMU. If traffic between 2 specific subnets is supposed to use a specific NIC, then the kernel routing tables should be configured to direct traffic accordingly. This does not need to be done at time of migration, routing is something that should be configured when creating subnets and/or deploying hosts.
It also isn't clear if it is even possible todo what it requested via the sockets API. To set the source address you need to call 'bind' on the socket before 'connect'. This can be demonstrated using the 'nc' tool and its '-s' option.
I have a host with 2 nics on different subnets, both subnets are able to reach a third subnet 'gsslab.fab.redhat.com'. The default route favours eth1 on the 192.168.122.130.0/24 subnet:
# ip -4 addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
inet 127.0.0.1/8 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
inet 192.168.122.66/24 brd 192.168.122.255 scope global eth0
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
inet 192.168.130.68/24 brd 192.168.130.255 scope global eth1
# ip -4 route
192.168.130.0/24 dev eth1 proto kernel scope link src 192.168.130.68
192.168.122.0/24 dev eth0 proto kernel scope link src 192.168.122.66
169.254.0.0/16 dev eth0 scope link metric 1002
169.254.0.0/16 dev eth1 scope link metric 1003
default via 192.168.130.1 dev eth1
Connecting with no source address works
# nc avocado.gsslab.fab.redhat.com 22
Connecting with an explicit source address on the subnet corresponding to eth1 / 192.168.130.0/24 works:
# nc -s 192.168.130.68 avocado.gsslab.fab.redhat.com 22
Connecting with an explicit source address on the subnet which is *not* corresponding to the default route does not work:
# nc -s 192.168.122.66 avocado.gsslab.fab.redhat.com 22
...no connection established..
Setting source addresses on sockets mostly just seems to be useful in receiving broadcast/multicast traffic on UDP sockets. It can't override the kernel's routing table decisions for outbound traffic.
So AFAICT, there is no technical way to satisfy this request.
I'm closing this bug based on comments 1 and 3. The hosts' routing
tables is the place to configure which way packets between hosts