Bug 1490482

This is assigned to libvirt, but there isn't enough information in the description to understand what it is that libvirt is alleged to be doing or not doing (or what it *should* be doing), nor any particulars about the configuration.

The problem statement talks about "Linux bridges", but also about VLANs. Since there is no support within libvirt for transparently setting VLANs of guest interfaces attached to a Linux Host Bridge, I assume that this setup is using Open vSwitch, but there's no mention of that here.

Also, it says that "VM2 vNIC is moved to a different network". I'm *guessing* that this means that the guest interface remains connected to the same bridge, but that its VLAN changes - is that correct? Where are you changing the VLAN tag? In the guest (VM) itself? Of is your bridge an OVS bridge, and you're changing the VLAN tag transparent to the guest? If so, how?

Can you provide configuration (RHV config would be useful for a RHV networking person, but for libvirt we would really appreciate libvirt XML) to describe the setup, and the exact commands you use (and on which machine) to change the VLAN tag?

Created attachment 1326215 [details]
qemu_networks

From the ACI, we are creating networks. These networks are pushed to the RHEV Manager. Each network will have a vlan. In the RHEV Manager, When the vNIC of the VM is attached to the network, we expect RARP to be sent out by the linux bridge created for this network(VLAN).

vNIC vlan change is done from the RHEV Manager by selecting the VM's vNIC and assigning a particular network. The vlan change is done in the VM itself from the RHEV Manager.

libvirt xmls attached.

From the ACI, we are creating networks. These networks are pushed to the RHEV Manager. Each network will have a vlan. In the RHEV Manager, When the vNIC of the VM is attached to the network, we expect RARP to be sent out by the linux bridge created for this network(VLAN).

vNIC vlan change is done from the RHEV Manager by selecting the VM's vNIC and assigning a particular network. The vlan change is done in the VM itself from the RHEV Manager.

libvirt xmls attached.

qemu networks is a tgz archive

qemu networks is a tgz archive

The libvirt network configs show that these are standard Linux host bridges, so libvirt isn't involved in any setting of VLAN tags. Based on the description, the change in networks is being initiated by RHV, which I think is using vdsm to change the VLAN tag (it sounds like maybe it is sending a command to the guest OS via a guest agent? Not sure, since I'm unfamiliar with the details of RHV and vdsm). I don't really think there is anything to be done about this problem at the layer of libvirt.

Dan K.: I'm not sure what is the correct place to assign this BZ, since the RHV and vdsm versions aren't known, and also because the Product vs. Component matrix in bugzilla seems mixed up (it's possible to assign to a *Product* called vdsm, and also to a *Component* called vdsm that is under the "Red Hat Enterprise Virtualization Manager" Product. I don't want to accidentally assign it to the wrong place and have it lost, so can you please put it in the right place?

Thanks!

Per comment 9 I cloned the bug to vdsm and thus closing this one.

Laine, I do not consider this a problem; I would phrase this as a request for extension. Let me rephrase the problem, and politely re-open this bug for continued discussion.

Libvirt connects a tap device to a bridge, per request of RHV. Then at some point in time, RHV asks libvirt to change to another bridge, which connects the VM to a fresh LAN. The new LAN, regardless of its vlan id, is unaware of the new location of the VM.

I am suggesting that libvirt triggers a RARP packet notifying the new LAN of the new location of the VM. This is similar to what qemu does when live migration makes a VM pop in a new network location.

Now that Dan has re-described the scenario in terms of libvirt, I understand the situation; since libvirt *does* have control at the point the change is made (and the change is to connect the tap to a new bridge device, *not* to change the vlan tag) it's probably reasonable for libvirt to send the RARP (or whatever) packet.

(The original sounded like the tap device was remaining connected to the same bridge, but the VLAN tag was being changed - in that case there would have been no libvirt-visible event that could be used to trigger anything).


(BTW, this BZ was marked CLOSED/NOTABUG because based on the original description we thought only vdsm would have control during this event, but it is impossible to reassign a libvirt (or qemu or kernel or any other RHEL) bug report to anything under the RHEV product, due to the mandatory "oVirt Team" field - it must be set before the modified bug can be saved, but the field isn't made visible until the bug is saved with the new product name, so you can't set it. As a result, the only way to "reassign" a bug from any RHEL component to a RHEV component is to clone the bug and close the original; Jarda had created the new Bug 1492119 to "reassign". See Bug 1323778 for details of the problem that necessitates such shenanigans.

There are a few complications with this:

* The packet needs to be transmitted by the tap device so that it is received on the bridge port, but libvirt can't really do that, since qemu has control of the other end of the tap, where packets would be injected.

* Depending on the bridge configuration, a packet sent immediately after the tap device is attached would be ignored by the bridge due to STP delay.

Solution: Since qemu already has code to transmit the RARP packet that we want, and sends it 5 times with delay between (thus increasing the chances that one will get through), maybe what we need is for qemu to have a new "advertise MAC" command, or something like that (maybe just "send RARP"). As soon as libvirt connects to the new bridge, it notifies qemu that the connection of the tap device has changed by sending a "send RARP" command, and qemu responds by calling qemu_announce_self() (which sets timer callbacks to send the 5 RARP packets). This gets yet another package involved, but I think it's the cleanest way to do it.

Michael, does this sound reasonable to you?

(In reply to Laine Stump from comment #14)
> There are a few complications with this:
> 
> * The packet needs to be transmitted by the tap device so that it is
> received on the bridge port, but libvirt can't really do that, since qemu
> has control of the other end of the tap, where packets would be injected.
> 
> * Depending on the bridge configuration, a packet sent immediately after the
> tap device is attached would be ignored by the bridge due to STP delay.
> 
> Solution: Since qemu already has code to transmit the RARP packet that we
> want, and sends it 5 times with delay between (thus increasing the chances
> that one will get through), maybe what we need is for qemu to have a new
> "advertise MAC" command, or something like that (maybe just "send RARP"). As
> soon as libvirt connects to the new bridge, it notifies qemu that the
> connection of the tap device has changed by sending a "send RARP" command,
> and qemu responds by calling qemu_announce_self() (which sets timer
> callbacks to send the 5 RARP packets). This gets yet another package
> involved, but I think it's the cleanest way to do it.
> 
> Michael, does this sound reasonable to you?

That command exists in a set that Vlad pulled together, with the command originally added by Germano:

https://lists.gnu.org/archive/html/qemu-devel/2017-05/threads.html#05594

it needs a bit of a tidy up and repost, but I think that has what you need.

As far as I can find, the patchset Dave pointed out in comment 15 was never pushed to upstream qemu. Vlad - are there any plans to do that?

I just realized that I had been equating this BZ with Bug 1094398 in my mind, since they're both asking for almost the exact same thing, and require the same capability from qemu. While this BZ asks for libvirt to trigger an announcement whenever the tap device is connected to a different bridge (which would be something done directly via libvirt) while Bug 1094398 is asking for the announcement when something in the underlying infrastructure changes (in their example, a bond failover).

In any case,  both of them require the qemu_self_announce() QMP command.

The qemu announce code changes just got merged upstream; if you need any help with them give me a poke.

*** This bug has been marked as a duplicate of bug 1094398 ***