Bug 2025303

Hi Shoval,

Does it only support on RHEL8? Would you help clone it to RHEL9 if we also support it there?

Thanks,
Qianqian

Hi
It is not just for RHEL8, it is a general RFE for the guest agent from my understanding
just wasn't sure what to fill in the product field,

I am not part of RHEL product, but part of CNV (kubevirt).

see please here the motivation in case it is interesting,
https://bugzilla.redhat.com/show_bug.cgi?id=2013455


Thanks

Moving this to RHEL9 for action. Once a resolution is closer we can make the decision to clone to RHEL8. No sense in having 2 bugs for now.

Assigned to developer w/ Medium priority and added CNV as dependent product.

Is the feature required for guests other than Linux?

It seems we will need to query it from a netlink socket. I can't find it from sysfs (nor inet ioctl).

Would optional interface "permanent-address" : "00:00:00:00:00:00" be good enough? (if it's not set either it's the same as "hardware-address", or unsupported)

It would be better if it is supported as well for other guests that support it and wide used, thanks

In case the "permanent-address" is unsupported it is fine to have "00:00:00:00:00:00".

In case it is the same as the "hardware-address", what is the motivation to not send it, less traffic ?
as long as the report won't be ambiguous in some scenarios it is fine, i just wonder if we
can find a scenario that is ambiguous.

Thanks

Alternatively, libvirt could emit an event when MAC address changes (QEMU is doing that already and libvirt does process it for internal needs). BTW: if a guest does change MAC address then corresponding <interface/> snippet in the domain XML is updated. CNV could listen to this event and update its internal state accordingly.

Hi Michal

The event is interesting, any reference please so i will learn more about it / how to listen to it ?

> BTW: if a guest does change MAC address then corresponding <interface/> snippet in the domain XML is updated. CNV could listen to this event and update its internal state accordingly.

I tried to change the MAC in the guest using
sudo ip link set dev eth0 address 02:00:b5:b5:b5:c9
ifconfig shows the new MAC,
ethtool -P shows the original MAC
but the virsh dumpxl still shows the original MAC as well (and not the new one)

We are using QEMU Guest Agent 4.2.1 on fedora.

Thanks

(In reply to oshoval from comment #8)
> Hi Michal
> 
> The event is interesting, any reference please so i will learn more about it
> / how to listen to it ?
> 
> > BTW: if a guest does change MAC address then corresponding <interface/> snippet in the domain XML is updated. CNV could listen to this event and update its internal state accordingly.
> 
> I tried to change the MAC in the guest using
> sudo ip link set dev eth0 address 02:00:b5:b5:b5:c9
> ifconfig shows the new MAC,
> ethtool -P shows the original MAC
> but the virsh dumpxl still shows the original MAC as well (and not the new
> one)
> 

You need to enable libvirt to trust guest first. This is done by setting trustGuestRxFilters attribute to yes, like this:

  <interface type='direct' trustGuestRxFilters='yes'>
    ...
  </interface>

https://libvirt.org/formatdomain.html#network-interfaces

but be aware that this works only for some models (those which can communicate the change outside, typically virtio).

And currently, there is no event for you to listen to, because it's not implemented yet.

Thanks

we are using virtio-non-transitional, and from my understanding we should not use "virtio"
according https://libvirt.org/formatdomain.html#virtio-transitional-devices
i just tried it there, and it didn't change it (as you said, as its non "virtio")


    <interface type='ethernet' trustGuestRxFilters='yes'>
      <mac address='52:54:00:97:e1:51'/>
      <target dev='tap0' managed='no'/>
      <model type='virtio-non-transitional'/>
      <mtu size='1480'/>
      <alias name='ua-default'/>
      <rom enabled='no'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>

(In reply to oshoval from comment #10)
> Thanks
> 
> we are using virtio-non-transitional, and from my understanding we should
> not use "virtio"
> according https://libvirt.org/formatdomain.html#virtio-transitional-devices
> i just tried it there, and it didn't change it (as you said, as its non
> "virtio")
> 
> 
>     <interface type='ethernet' trustGuestRxFilters='yes'>
>       <mac address='52:54:00:97:e1:51'/>
>       <target dev='tap0' managed='no'/>
>       <model type='virtio-non-transitional'/>

This is virtio! But also it is of type='ethernet' which means, an external entity is in charge of the interface.

>       <mtu size='1480'/>
>       <alias name='ua-default'/>
>       <rom enabled='no'/>
>       <address type='pci' domain='0x0000' bus='0x01' slot='0x00'
> function='0x0'/>
>     </interface>

Correct, libvirt doesn't reflect MAC address change even though QEMU did emit NIC_RX_FILTER_CHANGED event (the event means a MAC address inside the guest has changed). @laine do you remember what our reasoning was to reflect the change in domain XML only for certain <interface/> types? Since QEMU delives the event I believe we can update the XML regardless of the type of <interface/> and execute changes needed in the host only for a subset of types (just like we are doing now)?

@mprivozn , I am still unclear if changing the non-permanent address
is suppose to be reflected back to libvirt domxml or not.
I have not seen this happening, and per oshoval results, it is not seen.

If this would have worked with virtio, then I think we do not even
need the event getting forwarded by libvirt. kubevirt is already
refreshing the domain information periodically if it is idle for too
long.

But I must admit this is too odd and does not compute well logically for me.
In the domain xml, we set the desired MAC address for an interface, which
is suppose to be this permanent address, emulating a real HW NIC.
If the admin in the guest is changing the MAC, I would expect only
the soft address to be changed, and therefore, not to reflect back and
change the domxml details. But maybe I missed something here.

If the intention was that the user of libvirt needs to monitor an event
to catch such changes, and this not being persisted anywhere as a state,
then I find this unreliable. The client, kubevirt, may loose this information,
for various reasons (e.g. the app crashed). We depend on libvirt/qemu to
persist and monitor the state of the guest.

I would prefer the `"permanent-address"` option, it seems direct and simple
from a user perspective.

(In reply to Edward Haas from comment #12)
> @mprivozn , I am still unclear if changing the non-permanent address
> is suppose to be reflected back to libvirt domxml or not.
> I have not seen this happening, and per oshoval results, it is not seen.
> 
> If this would have worked with virtio, then I think we do not even
> need the event getting forwarded by libvirt. kubevirt is already
> refreshing the domain information periodically if it is idle for too
> long.
> 
> But I must admit this is too odd and does not compute well logically for me.
> In the domain xml, we set the desired MAC address for an interface, which
> is suppose to be this permanent address, emulating a real HW NIC.
> If the admin in the guest is changing the MAC, I would expect only
> the soft address to be changed, and therefore, not to reflect back and
> change the domxml details. But maybe I missed something here.
> 
> If the intention was that the user of libvirt needs to monitor an event
> to catch such changes, and this not being persisted anywhere as a state,
> then I find this unreliable. The client, kubevirt, may loose this
> information,
> for various reasons (e.g. the app crashed). We depend on libvirt/qemu to
> persist and monitor the state of the guest.
> 
> I would prefer the `"permanent-address"` option, it seems direct and simple
> from a user perspective.
 
The MAC address of a guest interface can be used for so much. If host blindly trusts guest then a malicious guest can set a MAC address of another guest and thus redirect that traffic onto itself. Then, libvirt has a submodule which allows users to automatically set up firewall rules. Again, MAC address can be used in those rules so a malicious guest can cut off another guest if it sets just right MAC address. Trusting guest blindly is very dangerous and should never ever be the default.

Having said that, libvirt introduced a knob which allows you to mark what interfaces within given guest you trust and thus reflect MAC address change in the domain XML.

I'm not against extending guest agent, but it will require new libvirt API as the current one does not offer a way to expose new fields.

Sorry, I should have jumped in with my memories of NIC_RX_FILTER_CHANGED back a couple months ago, but truthfully *had* no memories (that all happened a very long time ago!), so I resolved to look into it before typing anything, and then promptly forgot.

I've finally looked back at the code to remind myself, and what I've now recalled is that libvirt only added in processing of the NIC_RC_FILTER_CHANGED event for one specific purpose - in order to update the MAC address of the host ethernet interface when a macvtap passthrough mode interface had its MAC address changed in the guest (this is necessary because proper operation of macvtap passthrough mode requires that the hardware interface on the host have exactly the same MAC address and RX_FILTER as the emulated interface in the guest). There was never any intention of responding to this event for interface types other than macvtap passthrough mode, nor of mirroring that new setting back into the domain XML, and I'm pretty sure that doesn't ever happen anywhere (the code is just so convoluted that someone might assume that had happened somewhere :-))

BTW, with one exception [*], libvirt processing of NIC_RX_FILTER_CHANGED doesn't take any action for other types of interfaces (e.g. tap device-based interfaces, or even macvtap bridged mode) because there is no direct correlation between host and guest MAC address needed for proper operation of those types of interface. As a matter of fact, in the case of a tap device, the two ends of the tap must *not* match!

([*] the exception to the "no action for interface types other than macvtap/passthrough" is that I found Michal at some point added support for updating the bandwidth shaping rules for tap devices on a virtual network when the MAC address in the guest changes)

As Michal pointed out, since MAC address is used as an index for finding an interface in so many places (before user-named device aliases were available in libvirt, it was pretty much the *only* way to find an interface) we can't change the MAC address of an interface in the XML at runtime. Instead it should be reported in some other way, perhaps by a more extensible new generation of the virDomainInterfaceAddresses() API (which unfortunately, as Michal says, was designed to only report a name, a single hardware address (MAC), and a list of IP addresses with no extensibility in mind).

Along with that, libvirt could also add public reporting of the NIC_RX_FILTER_CHANGED event to allow libvirt consumers to deal with keeping track of current MAC address themselves.

Reassigning since you've been picking up qga tasks from Marc-Andre.

May be worth the cycles to just figure out what is necessary given the above and the reality that this bug hasn't had cycles for 9+ months.

I'm still wondering whether there's something that libvirt can do better here. I mean, QEMU already emits an even on MAC change, and for some <interface/> types libvirt acts upon, BUT:

1) it never forwards the event to users,
2) it does not reflect the MAC address change in the domain XML.

Now, I think 2) is equally important, because that's what libvirt does for other device types (i.e. reflect the current state of device, e.g. connected/disconnected state for <channel/>-s), the new MAC address is reflected in firewall/QoS rules (because packets from the guest have this new MAC address). The configured MAC can be obtained via 'virsh dumpxml --config', just like config state for other devices. Although, for some reason, the host side of TAP device still shows the old (config) MAC address regardless of MAC changes in the guest.

Laine, what do you think?

After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.

I've written some RFC patches and posted them on the list to get the discussion going:

https://listman.redhat.com/archives/libvir-list/2023-June/240519.html

Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.

This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information.

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days