RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2144738 - vm start successfully even there is a pre-created tap device with the same name (type="network")
Summary: vm start successfully even there is a pre-created tap device with the same na...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: libvirt
Version: 9.2
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Michal Privoznik
QA Contact: yalzhang@redhat.com
URL:
Whiteboard:
Depends On:
Blocks: 2166235
TreeView+ depends on / blocked
 
Reported: 2022-11-22 07:33 UTC by yalzhang@redhat.com
Modified: 2023-05-09 08:10 UTC (History)
10 users (show)

Fixed In Version: libvirt-9.0.0-2.el9
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2166235 (view as bug list)
Environment:
Last Closed: 2023-05-09 07:27:43 UTC
Type: Bug
Target Upstream Version: 9.0.0
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker LIBVIRTAT-14002 0 None None None 2023-02-27 03:44:43 UTC
Red Hat Issue Tracker RHELPLAN-140124 0 None None None 2022-11-22 07:47:26 UTC
Red Hat Product Errata RHBA-2023:2171 0 None None None 2023-05-09 07:29:09 UTC

Description yalzhang@redhat.com 2022-11-22 07:33:00 UTC
Description of problem:
vm start successfully even there is a pre-created tap device with the same name

Version-Release number of selected component (if applicable):
libvirt-8.9.0-2.el9.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Create a tap device named "mytap0"
# ip tuntap add mode tap name mytap0
# ip tuntap
mytap0: tap persist
# ip l show mytap0
15: mytap0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether fa:90:50:ce:ae:fc brd ff:ff:ff:ff:ff:ff

2. Try to start vm with target dev name as "mytap0"
# virsh dumpxml rhel --xpath //interface
<interface type="network">
  <mac address="52:54:00:06:7d:20"/>
  <source network="default"/>
  <target dev="mytap0"/>
  <model type="virtio"/>
  <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
</interface>

# virsh start rhel
Domain 'rhel' started

# ip tuntap
mytap0: tap vnet_hdr persist

# ip l show mytap0
15: mytap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master virbr0 state UP mode DEFAULT group default qlen 1000
    link/ether fe:54:00:06:7d:20 brd ff:ff:ff:ff:ff:ff

3. destroy the vm and check the tap device
# virsh destroy rhel
Domain 'rhel' destroyed

# ip l show mytap0
15: mytap0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master virbr0 state DOWN mode DEFAULT group default qlen 1000
    link/ether fe:54:00:06:7d:20 brd ff:ff:ff:ff:ff:ff
# ip tuntap
mytap0: tap vnet_hdr persist

4. Try to start vm again, failed
# virsh start rhel 
error: Failed to start domain 'rhel'
error: Unable to add bridge virbr0 port mytap0: Device or resource busy

Actual results:
In step 2, vm start successfully even there is a pre-created tap device with the same name; 
in step 3, when the vm destroyed, the pre-created tap device setting will be override

Expected results:
In step 2, vm should fail to start

Additional info:

Comment 1 yalzhang@redhat.com 2022-11-22 08:13:53 UTC
Another scenario with direct type interface, the vm failed to start, but the tap device disappear.

Test on libvirt-8.9.0-2.el9.x86_64:
1. add a tap device named "test":
# ip tuntap add test mode tap
# ip tuntap
test: tap persist
# ip l show test
18: test: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 3e:d7:8f:2a:a7:f7 brd ff:ff:ff:ff:ff:ff

2. start a vm with target dev named "test", vm start fail as expected:
# virsh dumpxml rhel --xpath //interface 
<interface type="direct">
  <mac address="52:54:00:3c:e8:b3"/>
  <source dev="eno1" mode="bridge"/>
  <target dev="test"/>
  <model type="virtio"/>
  <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
</interface>

# virsh start rhel 
error: Failed to start domain 'rhel'
error: error creating macvtap interface test@eno1 (52:54:00:3c:e8:b3): File exists

3. check the pre-created tap device, it does not exist, which is not expected:
# ip l  show test 
Device "test" does not exist.
# ip tuntap
# (no outputs)

Comment 2 yalzhang@redhat.com 2022-11-22 08:19:09 UTC
one more scenario:
1. create macvtap device named "test"
# ip l add l eno1 name test type macvtap mode bridge
# ip l  show test 
20: test@eno1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 500
    link/ether 4a:39:54:4c:f6:96 brd ff:ff:ff:ff:ff:ff

2. start vm, fail as expected
# virsh dumpxml rhel --xpath //interface 
<interface type="direct">
  <mac address="52:54:00:3c:e8:b3"/>
  <source dev="eno1" mode="bridge"/>
  <target dev="test"/>
  <model type="virtio"/>
  <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
</interface>
# virsh start rhel 
error: Failed to start domain 'rhel'
error: error creating macvtap interface test@eno1 (52:54:00:3c:e8:b3): File exists

3. check the macvtap device, it doest not exist, which is not expected
# ip l show test 
Device "test" does not exist.

Comment 3 Michal Privoznik 2022-12-06 08:42:20 UTC
The problem here is that when shutting down a guest, the qemuProcessStop() is called and it should remove the TAP devices:


void qemuProcessStop(virQEMUDriver *driver,
                     virDomainObj *vm,
                     virDomainShutoffReason reason,
                     virDomainAsyncJob asyncJob,
                     unsigned int flags)
{
    ...

    for (i = 0; i < def->nnets; i++) {
        virDomainNetDef *net = def->nets[i];
        vport = virDomainNetGetActualVirtPortProfile(net);
        switch (virDomainNetGetActualType(net)) {
        case VIR_DOMAIN_NET_TYPE_DIRECT:
            ignore_value(virNetDevMacVLanDeleteWithVPortProfile(
                             net->ifname, &net->mac,
                             virDomainNetGetActualDirectDev(net),
                             virDomainNetGetActualDirectMode(net),
                             virDomainNetGetActualVirtPortProfile(net),
                             cfg->stateDir));
            break;
        case VIR_DOMAIN_NET_TYPE_ETHERNET:
            if (net->managed_tap != VIR_TRISTATE_BOOL_NO && net->ifname) {
                ignore_value(virNetDevTapDelete(net->ifname, net->backend.tap));
                VIR_FREE(net->ifname);
            }
            break;
        case VIR_DOMAIN_NET_TYPE_BRIDGE:
        case VIR_DOMAIN_NET_TYPE_NETWORK:
#ifdef VIR_NETDEV_TAP_REQUIRE_MANUAL_CLEANUP
            if (!(vport && vport->virtPortType == VIR_NETDEV_VPORT_PROFILE_OPENVSWITCH))
                ignore_value(virNetDevTapDelete(net->ifname, net->backend.tap));
#endif
            break;
        case VIR_DOMAIN_NET_TYPE_USER:
        case VIR_DOMAIN_NET_TYPE_VHOSTUSER:
        case VIR_DOMAIN_NET_TYPE_SERVER:
        case VIR_DOMAIN_NET_TYPE_CLIENT:
        case VIR_DOMAIN_NET_TYPE_MCAST:
        case VIR_DOMAIN_NET_TYPE_INTERNAL:
        case VIR_DOMAIN_NET_TYPE_HOSTDEV:
        case VIR_DOMAIN_NET_TYPE_UDP:
        case VIR_DOMAIN_NET_TYPE_VDPA:
        case VIR_DOMAIN_NET_TYPE_NULL:
        case VIR_DOMAIN_NET_TYPE_VDS:
        case VIR_DOMAIN_NET_TYPE_LAST:
            /* No special cleanup procedure for these types. */
            break;
        }

But here, the VIR_NETDEV_TAP_REQUIRE_MANUAL_CLEANUP is defined only for FreeBSD. The reason this works for other TAPs is simple: a TAP device is created by opening a /dev/net/tun file and the TAP device lifetime is limited to that FD. Therefore, libvirt opens the file, passes corresponding FD onto QEMU and when it dies, the FD is closed and the TAP device released. But in case of pre-existing TAP devices the refcounter starts at 1 (by the time libvirt gets to it), and therefore, when QEMU dies, the refcounter is still at its original non-zero value (1) and thus the device is not released. Worse, it's kept plugged into the corresponding network bridge and that is causing problems when the guest is started for the second time.

Having said all of this, there are two (mutually exclusive) way to proceed:

1) remove the TAP device even on Linux (basically, remove the #ifdef check), or
2) explicitly unplug the TAP device from the network bridge

I'm not sure which way to go. Maybe I lean slightly towards 2) so that the device can be re-used.
Oh, there's a third option: since this is (implicitly) managed='yes' (because currently that's supported only for type='ethernet', meaning Libvirt's responsible for managing the TAP device, we could simply remove any pre-existing TAP device, just before creating the new one. Which brings me to another idea, introduce managed='yes|no' to the rest of the interface types. It would default to 'yes' and then removing the pre-existing TAP device would be warranted.

Laine, what are your thoughts on this?

Comment 4 Laine Stump 2022-12-06 15:31:24 UTC
We need to take a look to see what historically happened in the case of a type='bridge|network' when a tap device name was specified and that tap device already existed. I had always thought/assumed that it should fail, otherwise there would be a potential for a new guest to screw up the networking of an existing guest. However, when I just now looked at all the IFF_ flags available for TUNSETIFF, I see that there is none that looks like it would cause an error when attempting to use an existing tap.

The only time I can think of that a pre-existing tap device *shouldn't* be causing an error is in the case of type='ethernet', and in that case whatever setup the tap device has should be left intact at the end as well. (also in this case it is up to the user/management software to assure that 2 guests don't attempt to use the same tap device simultaneously)

We should never explicitly delete a tap device in the case of Linux (because it would fail most of the time due to the device already being deleted if it had been created by libvirt, which should *always* be the case for anything other than type='ethernet', and in the case of type='ethernet' when the device already existed, we want it to continue to exist after the guest is finished with it), so I don't think  suggestion (1) is appropriate. And as for suggestion (2), the only time libvirt should be using a pre-existing tap device is for type='ethernet', and in that case we definitely don't want to be disconnecting the tap from any bridge device - we shouldn't undo any setup that was done outside libvirt by someone else. I also think that pre-emptively deleting any existing tap device (option (3)) would most probably lead to unnecessary extra steps in 99.99% of cases, and to obscuring a bad configuration (and breaking the networking of a different guest) in half of the other cases (and the other half of the 0.01% is doing something way outside the boundaries of what we recommend/support).

I think a more proper thing would be to guard against using an existing tap device except in the case of type='ethernet' by logging an error and refusing to start. However, I'm concerned that changing the behavior to error out in this obviously incorrect usage might alter behavior in a way the breaks some arcane use case (which I haven't thought of) in someone's existing setup.

(as for managed='no' - the entire purpose of managed='no' is to assure that absolutely nothing is done to the tap device other than opening it and sending/receiving packets; in the case of type='bridge|network' there is necessary setup that needs to be done, thus nullifying the concept).


In the end, if you are only creating tap devices within libvirt, then they will always be properly deleted when a guest exits; and if you're using existing tap devices with type='ethernet', then it's on you (ie. the user/management app) to assure there is no duplicate usage of a tap between multiple guests. So the only time the scenario in this BZ description happens is when somebody is messing around doing things they shouldn't be doing (i.e. I don't think this would happen naturally during correct usage; the only thing close to this I can envision seeing is that someone would erroneously specify the same non-standard tap device name in two separate guests, and in that case you would want an error message when the 2nd guest started, not to re-use the tap device (thus messing up the 1st guest), but again I'm concerned that changing this long-standing behavior could break someone's existing (although unusual and unexpected) working setup).

(I had seen this BZ when it was opened and planned to respond, but wanted to remind myself what was the old behavior before saying anything, and then as usual I forgot to follow up. I'm still not sure I've remained coherent, but I tried :-))

Comment 5 Michal Privoznik 2022-12-08 09:37:17 UTC
(In reply to Laine Stump from comment #4)
>

Well, apparently this is not so common use case, because this bug was discovered only now. And if somebody wants to use their pre-created TAP, then type='ethernet' is the way to go. Anything else might change whatever was set on the TAP (e.g. type='network' might change QoS, MAC address, etc.).

Comment 6 Michal Privoznik 2022-12-08 15:22:30 UTC
Patch posted upstream:

https://listman.redhat.com/archives/libvir-list/2022-December/236197.html

Comment 8 Michal Privoznik 2022-12-09 07:52:55 UTC
Merged upstream as:

a2ae3d299c virnetdevtap.c: Disallow pre-existing TAP devices
6cbbc71e0d virnetdev: Make virNetDevGenerateName() return 1 if no name was generated

v8.10.0-87-ga2ae3d299c

Comment 9 yalzhang@redhat.com 2023-01-11 06:29:26 UTC
Test on upstream v9.0.0-rc1-5-ga5738ab74c with the scenario in comment 0, vm start fail and the tap device unchanged.
The result is as expected.

1. create a tap device named 'mytap0':
# ip tuntap add mode tap name mytap0
# ip tuntap
mytap0: tap persist
# ip l show mytap0
5: mytap0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 7e:ce:1e:d9:bf:38 brd ff:ff:ff:ff:ff:ff

2. try to start a vm with target dev as mytap0:
# virsh dumpxml rhel  --xpath //interface 
<interface type="network">
  <mac address="52:54:00:c7:3f:8a"/>
  <source network="default"/>
  <target dev="mytap0"/>
  <model type="e1000e"/>
  <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
</interface>

# virsh start rhel 
error: Failed to start domain 'rhel'
error: Requested operation is not valid: The mytap0 interface already exists

# ip l show mytap0
5: mytap0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 7e:ce:1e:d9:bf:38 brd ff:ff:ff:ff:ff:ff
#  ip tuntap
mytap0: tap persist

Comment 10 yalzhang@redhat.com 2023-01-11 06:37:41 UTC
Hi Michal, 

I have tested on upstream with the patch, for the scenario in comment 1 and comment 2, vm start fail as expected, but it delete the pre-created tap/macvtap device.
Please help to confirm the test result, Thank you! 

1. create a tap device(or a macvtap device), and start a vm with direct type interface with the same target dev name. The vm failed to start, which is expected:
# ip l show mytap0
6: mytap0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 7e:ce:1e:d9:bf:38 brd ff:ff:ff:ff:ff:ff

# virsh dumpxml rhel --xpath //interface 
<interface type="direct">
  <mac address="52:54:00:7a:51:7d"/>
  <source dev="eth0" mode="vepa"/>
  <model type="virtio"/>
  <address type="pci" domain="0x0000" bus="0x05" slot="0x00" function="0x0"/>
</interface>

# virsh start rhel 
error: Failed to start domain 'rhel'
error: error creating macvtap interface mytap0@eth0 (52:54:00:7a:51:7d): File exists

2. the start fail as expected, but the pre-created tap device is deleted, which is not expected:
# ip l show mytap0
Device "mytap0" does not exist.

Comment 11 Michal Privoznik 2023-01-16 10:05:48 UTC
(In reply to yalzhang from comment #10)
>

I'm unable to reproduce. Is there a missing step perhaps? Can you please attach debug logs?

Comment 12 Michal Privoznik 2023-01-23 11:06:12 UTC
My patches from comment 8 were not 100% correct. We need this follow up patches too:

https://listman.redhat.com/archives/libvir-list/2023-January/237282.html

specifically this one (which wasn't threaded correctly):

https://listman.redhat.com/archives/libvir-list/2023-January/237281.html

Comment 14 yalzhang@redhat.com 2023-01-28 03:10:28 UTC
Sorry, I missed '<target dev="mytap0"/>' in comment 10.

Test on libvirt-9.0.0-2.el9.x86_64, for network interface, the result is as expected; 
but for direct type interface, the pre-created tap device will be deleted, please check the scenario 2 below. Thank you.

Scenario 1: network interface
1. create tap device with name "mytap0":
# ip tuntap add mode tap name mytap0
# ip tuntap
mytap0: tap persist

# ip l show mytap0
10: mytap0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 1e:7b:07:fe:6d:72 brd ff:ff:ff:ff:ff:ff

2. Try to start vm with network interface and target dev name as "mytap0":
# virsh dumpxml rhel --xpath //interface 
<interface type="network">
  <mac address="52:54:00:99:af:16"/>
  <source network="default"/>
  <target dev="mytap0"/>
  <model type="virtio"/>
  <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
</interface>

# virsh start rhel 
error: Failed to start domain 'rhel'
error: Requested operation is not valid: The mytap0 interface already exists

# ip l show mytap0
10: mytap0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 1e:7b:07:fe:6d:72 brd ff:ff:ff:ff:ff:ff

# ip tuntap
mytap0: tap persist

Try to start vm several times, the result is the same, which is as expected.


Scenario 2: direct type interface

1. create a macvtap device:
# ip l add l eno1 name test type macvtap mode bridge
# ip l show test 
13: test@eno1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 500
    link/ether f2:dc:83:ac:a5:fb brd ff:ff:ff:ff:ff:ff

# virsh dumpxml rhel --xpath //interface 
<interface type="direct">
  <mac address="52:54:00:55:34:bc"/>
  <source dev="eno1" mode="bridge"/>
  <target dev="test"/>
  <model type="virtio"/>
  <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
</interface>

# virsh start rhel 
error: Failed to start domain 'rhel'
error: error creating macvtap interface test@eno1 (52:54:00:55:34:bc): File exists

# ip l show test 
Device "test" does not exist.

Try to start the vm the 2nd time, as the pre-created tap device is deleted, the vm start successfully:
# virsh start rhel 
Domain 'rhel' started

Comment 15 Michal Privoznik 2023-01-30 09:40:46 UTC
(In reply to yalzhang from comment #14) 
> Test on libvirt-9.0.0-2.el9.x86_64, for network interface, the result is as
> expected; 
> but for direct type interface, the pre-created tap device will be deleted,
> please check the scenario 2 below. Thank you.

Yeah, this type isn't fixed. Let me write and post patches.

Comment 17 Michal Privoznik 2023-02-01 08:59:02 UTC
Alright, so after I started working on patches, I came up with a wide variety of cleanup patches which would need to be backported as well. So let us track that issue in a separate bug and leave this one only for <interface type="network"/>.

Comment 21 yalzhang@redhat.com 2023-02-02 12:02:18 UTC
Test on libvirt-9.0.0-3.el9.x86_64 with interface type='network', the result is as expected.

1. Start vm with target dev name occupied as in comment 14, vm failed to start:
# virsh start rhel 
error: Failed to start domain 'rhel'
error: Requested operation is not valid: The mytap0 interface already exists

Try several times, the result is the same;
Restart virtqemud service and try again, the result is the same; 

2. Hotplug interface with duplicate target dev name:
# ip l show mytap0
Device "mytap0" does not exist.

# virsh domiflist rhel 
 Interface   Type      Source    Model    MAC
-------------------------------------------------------------
 mytap0      network   default   e1000e   52:54:00:a0:2a:75

# virsh start rhel 
Domain 'rhel' started


# virsh attach-interface rhel network default --model virtio --target mytap0
error: Failed to attach interface
error: Requested operation is not valid: The mytap0 interface already exists

Try several times, the result is the same;

3. Start vm with 2 interface with duplicate target dev name:
# ip l show mytap0
Device "mytap0" does not exist.

# virsh domiflist rhel 
 Interface   Type      Source    Model    MAC
-------------------------------------------------------------
 mytap0      network   default   virtio   52:54:00:92:9c:16
 mytap0      network   default   virtio   52:54:00:c1:c3:b4

# virsh start rhel 
error: Failed to start domain 'rhel'
error: Requested operation is not valid: The mytap0 interface already exists

# virsh list 
 Id   Name   State
--------------------

# ip l show mytap0
Device "mytap0" does not exist.

4. start a vm with interface target dev name as "mytap0", then try to start another vm with the same target dev name:
# virsh start test 
Domain 'test' started

# virsh domiflist test 
 Interface   Type      Source    Model    MAC
-------------------------------------------------------------
 mytap0      network   default   virtio   52:54:00:1f:d5:05

# virsh domiflist rhel
 Interface   Type      Source    Model    MAC
-------------------------------------------------------------
 mytap0      network   default   virtio   52:54:00:92:9c:16

# virsh start rhel
error: Failed to start domain 'rhel'
error: Requested operation is not valid: The mytap0 interface already exists

after then, check the network function on vm test, the network works well.

Comment 22 mhou 2023-02-14 11:12:24 UTC
I thought this configuration can still start guest as well. 
ip tuntap add tap_vnet1 mode tap
nmcli dev set tap_vnet1 managed no
ip link set tap_vnet1 mtu 9200
ip link set tap_vnet1 up
ovs-vsctl add-port ovsb1 tap_vnet1
uuid=$(ovs-vsctl get port tap_vnet1 interfaces | tr -d '[|]')

<interface type='bridge'>
        <mac address='00:de:ad:00:00:01'/>
        <source bridge='ovsbr1'/>
        <virtualport type='openvswitch'>
          <parameters interfaceid='${uuid}'/>
        </virtualport>
        <model type='virtio'/>
        <driver name='vhost' iommu='on' ats='on'/>
        <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
</interface>

Comment 23 mhou 2023-02-14 11:12:35 UTC
I thought this configuration can still start guest as well. 
ip tuntap add tap_vnet1 mode tap
nmcli dev set tap_vnet1 managed no
ip link set tap_vnet1 mtu 9200
ip link set tap_vnet1 up
ovs-vsctl add-port ovsb1 tap_vnet1
uuid=$(ovs-vsctl get port tap_vnet1 interfaces | tr -d '[|]')

<interface type='bridge'>
        <mac address='00:de:ad:00:00:01'/>
        <source bridge='ovsbr1'/>
        <virtualport type='openvswitch'>
          <parameters interfaceid='${uuid}'/>
        </virtualport>
        <model type='virtio'/>
        <driver name='vhost' iommu='on' ats='on'/>
        <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
</interface>

Comment 24 Michal Privoznik 2023-02-14 12:58:02 UTC
(In reply to mhou from comment #23)
> I thought this configuration can still start guest as well. 

That can be tracked in a separate bug.

Comment 25 yalzhang@redhat.com 2023-02-14 14:31:32 UTC
(In reply to mhou from comment #23)
> I thought this configuration can still start guest as well. 
> ip tuntap add tap_vnet1 mode tap
> nmcli dev set tap_vnet1 managed no
> ip link set tap_vnet1 mtu 9200
> ip link set tap_vnet1 up
> ovs-vsctl add-port ovsb1 tap_vnet1
> uuid=$(ovs-vsctl get port tap_vnet1 interfaces | tr -d '[|]')
> 
> <interface type='bridge'>
>         <mac address='00:de:ad:00:00:01'/>
>         <source bridge='ovsbr1'/>
>         <virtualport type='openvswitch'>
>           <parameters interfaceid='${uuid}'/>
>         </virtualport>
>         <model type='virtio'/>
>         <driver name='vhost' iommu='on' ats='on'/>
>         <address type='pci' domain='0x0000' bus='0x03' slot='0x00'
> function='0x0'/>
> </interface>

Hi Minxi, there is no <target dev="tap_vnet1"/> in your xml, so the vm can start successfully.
This is not the scenario for this bug. Could you please double confirm it?

Comment 26 mhou 2023-02-14 14:52:44 UTC
Hello yalan

tap_vnet1 already add to ovs bridge. so the uuid represents tap_vnet1
ovs-vsctl add-port ovsb1 tap_vnet1
ovs-vsctl list interface tap_vnet1
_uuid               : 626fce11-b0ff-4419-ab82-9cb5aafa8079
admin_state         : up
bfd                 : {}
bfd_status          : {}
cfm_fault           : []
cfm_fault_status    : []
cfm_flap_count      : []
cfm_health          : []
cfm_mpid            : []
cfm_remote_mpids    : []
cfm_remote_opstate  : []
duplex              : full
error               : []
external_ids        : {}
ifindex             : 13
ingress_policing_burst: 0
ingress_policing_kpkts_burst: 0
ingress_policing_kpkts_rate: 0
ingress_policing_rate: 0
lacp_current        : []
link_resets         : 0
link_speed          : 10000000
link_state          : down
lldp                : {}
mac                 : []
mac_in_use          : "42:ab:ef:24:91:76"
mtu                 : 9120
mtu_request         : []
name                : tap_vnet1
ofport              : 6
ofport_request      : []
options             : {}
other_config        : {}
statistics          : {collisions=0, rx_bytes=0, rx_crc_err=0, rx_dropped=0, rx_errors=0, rx_frame_err=0, rx_missed_errors=0, rx_multicast_packets=0, rx_
over_err=0, rx_packets=0, tx_bytes=0, tx_dropped=0, tx_errors=0, tx_packets=0}
status              : {driver_name=tun, driver_version="1.6", firmware_version=""}
type                : ""

Comment 27 yalzhang@redhat.com 2023-02-15 02:10:54 UTC
Hi Minxi,

I have tried comment23, when vm start, libvirt will create a new tap device.
If we need to use a pre-created tap/macvtap device, we can use ethernet + managed='no' instead.

continue with comment23:
1. check the uuid of tap_vnet1, and set the uuid in the interface xml:
# uuid=$(ovs-vsctl get port tap_vnet1 interfaces | tr -d '[|]')

# echo $uuid
d7d87a16-9f03-43d3-a283-452b260ac921

# virsh dumpxml rhel --xpath //interface 
<interface type="bridge">
  <mac address="52:54:00:c9:af:57"/>
  <source bridge="ovsbr1"/>
  <virtualport type="openvswitch">
    <parameters interfaceid="d7d87a16-9f03-43d3-a283-452b260ac921"/>
  </virtualport>
  <model type="virtio"/>
  <driver name="vhost" iommu="on" ats="on"/>
  <address type="pci" domain="0x0000" bus="0x07" slot="0x00" function="0x0"/>
</interface>

2. start vm and check the live xml:
# virsh start rhel 
Domain 'rhel' started

# virsh dumpxml rhel --xpath //interface 
<interface type="bridge">
  <mac address="52:54:00:c9:af:57"/>
  <source bridge="ovsb1"/>
  <virtualport type="openvswitch">
    <parameters interfaceid="d7d87a16-9f03-43d3-a283-452b260ac921"/>
  </virtualport>
  <target dev="vnet2"/>   ---------> libvirt create a new tap device named "vnet2"
  <model type="virtio"/>
  <driver name="vhost" iommu="on" ats="on"/>
  <alias name="net0"/>
  <address type="pci" domain="0x0000" bus="0x07" slot="0x00" function="0x0"/>
</interface>

3. check the interfaces on the ovs bridge:
# ovs-vsctl show 
06dd5f08-5f63-4d86-8bd6-00562d71c6aa
    Bridge ovsb1
        Port vnet2
            Interface vnet2 ---> libvirt created one
        Port tap_vnet1
            Interface tap_vnet1  ---> pre-created one
        Port ovsb1
            Interface ovsb1
                type: internal
    ovs_version: "2.15.8"

# ovs-vsctl list interface tap_vnet1 
_uuid               : d7d87a16-9f03-43d3-a283-452b260ac921
admin_state         : up
bfd                 : {}
bfd_status          : {}
cfm_fault           : []
cfm_fault_status    : []
cfm_flap_count      : []
cfm_health          : []
cfm_mpid            : []
cfm_remote_mpids    : []
cfm_remote_opstate  : []
duplex              : full
error               : []
external_ids        : {}
ifindex             : 14
ingress_policing_burst: 0
ingress_policing_rate: 0
lacp_current        : []
link_resets         : 0
link_speed          : 10000000
link_state          : down
lldp                : {}
mac                 : []
mac_in_use          : "be:7e:38:bb:3c:11"
mtu                 : 9200
mtu_request         : []
name                : tap_vnet1
ofport              : 1
ofport_request      : []
options             : {}
other_config        : {}
statistics          : {collisions=0, rx_bytes=0, rx_crc_err=0, rx_dropped=0, rx_errors=0, rx_frame_err=0, rx_missed_errors=0, rx_over_err=0, rx_packets=0, tx_bytes=0, tx_dropped=0, tx_errors=0, tx_packets=0}
status              : {driver_name=tun, driver_version="1.6", firmware_version=""}
type                : ""

# ovs-vsctl list interface tap_vnet1 
_uuid               : d7d87a16-9f03-43d3-a283-452b260ac921
admin_state         : up
bfd                 : {}
bfd_status          : {}
cfm_fault           : []
cfm_fault_status    : []
cfm_flap_count      : []
cfm_health          : []
cfm_mpid            : []
cfm_remote_mpids    : []
cfm_remote_opstate  : []
duplex              : full
error               : []
external_ids        : {}
ifindex             : 14
ingress_policing_burst: 0
ingress_policing_rate: 0
lacp_current        : []
link_resets         : 0
link_speed          : 10000000
link_state          : down
lldp                : {}
mac                 : []
mac_in_use          : "be:7e:38:bb:3c:11"
mtu                 : 9200
mtu_request         : []
name                : tap_vnet1
ofport              : 1
ofport_request      : []
options             : {}
other_config        : {}
statistics          : {collisions=0, rx_bytes=0, rx_crc_err=0, rx_dropped=0, rx_errors=0, rx_frame_err=0, rx_missed_errors=0, rx_over_err=0, rx_packets=0, tx_bytes=0, tx_dropped=0, tx_errors=0, tx_packets=0}
status              : {driver_name=tun, driver_version="1.6", firmware_version=""}
type                : ""


# ovs-vsctl list interface vnet2
_uuid               : e0533fc8-5a5c-4865-a425-5fd382b41b84
admin_state         : up
bfd                 : {}
bfd_status          : {}
cfm_fault           : []
cfm_fault_status    : []
cfm_flap_count      : []
cfm_health          : []
cfm_mpid            : []
cfm_remote_mpids    : []
cfm_remote_opstate  : []
duplex              : full
error               : []
external_ids        : {attached-mac="52:54:00:c9:af:57", iface-id="d7d87a16-9f03-43d3-a283-452b260ac921", iface-status=active, vm-id="a5fa1f75-d3b0-4e49-952a-a8c306cc30c7"}

                                                         ^^^^^^^^^^^^^^^^^^^^^^ this is the iface-id which we have set in the xml         
ifindex             : 19
ingress_policing_burst: 0
ingress_policing_rate: 0
lacp_current        : []
link_resets         : 1
link_speed          : 10000000
link_state          : up
lldp                : {}
mac                 : []
mac_in_use          : "fe:54:00:c9:af:57"
mtu                 : 9200
mtu_request         : []
name                : vnet2
ofport              : 2
ofport_request      : []
options             : {}
other_config        : {}
statistics          : {collisions=0, rx_bytes=9136, rx_crc_err=0, rx_dropped=0, rx_errors=0, rx_frame_err=0, rx_missed_errors=0, rx_over_err=0, rx_packets=56, tx_bytes=2432, tx_dropped=0, tx_errors=0, tx_packets=20}
status              : {driver_name=tun, driver_version="1.6", firmware_version=""}
type                : ""

From libvirtd.org, I think the result is as expected. The interfaceid is hornored, it's not the one shows in "_uuid"(I have no idea about the '_uuid').

"The Open vSwitch type virtualport accepts two parameters in its <parameters> element - an interfaceid which is a standard uuid used to uniquely identify this particular interface to Open vSwitch (if you do not specify one, a random interfaceid will be generated for you when you first define the interface), and an optional profileid which is sent to Open vSwitch as the interfaces "port-profile"."

Comment 28 mhou 2023-02-15 02:46:11 UTC
As per talk offline, I agree with you comments. I will close bug2169685.

Comment 30 errata-xmlrpc 2023-05-09 07:27:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (libvirt bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2171


Note You need to log in before you can comment on or make changes to this bug.