Bug 2209853

Summary: Starting vm with an interface from hostdev network reports unexpected error
Product: Red Hat Enterprise Linux 9 Reporter: Yanqiu Zhang <yanqzhan>
Component: libvirtAssignee: Michal Privoznik <mprivozn>
libvirt sub component: Networking QA Contact: Yanqiu Zhang <yanqzhan>
Status: CLOSED ERRATA Docs Contact:
Severity: unspecified    
Priority: unspecified CC: jdenemar, jsuchane, lmen, lvivier, mprivozn, virt-maint, xuzhang, yalzhang, yanqzhan, yicui
Version: 9.3Keywords: Automation, Regression, Triaged, Upstream
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-9.5.0-0rc1.1.el9 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-11-07 08:31:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version: 9.5.0
Embargoed:

Description Yanqiu Zhang 2023-05-25 04:40:19 UTC
Description of problem:
Define and start a hostdev network and do not manually detach the nodedev from host. Add an interface with this network to vm, try to start vm then got unexpected error.

Version-Release number of selected component (if applicable):
libvirt-9.3.0-2.el9.x86_64
qemu-kvm-8.0.0-4.el9.x86_64
kernel-5.14.0-316.el9.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Start a hostdev network with managed="no":
<network>
<name>hostdev_net</name>
<forward mode="hostdev" managed="no">
<pf dev="ens1f0" />
</forward>
</network>

2. Add to vm an interface from this network:
<interface type="network">
<source network="hostdev_net" />
<mac address="9a:a8:62:8b:f7:2a" />
<alias name="ua-89cbe690-6c6c-4f2f-adac-5826fe52ea74" />
</interface>

3. Try to start the vm:
# virsh start avocado-vt-vm1
error: Failed to start domain 'avocado-vt-vm1'
error: unsupported configuration: pci backend driver 'default' is not supported


Actual results:


Expected results:
error: Requested operation is not valid: Unmanaged PCI device 0000:3b:02.0 must be manually detached from the host


Additional info:
Not reproduced on libvirt-9.2.0-1.el9.x86_64. (should be involved from libvirt-9.3.0-1)

Comment 1 Yanqiu Zhang 2023-05-26 03:43:28 UTC
Also reproduces for managed='yes' hostdev network:

<network>
  <name>hostdev_net</name>
  <uuid>9803f659-ac1c-491e-b836-a30d3b84784a</uuid>
  <forward mode='hostdev' managed='yes'>
    <pf dev='ens1f0'/>
  </forward>
</network>

On libvirt-9.3.0-2.el9.x86_64:
# virsh start avocado-vt-vm1 
error: Failed to start domain 'avocado-vt-vm1'
error: unsupported configuration: pci backend driver 'default' is not supported


On libvirt-9.2.0-1.el9.x86_64:
# virsh start avocado-vt-vm1 
Domain 'avocado-vt-vm1' started

Comment 2 Michal Privoznik 2023-06-01 13:27:21 UTC
Yep, this is a result of some changes I made. Patch posted on the list:

https://listman.redhat.com/archives/libvir-list/2023-June/240150.html

Comment 3 Michal Privoznik 2023-06-05 10:28:19 UTC
Merged upstream as:

commit 8b9d2bda8a7a54e669b476d3c700d09124807f00
Author:     Michal Prívozník <mprivozn>
AuthorDate: Thu Jun 1 13:47:14 2023 +0200
Commit:     Michal Prívozník <mprivozn>
CommitDate: Mon Jun 5 12:18:53 2023 +0200

    qemu: Set proper PCI backend for <interface/>-s that are actually hostdevs
    
    When starting a domain, it's done so in two steps (actually more,
    but lets focus on just the following two):
    
      1) qemuProcessPrepareDomain(), followed by
    
      2) qemuProcessPrepareHost().
    
    Now, in the first step (PrepareDomain()), PCI backends for all
    hostdevs is set (qemuProcessPrepareDomain() ->
    qemuProcessPrepareDomainHostdevs() -> qemuDomainPrepareHostdev()
    -> qemuDomainPrepareHostdevPCI()). Perfect.
    
    But then, additional hostdevs may appear, because in the host
    prepare phase we may insert some hostdevs into domain definition
    (qemuProcessPrepareHost() -> qemuProcessNetworkPrepareDevices()).
    
    Now, these additional hostdevs don't undergo the same prepare as
    hostdevs that were already present in the domain definition (i.e.
    in qemuProcessPrepareDomain() phase). Therefore, we have to call
    corresponding prepare function explicitly.
    
    NB, the interface hotplug code (qemuDomainAttachNetDevice()) does
    not suffer from this problem, because it calls top level
    qemuDomainAttachHostDevice() which is used to hotplug regular
    hostdevs too and as such calls qemuDomainPrepareHostdev().
    
    Fixes: 3b87709c768480e085556e06bd8d08f62270d42d
    Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2209853
    Signed-off-by: Michal Privoznik <mprivozn>
    Reviewed-by: Martin Kletzander <mkletzan>

v9.4.0-14-g8b9d2bda8a

Comment 4 Yanqiu Zhang 2023-07-03 03:33:20 UTC
Verified with:
libvirt-9.5.0-0rc1.1.el9.x86_64

Steps:
1. prepare vm with following interface xml:
<interface type="network">
  <mac address="9a:a8:62:8b:f7:2a"/>
  <source network="hostdev_net"/>
  <alias name="ua-89cbe690-6c6c-4f2f-adac-5826fe52ea74"/>
  <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
</interface>

2. start vm from hostdev network with managed='yes'
<network>
  <name>hostdev_net</name>
  <uuid>30f180b5-3549-43f6-8a74-fb69588de226</uuid>
  <forward mode='hostdev' managed='yes'>
    <pf dev='ens1f0'/>
  </forward>
</network>

# virsh start avocado-vt-vm1 
Domain 'avocado-vt-vm1' started

3. start vm from hostdev network with managed='no'
<network>
<name>hostdev_net</name>
<forward mode="hostdev" managed="no">
<pf dev="ens1f0" />
</forward>
</network>

# virsh start avocado-vt-vm1 
error: Failed to start domain 'avocado-vt-vm1'
error: Requested operation is not valid: Unmanaged PCI device 0000:3b:02.0 must be manually detached from the host

And auto regression test is pass:
https://libvirt-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/libvirt-RHEL-9.3-runtest-x86_64-function-sriov/15/testReport/rhel/sriov/

Comment 9 errata-xmlrpc 2023-11-07 08:31:41 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: libvirt security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:6409