Bug 2166979

Summary: Creating guest with SR-IOV VF device fails due to permission denied when accessing /dev/vfio/
Product: Red Hat Enterprise Linux 9 Reporter: James Parker <jparker>
Component: libvirtAssignee: Michal Privoznik <mprivozn>
libvirt sub component: Networking QA Contact: Yanqiu Zhang <yanqzhan>
Status: CLOSED DUPLICATE Docs Contact:
Severity: unspecified    
Priority: unspecified CC: alex.williamson, bdobreli, bmv, juzhou, kchamart, lmen, mnietoji, mprivozn, pgrist, smooney, virt-maint, vkhitrin, yafu, yalzhang, yanghliu
Version: 9.2Keywords: Triaged
Target Milestone: rcFlags: pm-rhel: mirror+
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-02-21 13:37:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2166968    

Comment 1 Michal Privoznik 2023-02-06 12:14:49 UTC
Yeah, I think we need a bit more info. domain XML, steps to reproduce and logs would be a great start.

https://libvirt.org/kbase/debuglogs.html

Comment 17 Yanqiu Zhang 2023-02-16 03:57:57 UTC
I can not reproduce with:
libvirt-9.0.0-5.el9.x86_64
qemu-kvm-7.2.0-8.el9.x86_64
kernel-5.14.0-252.el9.x86_64

# virsh start avocado-vt-vm1 
Domain 'avocado-vt-vm1' started

# virsh dumpxml avocado-vt-vm1 |grep /interface -B 11
    <interface type='hostdev' managed='yes'>
      <mac address='fa:16:3e:3c:88:c1'/>
      <driver name='vfio'/>
      <source>
        <address type='pci' domain='0x0000' bus='0x3b' slot='0x02' function='0x0'/>
      </source>
      <vlan>
        <tag id='2000'/>
      </vlan>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>

Comment 18 Yanqiu Zhang 2023-02-16 04:22:54 UTC
1. Starting guest with hostdev interface, source address and vlan settings has been automated and always pass in our auto job:
(1)Auto case url:
https://libvirt-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/libvirt-RHEL-9.2-runtest-x86_64-function-sriov/36/testReport/rhel/sriov/vm_lifecycle_start_destroy_hostdev_interface_vf_address_vlan/
<interface managed="yes" type="hostdev">
      <mac address="9a:e8:93:1b:5a:4a" />
      <driver name="vfio" />
      <source>
        <address bus="0x3b" domain="0x0000" function="0x0" slot="0x10" type="pci" />
      </source>
      <vlan>
        <tag id="42" />
      </vlan>
      <alias name="ua-89cbe690-6c6c-4f2f-adac-5826fe52ea74" />
      <address bus="0x01" domain="0x0000" function="0x0" slot="0x00" type="pci" />
    </interface>

(2)History:
https://libvirt-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/libvirt-RHEL-9.2-runtest-x86_64-function-sriov/36/testReport/rhel/sriov/vm_lifecycle_start_destroy_hostdev_interface_vf_address_vlan/history/


2. Even with specified disk operations, vfio is not affected:
# mkdir /images
# dd if=/dev/zero of=/images/disk2.img bs=1M count=6104
# mkfs.ext4 /images/disk2.img
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type="file" device="disk">
    <driver name="qemu" type="raw"/>
    <source file="/images/disk2.img"/>
    <target dev="vda" bus="virtio"/>
    </disk>
...
        <interface type='hostdev' managed='yes'>
      <mac address='fa:16:3e:3c:88:c1'/>
      <driver name='vfio'/>
      <source>
        <address type='pci' domain='0x0000' bus='0x3b' slot='0x02' function='0x0'/>
      </source>
      <vlan>
        <tag id='2000'/>
      </vlan>
      <alias name='hostdev0'/>
    </interface>

# ls -al /dev/vfio/
total 0
drwxr-xr-x.  2 root root      60 Feb 15 22:54 .
drwxr-xr-x. 21 root root    3520 Feb 15 22:54 ..
crw-rw-rw-.  1 root root 10, 196 Feb 15 22:54 vfio

# virsh define avocado-vfio.xml 
Domain 'avocado-vt-vm1' defined from avocado-vfio.xml

# virsh start avocado-vt-vm1 
Domain 'avocado-vt-vm1' started

# ls -al /dev/vfio/
total 0
drwxr-xr-x.  2 root root       80 Feb 15 23:11 .
drwxr-xr-x. 21 root root     3520 Feb 15 22:54 ..
crw-------.  1 root root 235,   0 Feb 15 23:11 132
crw-rw-rw-.  1 root root  10, 196 Feb 15 22:54 vfio

Comment 24 Yanqiu Zhang 2023-02-17 01:54:22 UTC
On rhel9.2 host, we use default settings as:
# mount|grep hugepage
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel,pagesize=2M)

#  ll -d -Z /dev/hugepages
drwxr-xr-x. 3 root root system_u:object_r:hugetlbfs_t:s0 0 Feb 15 22:49 /dev/hugepages


On OSP, it may be:
[4:40 PM] [root@compute-0 qemu]#  ll -d -Z /dev/hugepages
drwxrwxr-x. 3 root hugetlbfs system_u:object_r:hugetlbfs_t:s0 0 Feb 16 07:33 /dev/hugepages


I'm not sure how OSP did this difference now.

Comment 29 Bogdan Dobrelya 2023-02-21 13:19:30 UTC
*** Bug 2167102 has been marked as a duplicate of this bug. ***

Comment 31 Bogdan Dobrelya 2023-02-21 13:37:33 UTC

*** This bug has been marked as a duplicate of bug 2167102 ***