Bug 1196185

This is fixed upstream already by:

commit f0bd70a940de690216c538b0ab1b71c8a7d2fbb6
Author:     Laine Stump <laine>
AuthorDate: Thu Apr 25 06:37:21 2013 -0400
Commit:     Laine Stump <laine>
CommitDate: Thu Apr 25 21:28:43 2013 -0400

    security: update hostdev labelling functions for VFIO
    
    Legacy kvm style pci device assignment requires changes to the
    labelling of several sysfs files for each device, but for vfio device
    assignment, the only thing that needs to be relabelled/chowned is the
    "group" device for the group that contains the device to be assigned.

v1.0.5-rc1

However, there were some subsequent fixes.

I'm trying to reproduce this bug with 
libvirt-1.2.13-1.el7.x86_64
qemu-kvm-rhev-2.3.0-9.el7.x86_64

1. Set qemu to run under non-root user

# echo '
user = "test"
group = "test"' >> /etc/libvirt/qemu.conf

# systemctl restart  libvirtd.service

2. Define a guest with hostdev attached

# virsh dumpxml r7|grep hostdev -A8
    <interface type='hostdev'>
      <mac address='52:54:00:6d:90:02'/>
      <driver name='vfio'/>
      <source>
        <address type='pci' domain='0x0000' bus='0x03' slot='0x10' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </interface>
    <serial type='pty'>

3. Try to start guest 

# virsh start r7 
Domain r7 started

# ps aux|grep qemu|grep vfio 
test      9566 63.2 13.8 1516168 1115372 ?     SLl  21:24   0:21 /usr/libexec/qemu-kvm -name r7 -S -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off -m 1024 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 37367d9f-dd35-4425-952c-70024253a082 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/r7.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/var/lib/libvirt/images/rhel7.img,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -device usb-tablet,id=input0 -spice port=5900,addr=127.0.0.1,disable-ticketing,seamless-migration=on -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1 -device vfio-pci,host=03:10.0,id=hostdev0,bus=pci.0,addr=0x9 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on

# ll /dev/vfio/* 
crw-------. 1 test test 245,   0 Jul 15 20:50 /dev/vfio/23
crw-rw-rw-. 1 root root  10, 196 Jul 15 16:25 /dev/vfio/vfio

So I can succeed booting up the guest with hostdev attached(sriov nic). 

Martin,

Can you show me details to reproduce this bug or correct my steps above?

Thanks

# virsh start r7 
Domain r7 started


# virsh dumpxml r7|grep hostdev -A 5
    <interface type='hostdev'>
      <mac address='52:54:00:6d:90:02'/>
      <driver name='vfio'/>
      <source>
        <address type='pci' domain='0x0000' bus='0x03' slot='0x10' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/1'/>
      <target port='0'/>
--
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x03' slot='0x10' function='0x2'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </hostdev>

# ll /dev/vfio/* 
crw-------. 1 test test 245,   0 Jul 15 20:50 /dev/vfio/23
crw-------. 1 test test 245,   1 Jul 15 21:41 /dev/vfio/24
crw-rw-rw-. 1 root root  10, 196 Jul 15 16:25 /dev/vfio/vfio

# ps aux|grep vfio
root      4355  0.0  0.0      0     0 ?        S<   16:25   0:00 [vfio-irqfd-clea]
test     10327 57.8 13.9 1532848 1121300 ?     SLl  21:41   0:22 /usr/libexec/qemu-kvm -name r7 -S -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off -m 1024 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 37367d9f-dd35-4425-952c-70024253a082 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/r7.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/var/lib/libvirt/images/rhel7.img,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -device usb-tablet,id=input0 -spice port=5902,addr=127.0.0.1,disable-ticketing,seamless-migration=on -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1 -device vfio-pci,host=03:10.0,id=hostdev0,bus=pci.0,addr=0x9 -device vfio-pci,host=03:10.2,id=hostdev1,bus=pci.0,addr=0x3 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on

Change this bug with verify status after do some testing with latest 
libvirtlibvirt-1.2.17-2.el7.x86_64

Steps as in comment 3 and comment 4.

It seems that the is similar 1248105 - RHEV by default sets dynamic_ownership=0, which caused the endpoint not to be accessible by qemu (and we explicitly told libvirt not to do it for us). Works with dynamic_ownership=1.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2202.html