Bug 1250333

Summary: qemu will complain: Bad file descriptor, if I didn't add vfio device and only add spapr-pci-vfio-host-bridge device
Product: Red Hat Enterprise Linux 7 Reporter: Zhengtong <zhengtli>
Component: qemu-kvm-rhevAssignee: David Gibson <dgibson>
Status: CLOSED WONTFIX QA Contact: Virtualization Bugs <virt-bugs>
Severity: low Docs Contact:
Priority: unspecified    
Version: 7.2CC: knoel, michen, qzhang, virt-maint
Target Milestone: rc   
Target Release: ---   
Hardware: ppc64le   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-11-05 03:11:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Zhengtong 2015-08-05 07:50:38 UTC
Description of problem:
Qemu always prompt: " vfio_dma_unmap(0x1001241be20, 0x7c837000, 0x1000) = -9 (Bad file descriptor)" If I only add spapr-pci-vfio-host-bridge , and no vfio-pci device in qemu command.

Version-Release number of selected component (if applicable):
qemu-kvm-rhev-2.3.0-13.el7

How reproducible:
100%

Steps to Reproduce:
1.boot guest with spapr-pci-vfio-host-bridge, and no vfio-pci device
/usr/libexec/qemu-kvm \
    -name vfio-liuzt-BE \
    -machine pseries,accel=kvm,usb=off \
    -m 4096 \
    -realtime mlock=off \
    -smp 4,sockets=2,cores=2,threads=1 \
    -uuid 5125cf27-4b01-4493-b46d-734d08becc6b \
    ...
    -netdev tap,id=hostnet0,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown,ifname=vnetvfio \
    -device spapr-vlan,netdev=hostnet0,id=net0,mac=52:54:00:5d:c5:9e,reg=0x2000 \
    -chardev socket,id=charserial0,path=serial,server,nowait \
    -device spapr-vty,chardev=charserial0,reg=0x30000000 \
    -device spapr-pci-vfio-host-bridge,id=vfiohost,iommu=1,index=0x1 \
    -no-shutdown \
 ...


2. Wait to see the booting process.


Actual results:
qemu will give bad file descriptor message until guest finish booting up 
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7fb08000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7fb09000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7fb0a000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7fb0b000, 0x1000) = -9 (Bad file descriptor)vfio_dma_unmap(0x100260bbe20, 0x7ffad000, 0x1000) = -11 (Resource temporarily unavailable)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffae000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffaf000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffb0000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffb1000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffb2000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffb3000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffb4000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffb5000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffb6000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffb7000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffb8000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffb9000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffba000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffbb000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffbc000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffbd000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffbe000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffbf000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffc0000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffc1000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffc2000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffc3000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffc4000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffc5000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffc6000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffc7000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffc8000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffc9000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffca000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffcb000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffcc000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffcd000, 0x1000) = -9 (Bad file descriptor)
qemu-kvm: VFIO_UNMAP_DMA: -9
qemu-kvm: vfio_dma_unmap(0x100260bbe20, 0x7ffce000, 0x1000) = -9 (Bad file descriptor)


Expected results:
Should not have these error msg.

Additional info:

This won't effect the guest running as for as I see.

Comment 2 Zhengtong 2015-08-05 08:08:26 UTC
More info :

If I hot plug  vfio-pci to this bridge, There will be ioctl warning msg

(qemu) device_add vfio-pci,host=0002:01:00.3,bus=vfiohost.0,addr=0x1,id=vfio_dev
Failed to create KVM VFIO device: No such device
(qemu) qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x60000000, 0x1000, 0x3ffed1170000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x60001000, 0x1000, 0x3ffed1171000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x60002000, 0x1000, 0x3ffed1172000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x60003000, 0x1000, 0x3ffed1173000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x60004000, 0x1000, 0x3ffed1174000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x60005000, 0x1000, 0x3ffed1175000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x60006000, 0x1000, 0x3ffed1176000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x60007000, 0x1000, 0x3ffed1177000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x60008000, 0x1000, 0x3ffed1178000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x60009000, 0x1000, 0x3ffed1179000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x6000a000, 0x1000, 0x3ffed117a000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x6000b000, 0x1000, 0x3ffed117b000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x6000c000, 0x1000, 0x3ffed117c000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x6000d000, 0x1000, 0x3ffed117d000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x6000e000, 0x1000, 0x3ffed117e000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x6000f000, 0x1000, 0x3ffed117f000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x60010000, 0x1000, 0x3ffed1110000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x60011000, 0x1000, 0x3ffed1111000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x60012000, 0x1000, 0x3ffed1112000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x60013000, 0x1000, 0x3ffed1113000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x60014000, 0x1000, 0x3ffed1114000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x60015000, 0x1000, 0x3ffed1115000) = -25 (Inappropriate ioctl for device)
qemu-kvm: VFIO_MAP_DMA: -25
qemu-kvm: vfio_dma_map(0x1003752be20, 0x60016000, 0x1000, 0x3ffed1116000) = -25 (Inappropriate ioctl for device)


about the message "Failed to create KVM VFIO device: No such device" , there is a bug related https://bugzilla.redhat.com/show_bug.cgi?id=1237034

Comment 3 David Gibson 2015-08-06 01:11:59 UTC
I suspect the errors after hotplugging a device are a different bug.  Let's deal with the errors on the bridge with no devices first.  If the errors after hotplug still appear, we can file a new BZ.

Comment 5 David Gibson 2015-11-05 03:11:20 UTC
With the latest changes, we're planning to deprecate spapr-vfio-pci-host-bridge, so I think this bug becomes irrelevant.