Bug 1263795

Summary: vfio device can't be hot unplugged on powerpc guest
Product: Red Hat Enterprise Linux 7 Reporter: Laurent Vivier <lvivier>
Component: qemu-kvm-rhevAssignee: Laurent Vivier <lvivier>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 7.2CC: dgibson, jen, knoel, lvivier, michen, mrezanin, qzhang, tlavigne, virt-bugs, virt-maint, zhengtli
Target Milestone: rc   
Target Release: ---   
Hardware: ppc64le   
OS: Linux   
Whiteboard:
Fixed In Version: qemu-kvm-rhev-2.3.0-25.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1250326 Environment:
Last Closed: 2015-12-04 16:58:19 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 1250326    
Bug Blocks:    

Comment 3 Miroslav Rezanina 2015-09-18 11:54:45 UTC 
Fix included in qemu-kvm-rhev-2.3.0-24.el7
Comment 4 Zhengtong 2015-09-22 03:23:51 UTC 
(Since this bug is clone of bug 1250326, the verified method is the same)

Tested with SLOF-20150313-5.gitc89b0df.el7 and qemu-kvm-rhev2.3.0-24. Succeed in host unplug the vfio device.

Details:
1. Boot guest with vfio device:
/usr/libexec/qemu-kvm \
...
    -device spapr-pci-vfio-host-bridge,id=vfiohost,iommu=1,index=0x1 \
    -device vfio-pci,host=0003:09:00.0,bus=vfiohost.0,addr=0x1,id=vfio_dev \
...

2. Check device in hmp interface
(qemu) info pci
  Bus  0, device   1, function 0:
    Ethernet controller: PCI device 14e4:1657
      IRQ 0.
      BAR0: 64 bit prefetchable memory at 0x100000000 [0x10000ffff].
      BAR2: 64 bit prefetchable memory at 0x100010000 [0x10001ffff].
      BAR4: 64 bit prefetchable memory at 0x100020000 [0x10002ffff].
      BAR6: 32 bit memory at 0xffffffffffffffff [0x0007fffe].
      id "vfio_dev"
  Bus  0, device   1, function 0:
    USB controller: PCI device 106b:003f
      IRQ 0.
      BAR0: 32 bit memory at 0xc0021000 [0xc00210ff].
      id "usb"
...

3. In guest: get an IP to the target device, and ping this IP from an external host

4. Hot unplug the device
(qemu) device_del vfio_dev 

5.Checked the result as follows:

 In guest: the device disappeared.
 [root@localhost ~]# ip link
ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00


In hmp interface. no this device after "info pci"
(qemu) info pci
 Bus  0, device   1, function 0:
    USB controller: PCI device 106b:003f
      IRQ 0.
      BAR0: 32 bit memory at 0xc0021000 [0xc00210ff].
      id "usb"
...


So,the bug could be marked verified.
Comment 5 Miroslav Rezanina 2015-09-23 13:27:00 UTC 
Fix included in qemu-kvm-rhev-2.3.0-25.el7
Comment 6 Zhengtong 2015-09-24 04:35:57 UTC 
Retested with SLOF-20150313-5.gitc89b0df.el7 and qemu-kvm-rhev-2.3.0-25.el7. Succeed in hot unplug the vfio device. 

Details:
1. Boot guest with vfio device:
/usr/libexec/qemu-kvm \
...
    -device spapr-pci-vfio-host-bridge,id=vfiohost,iommu=1,index=0x1 \
    -device vfio-pci,host=0003:09:00.0,bus=vfiohost.0,addr=0x1,id=vfio_dev \
...

2.Check the device by "info pci" under HMP. and do ping from external host
(qemu) info pci
  Bus  0, device   1, function 0:
    Ethernet controller: PCI device 14e4:1657
      IRQ 0.
      BAR0: 64 bit prefetchable memory at 0x100000000 [0x10000ffff].
      BAR2: 64 bit prefetchable memory at 0x100010000 [0x10001ffff].
      BAR4: 64 bit prefetchable memory at 0x100020000 [0x10002ffff].
      BAR6: 32 bit memory at 0xffffffffffffffff [0x0007fffe].
      id "vfio_dev"
  Bus  0, device   1, function 0:
    USB controller: PCI device 106b:003f
      IRQ 0.
      BAR0: 32 bit memory at 0xc0021000 [0xc00210ff].
      id "usb"
  Bus  0, device   2, function 0:
    SCSI controller: PCI device 1af4:1004
      IRQ 0.
      BAR0: I/O at 0x0040 [0x007f].
      BAR1: 32 bit memory at 0xc0020000 [0xc0020fff].
      id "scsi0"
  Bus  0, device   3, function 0:
    VGA controller: PCI device 1234:1111
      BAR0: 32 bit prefetchable memory at 0x80000000 [0x80ffffff].
      BAR2: 32 bit memory at 0xc0000000 [0xc0000fff].
      BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe].
      id "video0"

IN Guest:
[root@localhost ~]# ip link
ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enP1p0s1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 40:f2:e9:5d:9c:a8 brd ff:ff:ff:ff:ff:ff

3. Hot unplug vfio device.
(qemu) device_del vfio_dev

4. Check device in HMP and in Guest.
(qemu) info pci
  Bus  0, device   1, function 0:
    USB controller: PCI device 106b:003f
      IRQ 0.
      BAR0: 32 bit memory at 0xc0021000 [0xc00210ff].
      id "usb"
  Bus  0, device   2, function 0:
    SCSI controller: PCI device 1af4:1004
      IRQ 0.
      BAR0: I/O at 0x0040 [0x007f].
      BAR1: 32 bit memory at 0xc0020000 [0xc0020fff].
      id "scsi0"
  Bus  0, device   3, function 0:
    VGA controller: PCI device 1234:1111
      BAR0: 32 bit prefetchable memory at 0x80000000 [0x80ffffff].
      BAR2: 32 bit memory at 0xc0000000 [0xc0000fff].
      BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe].
      id "video0"

In Guest:
[root@localhost ~]# ip link
ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

The device hot unplugged successfully.
Comment 8 errata-xmlrpc 2015-12-04 16:58:19 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2546.html