Bug 1370026

Summary: vfio-pci device can not work with pxb
Product: Red Hat Enterprise Linux 7 Reporter: mazhang <mazhang>
Component: qemu-kvm-rhevAssignee: David Gibson <dgibson>
Status: CLOSED NOTABUG QA Contact: Zhengtong <zhengtli>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.3CC: chayang, juzhang, knoel, michen, qzhang, thuth, virt-maint, yfu, zhengtli
Target Milestone: rc   
Target Release: ---   
Hardware: ppc64le   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-01-03 01:38:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
info qtree
none
guest dmesg none

Comment 1 mazhang 2016-08-25 05:43:14 UTC
Created attachment 1193880 [details]
info qtree

Comment 2 mazhang 2016-08-25 05:52:44 UTC
Created attachment 1193881 [details]
guest dmesg

Comment 4 David Gibson 2016-09-07 00:47:13 UTC
I suspect this is a duplicate of bug 1366953.  We'll look at this again once we have a fix for that bug.

Comment 5 David Gibson 2016-09-07 23:52:49 UTC
This is probably a dupe of 1366953 which has been deferred to 7.4.  In addition there's not much call to use the pxb at all on Power.  Therefore, deferring to 7.4.

Comment 11 Zhengtong 2016-11-21 05:38:12 UTC
Hi Thomas , I tried with the scratch SLOF package ,and no luck , it failed.

Qemu-kvm-rhel-2.6.0-27.el7
Host kernel: 3.10.0-514.el7.ppc64le
Guest kernel: 3.10.0-514.el7.ppc64le


Steps:
1. Switch the SLOF package to the testing one

[root@ibm-p8-rhevm-13 bug_verify]# rpm -qa | grep SLOF
SLOF-20160223-6.gitdbbfda4.el7.test.noarch

2. Boot guest with vfio-pci device.

3. Check the devices inside guest.
[root@localhost ~]# ip link show
ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000
    link/ether 9a:d3:d4:d5:d6:d7 brd ff:ff:ff:ff:ff:ff

[root@localhost ~]# lspci
lspci
00:00.0 VGA compatible controller: Device 1234:1111 (rev 02)
00:01.0 Keyboard controller: Red Hat, Inc Virtio input (rev 01)
00:02.0 Host bridge: Red Hat, Inc. Device 0009
00:03.0 USB controller: Apple Inc. KeyLargo/Intrepid USB
00:04.0 SCSI storage controller: Red Hat, Inc Virtio SCSI
00:05.0 Ethernet controller: Red Hat, Inc Virtio network device

[root@localhost ~]# lspci -vvv -n -s 00:02.0
lspci -vvv -n -s 00:02.0
00:02.0 0600: 1b36:0009
        Subsystem: 1af4:1100
        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap- 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-


Result:
***The vfio device didn't shows up in output of "ip link show" cmd.


Guest boot command:
[root@ibm-p8-rhevm-13 bug_verify]# cat boot_guest.sh 
/usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm1'  \
    -sandbox off  \
    -nodefaults  \
    -vga std  \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20160801-082615-jcEqPDZo,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=qmp_id_catch_monitor,path=/tmp/monitor-catch_monitor-20160801-082615-jcEqPDZo,server,nowait \
    -mon chardev=qmp_id_catch_monitor,mode=control  \
    -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20160801-082615-jcEqPDZo,server,nowait \
    -device spapr-vty,reg=0x30000000,chardev=serial_id_serial0 \
    -device pci-ohci,id=usb1,bus=pci.0,addr=03 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=04 \
    -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/bug_verify/RHEL-Server-7.3-ppc64le-virtio-scsi.qcow2 \
    -device scsi-hd,id=image1,drive=drive_image1 \
    -netdev tap,id=idWfrmjh,vhost=on \
    -device virtio-net-pci,mac=9a:d3:d4:d5:d6:d7,id=id7DYoYI,netdev=idWfrmjh,bus=pci.0,addr=05 \
    -m 4096 \
    -smp 32 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -device usb-mouse \
    -vnc :0  \
    -rtc base=utc,clock=host  \
    -boot order=cdn,once=c,menu=off,strict=off \
    -enable-kvm \
    -monitor stdio \
    -device virtio-keyboard-pci,id=kbd0 \
    -M pseries \
    -device pxb,id=pxb,bus_nr=4,numa_node=0 \
    -object memory-backend-ram,host-nodes=0,policy=bind,id=mem-0,size=4096M,prealloc=yes -numa node,memdev=mem-0,nodeid=0 \
    -device vfio-pci,host=0003:09:00.0,id=vf1,bus=pxb,addr=0x1 \


**Please tell me if any of my steps is not correct. thanks.

Comment 12 David Gibson 2016-12-02 04:58:53 UTC
Looking into how pxb works, I'm reasonably convinced it doesn't make any sense for ppc at all.

I'll try to confirm this, then file a bug to get it removed from the build.

Comment 13 David Gibson 2017-01-03 01:38:12 UTC
I've confirmed that pxb doesn't make sense for POWER.  Removal is being tracked in bug 1400785.