Bug 1356376
Summary: | [Q35] Nic which passthrough from host didn't be found in guest when enable multifunction | ||||||||
---|---|---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Linux 7 | Reporter: | jingzhao <jinzhao> | ||||||
Component: | qemu-kvm-rhev | Assignee: | Alex Williamson <alex.williamson> | ||||||
Status: | CLOSED ERRATA | QA Contact: | jingzhao <jinzhao> | ||||||
Severity: | high | Docs Contact: | |||||||
Priority: | high | ||||||||
Version: | 7.3 | CC: | ailan, alex.williamson, chayang, jinzhao, juzhang, knoel, marcel, mrezanin, virt-maint, yfu, yiwei | ||||||
Target Milestone: | rc | Keywords: | Reopened | ||||||
Target Release: | --- | ||||||||
Hardware: | Unspecified | ||||||||
OS: | Unspecified | ||||||||
Whiteboard: | |||||||||
Fixed In Version: | qemu-kvm-rhev-2.6.0-15.el7 | Doc Type: | If docs needed, set a value | ||||||
Doc Text: | Story Points: | --- | |||||||
Clone Of: | Environment: | ||||||||
Last Closed: | 2016-11-07 21:24:28 UTC | Type: | Bug | ||||||
Regression: | --- | Mount Type: | --- | ||||||
Documentation: | --- | CRM: | |||||||
Verified Versions: | Category: | --- | |||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||
Embargoed: | |||||||||
Attachments: |
|
Description
jingzhao
2016-07-14 02:59:02 UTC
The Nic info of host: [root@hp-z800-01 home]# lspci |grep Ether 01:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5764M Gigabit Ethernet PCIe (rev 10) 02:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5764M Gigabit Ethernet PCIe (rev 10) 03:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) 03:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) 03:10.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01) 03:10.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01) 03:10.4 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01) 03:10.6 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01) For any device behind a downstream port, ie. root port or downstream switch port, the only valid slot address is 0. Using 0xa is very much non-standard. If this cannot be reproduced with addr=0x0.0 and addr=0x0.1, then there's no bug here. I'm going to assume that case, re-open otherwise. Hi Alex Nic really can be found in guest, but I hit another problem, and I want to confrim with you Only 1 Nic can be found in guest, but I think we can found 2 nic normally, and actually we can found 2 nic in hmp Following is the info of guest: [root@unused ~]# lspci 00:00.0 Host bridge: Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller 00:01.0 VGA compatible controller: Red Hat, Inc. QXL paravirtual graphic card (rev 04) 00:02.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02) 00:03.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02) 00:04.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02) 00:1f.0 ISA bridge: Intel Corporation 82801IB (ICH9) LPC Interface Controller (rev 02) 00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] (rev 02) 00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 02) 01:00.0 SCSI storage controller: Red Hat, Inc Virtio block device (rev 01) 02:00.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Upstream) (rev 02) 03:00.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01) 03:01.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01) 03:02.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01) 04:00.0 SCSI storage controller: Red Hat, Inc Virtio block device (rev 01) 05:00.0 SCSI storage controller: Red Hat, Inc Virtio block device (rev 01) 07:00.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01) [root@unused ~]# lspci -vvv -t -[0000:00]-+-00.0 Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller +-01.0 Red Hat, Inc. QXL paravirtual graphic card +-02.0-[01]----00.0 Red Hat, Inc Virtio block device +-03.0-[02-06]----00.0-[03-06]--+-00.0-[04]----00.0 Red Hat, Inc Virtio block device | +-01.0-[05]----00.0 Red Hat, Inc Virtio block device | \-02.0-[06]-- +-04.0-[07]----00.0 Intel Corporation 82576 Virtual Function +-1f.0 Intel Corporation 82801IB (ICH9) LPC Interface Controller +-1f.2 Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] \-1f.3 Intel Corporation 82801I (ICH9 Family) SMBus Controller [root@unused ~]# ifconfig ens3f0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.66.152.155 netmask 255.255.255.0 broadcast 10.66.152.255 inet6 2620:52:0:4298:8c9f:d0ff:fe13:801d prefixlen 64 scopeid 0x0<global> inet6 fe80::8c9f:d0ff:fe13:801d prefixlen 64 scopeid 0x20<link> ether 8e:9f:d0:13:80:1d txqueuelen 1000 (Ethernet) RX packets 159 bytes 21199 (20.7 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 149 bytes 22478 (21.9 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1 (Local Loopback) RX packets 212 bytes 19172 (18.7 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 212 bytes 19172 (18.7 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 virbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 inet 192.168.122.1 netmask 255.255.255.0 broadcast 0.0.0.0 ether 00:00:00:00:00:00 txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 virbr0-nic: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 ether 52:54:00:41:03:50 txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 Following is the info of hmp: (qemu) info qtree bus: main-system-bus type System dev: kvm-ioapic, id "" gpio-in "" 24 gsi_base = 0 (0x0) mmio 00000000fec00000/0000000000001000 dev: q35-pcihost, id "" MCFG = 2952790016 (0xb0000000) pci-hole64-size = 0 (0 B) short_root_bus = 0 (0x0) bus: pcie.0 type PCIE dev: ioh3420, id "root.2" power_controller_present = true chassis = 0 (0x0) slot = 3 (0x3) port = 0 (0x0) aer_log_max = 8 (0x8) addr = 04.0 romfile = "" rombar = 1 (0x1) multifunction = false command_serr_enable = true class PCI bridge, addr 00:04.0, pci id 8086:3420 (sub 0000:0000) bus: root.2 type PCIE dev: vfio-pci, id "vf-10.2" host = "0000:03:10.2" sysfsdev = "/sys/bus/pci/devices/0000:03:10.2" x-intx-mmap-timeout-ms = 1100 (0x44c) x-vga = false x-req = true x-no-mmap = false x-no-kvm-intx = false x-no-kvm-msi = false x-no-kvm-msix = false x-pci-vendor-id = 32902 (0x8086) x-pci-device-id = 4298 (0x10ca) x-pci-sub-vendor-id = 4294967295 (0xffffffff) x-pci-sub-device-id = 4294967295 (0xffffffff) addr = 00.1 romfile = "" rombar = 1 (0x1) multifunction = false command_serr_enable = true class Ethernet controller, addr 07:00.1, pci id 8086:10ca (sub 8086:a03c) bar 0: mem at 0xf8608000 [0xf860bfff] bar 3: mem at 0xf860c000 [0xf860ffff] dev: vfio-pci, id "vf-10.0" host = "0000:03:10.0" sysfsdev = "/sys/bus/pci/devices/0000:03:10.0" x-intx-mmap-timeout-ms = 1100 (0x44c) x-vga = false x-req = true x-no-mmap = false x-no-kvm-intx = false x-no-kvm-msi = false x-no-kvm-msix = false x-pci-vendor-id = 32902 (0x8086) x-pci-device-id = 4298 (0x10ca) x-pci-sub-vendor-id = 4294967295 (0xffffffff) x-pci-sub-device-id = 4294967295 (0xffffffff) addr = 00.0 romfile = "" rombar = 1 (0x1) multifunction = true command_serr_enable = true class Ethernet controller, addr 07:00.0, pci id 8086:10ca (sub 8086:a03c) bar 0: mem at 0xf8600000 [0xf8603fff] bar 3: mem at 0xf8604000 [0xf8607fff] .................. (qemu) info pci Bus 0, device 0, function 0: Host bridge: PCI device 8086:29c0 id "" Bus 0, device 1, function 0: VGA controller: PCI device 1b36:0100 IRQ 10. BAR0: 32 bit memory at 0xf0000000 [0xf3ffffff]. BAR1: 32 bit memory at 0xf4000000 [0xf7ffffff]. BAR2: 32 bit memory at 0xf8a10000 [0xf8a11fff]. BAR3: I/O at 0xc040 [0xc05f]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe]. id "" Bus 0, device 2, function 0: PCI bridge: PCI device 8086:3420 BUS 0. secondary bus 1. subordinate bus 1. IO range [0x1000, 0x1fff] memory range [0xf8800000, 0xf89fffff] prefetchable memory range [0xfe000000, 0xfe7fffff] id "root.0" Bus 1, device 0, function 0: SCSI controller: PCI device 1af4:1042 IRQ 11. BAR1: 32 bit memory at 0xf8800000 [0xf8800fff]. BAR4: 64 bit prefetchable memory at 0xfe000000 [0xfe7fffff]. id "virtio-disk0" Bus 0, device 3, function 0: PCI bridge: PCI device 8086:3420 BUS 0. secondary bus 2. subordinate bus 6. IO range [0x2000, 0x4fff] memory range [0xf8000000, 0xf85fffff] prefetchable memory range [0xfc800000, 0xfdffffff] id "root.1" Bus 2, device 0, function 0: PCI bridge: PCI device 104c:8232 BUS 2. secondary bus 3. subordinate bus 6. IO range [0x2000, 0x4fff] memory range [0xf8000000, 0xf85fffff] prefetchable memory range [0xfc800000, 0xfdffffff] id "upstream1" Bus 3, device 0, function 0: PCI bridge: PCI device 104c:8233 BUS 3. secondary bus 4. subordinate bus 4. IO range [0x2000, 0x2fff] memory range [0xf8400000, 0xf85fffff] prefetchable memory range [0xfd000000, 0xfd7fffff] id "downstream1" Bus 4, device 0, function 0: SCSI controller: PCI device 1af4:1042 IRQ 11. BAR1: 32 bit memory at 0xf8400000 [0xf8400fff]. BAR4: 64 bit prefetchable memory at 0xfd000000 [0xfd7fffff]. id "virtio-disk1" Bus 3, device 1, function 0: PCI bridge: PCI device 104c:8233 BUS 3. secondary bus 5. subordinate bus 5. IO range [0x3000, 0x3fff] memory range [0xf8200000, 0xf83fffff] prefetchable memory range [0xfc800000, 0xfcffffff] id "downstream2" Bus 5, device 0, function 0: SCSI controller: PCI device 1af4:1042 IRQ 10. BAR1: 32 bit memory at 0xf8200000 [0xf8200fff]. BAR4: 64 bit prefetchable memory at 0xfc800000 [0xfcffffff]. id "virtio-disk2" Bus 3, device 2, function 0: PCI bridge: PCI device 104c:8233 BUS 3. secondary bus 6. subordinate bus 6. IO range [0x4000, 0x4fff] memory range [0xf8000000, 0xf81fffff] prefetchable memory range [0xfd800000, 0xfd9fffff] id "downstream3" Bus 0, device 4, function 0: PCI bridge: PCI device 8086:3420 BUS 0. secondary bus 7. subordinate bus 7. IO range [0x5000, 0x5fff] memory range [0xf8600000, 0xf87fffff] prefetchable memory range [0xfe800000, 0xfe9fffff] id "root.2" Bus 7, device 0, function 0: Ethernet controller: PCI device 8086:10ca BAR0: 64 bit memory at 0xf8600000 [0xf8603fff]. BAR3: 64 bit memory at 0xf8604000 [0xf8607fff]. id "vf-10.0" Bus 7, device 0, function 1: Ethernet controller: PCI device 8086:10ca BAR0: 64 bit memory at 0xf8608000 [0xf860bfff]. BAR3: 64 bit memory at 0xf860c000 [0xf860ffff]. id "vf-10.2" Bus 0, device 31, function 0: ISA bridge: PCI device 8086:2918 id "" Bus 0, device 31, function 2: SATA controller: PCI device 8086:2922 IRQ 10. BAR4: I/O at 0xc060 [0xc07f]. BAR5: 32 bit memory at 0xf8a12000 [0xf8a12fff]. id "" Bus 0, device 31, function 3: SMBus: PCI device 8086:2930 IRQ 10. BAR4: I/O at 0x0700 [0x073f]. id "" Could you help to confirm it, I will reopen it it's a issue Thanks Jing Zhao It looks like the 2nd function should have been found, can it be reproduced with a non-assigned device, like an emulated e1000? Include the complete new command line and guest dmesg when re-opening. The 'info pci' output seems to indicate that resources were assigned to the device, so at some level the VM has seen and programmed resources of the 2nd function. Thanks. Re-open it because the nic of 2nd function didn't be found in the guest. Also, checked it with emulated virtio net, but didn't hit the issue. Following is the detailed message of emulated virtio on guest Guest info: [root@openshift-190 ~]# lspci 00:00.0 Host bridge: Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller 00:01.0 VGA compatible controller: Red Hat, Inc. QXL paravirtual graphic card (rev 04) 00:02.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02) 00:03.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02) 00:04.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02) 00:1f.0 ISA bridge: Intel Corporation 82801IB (ICH9) LPC Interface Controller (rev 02) 00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] (rev 02) 00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 02) 01:00.0 SCSI storage controller: Red Hat, Inc Virtio block device (rev 01) 02:00.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Upstream) (rev 02) 03:00.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01) 03:01.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01) 03:02.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01) 04:00.0 SCSI storage controller: Red Hat, Inc Virtio block device (rev 01) 05:00.0 SCSI storage controller: Red Hat, Inc Virtio block device (rev 01) 07:00.0 Ethernet controller: Red Hat, Inc Virtio network device (rev 01) 07:00.1 Ethernet controller: Red Hat, Inc Virtio network device (rev 01) [root@openshift-190 ~]# lspci -vvv -t -[0000:00]-+-00.0 Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller +-01.0 Red Hat, Inc. QXL paravirtual graphic card +-02.0-[01]----00.0 Red Hat, Inc Virtio block device +-03.0-[02-06]----00.0-[03-06]--+-00.0-[04]----00.0 Red Hat, Inc Virtio block device | +-01.0-[05]----00.0 Red Hat, Inc Virtio block device | \-02.0-[06]-- +-04.0-[07]--+-00.0 Red Hat, Inc Virtio network device | \-00.1 Red Hat, Inc Virtio network device +-1f.0 Intel Corporation 82801IB (ICH9) LPC Interface Controller +-1f.2 Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] \-1f.3 Intel Corporation 82801I (ICH9 Family) SMBus Controller [root@openshift-190 ~]# ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.66.144.137 netmask 255.255.252.0 broadcast 10.66.147.255 inet6 2620:52:0:4292:986a:6bff:fe6c:6d6e prefixlen 64 scopeid 0x0<global> inet6 fe80::986a:6bff:fe6c:6d6e prefixlen 64 scopeid 0x20<link> ether 9a:6a:6b:6c:6d:6e txqueuelen 1000 (Ethernet) RX packets 92 bytes 10536 (10.2 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 34 bytes 5782 (5.6 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.66.147.190 netmask 255.255.252.0 broadcast 10.66.147.255 inet6 2620:52:0:4292:986a:6bff:fe6c:6d6f prefixlen 64 scopeid 0x0<global> inet6 fe80::986a:6bff:fe6c:6d6f prefixlen 64 scopeid 0x20<link> ether 9a:6a:6b:6c:6d:6f txqueuelen 1000 (Ethernet) RX packets 197 bytes 27474 (26.8 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 106 bytes 17349 (16.9 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1 (Local Loopback) RX packets 460 bytes 41200 (40.2 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 460 bytes 41200 (40.2 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 virbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 inet 192.168.122.1 netmask 255.255.255.0 broadcast 0.0.0.0 ether 00:00:00:00:00:00 txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 virbr0-nic: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 ether 52:54:00:41:03:50 txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 Created attachment 1180015 [details]
dmesg for VF
Created attachment 1180016 [details]
dmesg of virtio-net
paste the command line of test: /usr/libexec/qemu-kvm \ -M q35 \ -cpu SandyBridge \ -nodefaults -rtc base=utc \ -m 4G \ -smp 2,sockets=2,cores=1,threads=1 \ -enable-kvm \ -name rhel7.3 \ -uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \ -smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \ -k en-us \ -serial unix:/tmp/serial0,server,nowait \ -boot menu=on \ -bios /usr/share/seabios/bios.bin \ -chardev file,path=/home/seabios.log,id=seabios \ -device isa-debugcon,chardev=seabios,iobase=0x402 \ -qmp tcp::8887,server,nowait \ -vga qxl \ -spice port=5932,disable-ticketing \ -device ioh3420,id=root.0,slot=1 \ -drive file=/home/q35-seabios.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop \ -device virtio-blk-pci,bus=root.0,drive=drive-virtio-disk0,id=virtio-disk0,disable-legacy=on,disable-modern=off,bootindex=1 \ -device ioh3420,id=root.1,slot=2 \ -device x3130-upstream,bus=root.1,id=upstream1 \ -device xio3130-downstream,bus=upstream1,id=downstream1,chassis=1 \ -device xio3130-downstream,bus=upstream1,id=downstream2,chassis=2 \ -device xio3130-downstream,bus=upstream1,id=downstream3,chassis=3 \ -drive file=/home/block1.qcow2,if=none,id=drive-virtio-disk1,format=qcow2,cache=none,werror=stop,rerror=stop \ -device virtio-blk-pci,bus=downstream1,drive=drive-virtio-disk1,id=virtio-disk1,disable-legacy=on,disable-modern=off \ -drive file=/home/block2.raw,if=none,id=drive-virtio-disk2,format=raw,cache=none,werror=stop,rerror=stop \ -device virtio-blk-pci,bus=downstream2,drive=drive-virtio-disk2,id=virtio-disk2,disable-legacy=on,disable-modern=off \ -device ioh3420,id=root.2,slot=3 \ -device vfio-pci,host=03:10.0,id=vf-10.0,bus=root.2,multifunction=on,addr=0x0.0 \ -device vfio-pci,host=03:10.2,id=vf-10.2,bus=root.2,addr=0x0.1 \ -monitor stdio \ Do you still see multifunction work with assigned devices in other contexts, for instance the devices placed on bus pcie.0 or a 440fx environment? Is the failure unique to q35 or is this a regression? (In reply to Alex Williamson from comment #10) > Do you still see multifunction work with assigned devices in other contexts, > for instance the devices placed on bus pcie.0 or a 440fx environment? Is > the failure unique to q35 or is this a regression? Alex, I think it's the failure unique to q35, because I tried it on 440fx machine type and didn't reproduce it /usr/libexec/qemu-kvm \ -M pc \ -cpu Nehalem \ -nodefaults -rtc base=utc \ -m 4G \ -smp 2,sockets=2,cores=1,threads=1 \ -enable-kvm \ -name rhel7.3 \ -uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \ -smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \ -k en-us \ -serial unix:/tmp/serial0,server,nowait \ -boot menu=on \ -bios /usr/share/seabios/bios.bin \ -chardev file,path=/home/seabios.log,id=seabios \ -device isa-debugcon,chardev=seabios,iobase=0x402 \ -qmp tcp::8887,server,nowait \ -vga qxl \ -spice port=5932,disable-ticketing \ -drive file=/home/q35-seabios.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop \ -device virtio-blk-pci,drive=drive-virtio-disk0,id=virtio-disk0,disable-legacy=on,disable-modern=off,bootindex=1 \ -device vfio-pci,host=03:10.0,id=vf-10.0,multifunction=on,addr=0xa.0 \ -device vfio-pci,host=03:10.2,id=vf-10.2,addr=0xa.1 \ -monitor stdio \ Also, I tried it with the devices placed on bus pcie.0 and didn't reproduce it. About regression, I think I need to re-install the machine and then try it, will update the result later Thanks Jing Zhao Didn't reproduce it with PF /usr/libexec/qemu-kvm \ -M q35 \ -cpu SandyBridge \ -nodefaults -rtc base=utc \ -m 4G \ -smp 2,sockets=2,cores=1,threads=1 \ -enable-kvm \ -name rhel7.3 \ -uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \ -smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \ -k en-us \ -serial unix:/tmp/serial0,server,nowait \ -boot menu=on \ -bios /usr/share/seabios/bios.bin \ -chardev file,path=/home/seabios.log,id=seabios \ -device isa-debugcon,chardev=seabios,iobase=0x402 \ -qmp tcp::8887,server,nowait \ -vga qxl \ -spice port=5932,disable-ticketing \ -device ioh3420,id=root.0,slot=1 \ -drive file=/home/q35-seabios.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop \ -device virtio-blk-pci,bus=root.0,drive=drive-virtio-disk0,id=virtio-disk0,disable-legacy=on,disable-modern=off,bootindex=1 \ -device ioh3420,id=root.1,slot=2 \ -device x3130-upstream,bus=root.1,id=upstream1 \ -device xio3130-downstream,bus=upstream1,id=downstream1,chassis=1 \ -device xio3130-downstream,bus=upstream1,id=downstream2,chassis=2 \ -device xio3130-downstream,bus=upstream1,id=downstream3,chassis=3 \ -drive file=/home/block1.qcow2,if=none,id=drive-virtio-disk1,format=qcow2,cache=none,werror=stop,rerror=stop \ -device virtio-blk-pci,bus=downstream1,drive=drive-virtio-disk1,id=virtio-disk1,disable-legacy=on,disable-modern=off \ -drive file=/home/block2.raw,if=none,id=drive-virtio-disk2,format=raw,cache=none,werror=stop,rerror=stop \ -device virtio-blk-pci,bus=downstream2,drive=drive-virtio-disk2,id=virtio-disk2,disable-legacy=on,disable-modern=off \ -device ioh3420,id=root.2,slot=3 \ -device vfio-pci,host=03:00.0,id=vf-00.0,bus=root.2,multifunction=on,addr=0x0.0 \ -device vfio-pci,host=03:00.1,id=vf-00.1,bus=root.2,addr=0x0.1 \ -monitor stdio \ Not a regression issue. Reproduced it with kernel-3.10.0-327.el7.x86_64 and qemu-kvm-rhev-2.3.0-31.el7_2.18.x86_64. /usr/libexec/qemu-kvm \ -M q35 \ -cpu SandyBridge \ -nodefaults -rtc base=utc \ -m 4G \ -smp 2,sockets=2,cores=1,threads=1 \ -enable-kvm \ -name rhel7.3 \ -uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \ -smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \ -k en-us \ -serial unix:/tmp/serial0,server,nowait \ -boot menu=on \ -bios /usr/share/seabios/bios.bin \ -chardev file,path=/home/seabios.log,id=seabios \ -device isa-debugcon,chardev=seabios,iobase=0x402 \ -qmp tcp::8887,server,nowait \ -vga qxl \ -spice port=5932,disable-ticketing \ -device ioh3420,id=root.0,slot=1 \ -drive file=/home/q35-seabios.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop \ -device virtio-blk-pci,bus=root.0,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 \ -device ioh3420,id=root.1,slot=2 \ -device x3130-upstream,bus=root.1,id=upstream1 \ -device xio3130-downstream,bus=upstream1,id=downstream1,chassis=1 \ -device xio3130-downstream,bus=upstream1,id=downstream2,chassis=2 \ -device xio3130-downstream,bus=upstream1,id=downstream3,chassis=3 \ -device ioh3420,id=root.2,slot=3 \ -device vfio-pci,host=03:10.0,id=vf-10.0,bus=root.2,multifunction=on,addr=0x0.0 \ -device vfio-pci,host=03:10.2,id=vf-10.2,bus=root.2,addr=0x0.1 \ -monitor stdio \ Patch posted upstream: http://lists.nongnu.org/archive/html/qemu-devel/2016-07/msg03521.html Fix included in qemu-kvm-rhev-2.6.0-15.el7 reproduced the issue with qemu-kvm-rhev-2.6.0-12.el7.x86_64, verified it with qemu-kvm-rhev-2.6.0-17.el7.x86_64. Following is the detailed verified steps 1.Boot guest with following cli: /usr/libexec/qemu-kvm \ -M q35 \ -cpu SandyBridge \ -nodefaults -rtc base=utc \ -m 4G \ -smp 2,sockets=2,cores=1,threads=1 \ -enable-kvm \ -name rhel7.3 \ -uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \ -smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \ -k en-us \ -serial unix:/tmp/console,server,nowait \ -boot menu=on \ -bios /usr/share/seabios/bios.bin \ -chardev file,path=/home/seabios.log,id=seabios \ -device isa-debugcon,chardev=seabios,iobase=0x402 \ -qmp tcp::8887,server,nowait \ -vga qxl \ -spice port=5932,disable-ticketing \ -device ioh3420,id=root.0,slot=1 \ -drive file=/home/q35-seabios.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop \ -device virtio-blk-pci,bus=root.0,drive=drive-virtio-disk0,id=virtio-disk0,disable-legacy=on,disable-modern=off,bootindex=1 \ -device ioh3420,id=root.1,slot=2 \ -device x3130-upstream,bus=root.1,id=upstream1 \ -device xio3130-downstream,bus=upstream1,id=downstream1,chassis=1 \ -device xio3130-downstream,bus=upstream1,id=downstream2,chassis=2 \ -device xio3130-downstream,bus=upstream1,id=downstream3,chassis=3 \ -drive file=/home/block1.qcow2,if=none,id=drive-virtio-disk1,format=qcow2,cache=none,werror=stop,rerror=stop \ -device virtio-blk-pci,bus=downstream1,drive=drive-virtio-disk1,id=virtio-disk1,disable-legacy=on,disable-modern=off \ -drive file=/home/block2.raw,if=none,id=drive-virtio-disk2,format=raw,cache=none,werror=stop,rerror=stop \ -device virtio-blk-pci,bus=downstream2,drive=drive-virtio-disk2,id=virtio-disk2,disable-legacy=on,disable-modern=off \ -device ioh3420,id=root.2,slot=3 \ -device vfio-pci,host=03:10.0,id=vf-10.0,bus=root.2,multifunction=on,addr=0x0.0 \ -device vfio-pci,host=03:10.2,id=vf-10.2,bus=root.2,addr=0x0.1 \ -monitor stdio \ 2.check the info through hmp (qemu) info pci Bus 7, device 0, function 0: Ethernet controller: PCI device 8086:10ca BAR0: 64 bit memory at 0xf8600000 [0xf8603fff]. BAR3: 64 bit memory at 0xf8604000 [0xf8607fff]. id "vf-10.0" Bus 7, device 0, function 1: Ethernet controller: PCI device 8086:10ca BAR0: 64 bit memory at 0xf8608000 [0xf860bfff]. BAR3: 64 bit memory at 0xf860c000 [0xf860ffff]. id "vf-10.2" 3.check the nic info in guest [root@localhost ~]# lspci -vvv -t lspci -vvv -t -[0000:00]-+-00.0 Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller +-01.0 Red Hat, Inc. QXL paravirtual graphic card +-02.0-[01]----00.0 Red Hat, Inc Virtio block device +-03.0-[02-06]----00.0-[03-06]--+-00.0-[04]----00.0 Red Hat, Inc Virtio block device | +-01.0-[05]----00.0 Red Hat, Inc Virtio block device | \-02.0-[06]-- +-04.0-[07]--+-00.0 Intel Corporation 82576 Virtual Function | \-00.1 Intel Corporation 82576 Virtual Function +-1f.0 Intel Corporation 82801IB (ICH9) LPC Interface Controller +-1f.2 Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] \-1f.3 Intel Corporation 82801I (ICH9 Family) SMBus Controller [root@localhost ~]# ifconfig ifconfig ens3f0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 ether 6a:36:21:1e:d8:75 txqueuelen 1000 (Ethernet) RX packets 22 bytes 1346 (1.3 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 ens3f1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 ether ba:3e:a9:a2:d5:8d txqueuelen 1000 (Ethernet) RX packets 22 bytes 1346 (1.3 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1 (Local Loopback) RX packets 364 bytes 32344 (31.5 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 364 bytes 32344 (31.5 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 virbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 inet 192.168.122.1 netmask 255.255.255.0 broadcast 0.0.0.0 ether 00:00:00:00:00:00 txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 [root@unused home]# ping 10.66.4.211 -I ens3f0 PING 10.66.4.211 (10.66.4.211) from 10.66.152.180 ens3f0: 56(84) bytes of data. 64 bytes from 10.66.4.211: icmp_seq=1 ttl=61 time=0.603 ms 64 bytes from 10.66.4.211: icmp_seq=2 ttl=61 time=0.593 ms ^C --- 10.66.4.211 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms rtt min/avg/max/mdev = 0.593/0.598/0.603/0.005 ms [root@unused home]# ping 10.66.4.211 -I ens3f1 PING 10.66.4.211 (10.66.4.211) from 10.66.152.179 ens3f1: 56(84) bytes of data. 64 bytes from 10.66.4.211: icmp_seq=1 ttl=61 time=0.526 ms 64 bytes from 10.66.4.211: icmp_seq=2 ttl=61 time=0.573 ms 64 bytes from 10.66.4.211: icmp_seq=3 ttl=61 time=0.560 ms 64 bytes from 10.66.4.211: icmp_seq=4 ttl=61 time=0.566 ms Thanks Jing Zhao Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-2673.html |