Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1356376

Summary: [Q35] Nic which passthrough from host didn't be found in guest when enable multifunction
Product: Red Hat Enterprise Linux 7 Reporter: jingzhao <jinzhao>
Component: qemu-kvm-rhevAssignee: Alex Williamson <alex.williamson>
Status: CLOSED ERRATA QA Contact: jingzhao <jinzhao>
Severity: high Docs Contact:
Priority: high    
Version: 7.3CC: ailan, alex.williamson, chayang, jinzhao, juzhang, knoel, marcel, mrezanin, virt-maint, yfu, yiwei
Target Milestone: rcKeywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-rhev-2.6.0-15.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-07 21:24:28 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
dmesg for VF
none
dmesg of virtio-net none

Description jingzhao 2016-07-14 02:59:02 UTC 
Description of problem:
Nic which passthrough from host didn't be found in guest when enable multifunction

Version-Release number of selected component (if applicable):
host kernel:3.10.0-464.el7.x86_64
qemu-kvm-rhev-2.6.0-12.el7.x86_64
seabios-bin-1.9.1-4.el7.noarch
guest kernel:3.10.0-456.el7.x86_64

How reproducible:
3/3

Steps to Reproduce:
1. Boot guest with following cli:
/usr/libexec/qemu-kvm \
-M q35 \
-cpu SandyBridge \
-nodefaults -rtc base=utc \
-m 4G \
-smp 2,sockets=2,cores=1,threads=1 \
-enable-kvm \
-name rhel7.3 \
-uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
-smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \
-k en-us \
-serial unix:/tmp/serial0,server,nowait \
-boot menu=on \
-bios /usr/share/seabios/bios.bin \
-chardev file,path=/home/seabios.log,id=seabios \
-device isa-debugcon,chardev=seabios,iobase=0x402 \
-qmp tcp::8887,server,nowait \
-vga qxl \
-spice port=5932,disable-ticketing \
-device ioh3420,id=root.0,slot=1 \
-drive file=/home/q35-seabios.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=root.0,drive=drive-virtio-disk0,id=virtio-disk0,disable-legacy=on,disable-modern=off,bootindex=1 \
-device ioh3420,id=root.1,slot=2 \
-device x3130-upstream,bus=root.1,id=upstream1 \
-device xio3130-downstream,bus=upstream1,id=downstream1,chassis=1 \
-device xio3130-downstream,bus=upstream1,id=downstream2,chassis=2 \
-device xio3130-downstream,bus=upstream1,id=downstream3,chassis=3 \
-drive file=/home/block1.qcow2,if=none,id=drive-virtio-disk1,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=downstream1,drive=drive-virtio-disk1,id=virtio-disk1,disable-legacy=on,disable-modern=off \
-drive file=/home/block2.raw,if=none,id=drive-virtio-disk2,format=raw,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=downstream2,drive=drive-virtio-disk2,id=virtio-disk2,disable-legacy=on,disable-modern=off \
-device virtio-net-pci,bus=downstream3,netdev=tap10,mac=9a:6a:6b:6c:6d:6e -netdev tap,id=tap10 \
-device ioh3420,id=root.2,slot=3 \
-device vfio-pci,host=0000:03:10.0,bus=root.2,multifunction=on,addr=0xa.0,id=pf1 \
-device vfio-pci,host=0000:03:10.2,bus=root.2,addr=0xa.1,id=pf2 \
-monitor stdio \

2. Check the Nic which passthrough from host on hmp
(qemu) info pci
  Bus  0, device   0, function 0:
    Host bridge: PCI device 8086:29c0
      id ""
  Bus  0, device   1, function 0:
    VGA controller: PCI device 1b36:0100
      IRQ 10.
      BAR0: 32 bit memory at 0xf0000000 [0xf3ffffff].
      BAR1: 32 bit memory at 0xf4000000 [0xf7ffffff].
      BAR2: 32 bit memory at 0xf8a10000 [0xf8a11fff].
      BAR3: I/O at 0xd040 [0xd05f].
      BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe].
      id ""
  Bus  0, device   2, function 0:
    PCI bridge: PCI device 8086:3420
      BUS 0.
      secondary bus 1.
      subordinate bus 1.
      IO range [0x1000, 0x1fff]
      memory range [0xf8800000, 0xf89fffff]
      prefetchable memory range [0xfe000000, 0xfe7fffff]
      id "root.0"
  Bus  1, device   0, function 0:
    SCSI controller: PCI device 1af4:1042
      IRQ 11.
      BAR1: 32 bit memory at 0xf8800000 [0xf8800fff].
      BAR4: 64 bit prefetchable memory at 0xfe000000 [0xfe7fffff].
      id "virtio-disk0"
  Bus  0, device   3, function 0:
    PCI bridge: PCI device 8086:3420
      BUS 0.
      secondary bus 2.
      subordinate bus 6.
      IO range [0xc000, 0xcfff]
      memory range [0xf8000000, 0xf85fffff]
      prefetchable memory range [0xfc800000, 0xfdffffff]
      id "root.1"
  Bus  2, device   0, function 0:
    PCI bridge: PCI device 104c:8232
      BUS 2.
      secondary bus 3.
      subordinate bus 6.
      IO range [0xc000, 0xcfff]
      memory range [0xf8000000, 0xf85fffff]
      prefetchable memory range [0xfc800000, 0xfdffffff]
      id "upstream1"
  Bus  3, device   0, function 0:
    PCI bridge: PCI device 104c:8233
      BUS 3.
      secondary bus 4.
      subordinate bus 4.
      IO range [0xf000, 0x0fff]
      memory range [0xf8400000, 0xf85fffff]
      prefetchable memory range [0xfd000000, 0xfd7fffff]
      id "downstream1"
  Bus  4, device   0, function 0:
    SCSI controller: PCI device 1af4:1042
      IRQ 11.
      BAR1: 32 bit memory at 0xf8400000 [0xf8400fff].
      BAR4: 64 bit prefetchable memory at 0xfd000000 [0xfd7fffff].
      id "virtio-disk1"
  Bus  3, device   1, function 0:
    PCI bridge: PCI device 104c:8233
      BUS 3.
      secondary bus 5.
      subordinate bus 5.
      IO range [0xf000, 0x0fff]
      memory range [0xf8200000, 0xf83fffff]
      prefetchable memory range [0xfc800000, 0xfcffffff]
      id "downstream2"
  Bus  5, device   0, function 0:
    SCSI controller: PCI device 1af4:1042
      IRQ 10.
      BAR1: 32 bit memory at 0xf8200000 [0xf8200fff].
      BAR4: 64 bit prefetchable memory at 0xfc800000 [0xfcffffff].
      id "virtio-disk2"
  Bus  3, device   2, function 0:
    PCI bridge: PCI device 104c:8233
      BUS 3.
      secondary bus 6.
      subordinate bus 6.
      IO range [0xc000, 0xcfff]
      memory range [0xf8000000, 0xf81fffff]
      prefetchable memory range [0xfd800000, 0xfd9fffff]
      id "downstream3"
  Bus  6, device   0, function 0:
    Ethernet controller: PCI device 1af4:1000
      IRQ 10.
      BAR0: I/O at 0xc000 [0xc01f].
      BAR1: 32 bit memory at 0xf8040000 [0xf8040fff].
      BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe].
      id ""
  Bus  0, device   4, function 0:
    PCI bridge: PCI device 8086:3420
      BUS 0.
      secondary bus 7.
      subordinate bus 7.
      IO range [0x2000, 0x2fff]
      memory range [0xf8600000, 0xf87fffff]
      prefetchable memory range [0xfe800000, 0xfe9fffff]
      id "root.2"
  Bus  0, device  31, function 0:
    ISA bridge: PCI device 8086:2918
      id ""
  Bus  0, device  31, function 2:
    SATA controller: PCI device 8086:2922
      IRQ 10.
      BAR4: I/O at 0xd060 [0xd07f].
      BAR5: 32 bit memory at 0xf8a12000 [0xf8a12fff].
      id ""
  Bus  0, device  31, function 3:
    SMBus: PCI device 8086:2930
      IRQ 10.
      BAR4: I/O at 0x0700 [0x073f].
      id ""
(qemu) info qtree
..........
  dev: q35-pcihost, id ""
    MCFG = 2952790016 (0xb0000000)
    pci-hole64-size = 0 (0 B)
    short_root_bus = 0 (0x0)
    bus: pcie.0
      type PCIE
      dev: ioh3420, id "root.2"
        power_controller_present = true
        chassis = 0 (0x0)
        slot = 3 (0x3)
        port = 0 (0x0)
        aer_log_max = 8 (0x8)
        addr = 04.0
        romfile = ""
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        class PCI bridge, addr 00:04.0, pci id 8086:3420 (sub 0000:0000)
        bus: root.2
          type PCIE
........
3. Check the info in guest
[root@unused ~]# lspci 
00:00.0 Host bridge: Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
00:01.0 VGA compatible controller: Red Hat, Inc. QXL paravirtual graphic card (rev 04)
00:02.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
00:03.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
00:04.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
00:1f.0 ISA bridge: Intel Corporation 82801IB (ICH9) LPC Interface Controller (rev 02)
00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] (rev 02)
00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 02)
01:00.0 SCSI storage controller: Red Hat, Inc Virtio block device (rev 01)
02:00.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Upstream) (rev 02)
03:00.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01)
03:01.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01)
03:02.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01)
04:00.0 SCSI storage controller: Red Hat, Inc Virtio block device (rev 01)
05:00.0 SCSI storage controller: Red Hat, Inc Virtio block device (rev 01)
06:00.0 Ethernet controller: Red Hat, Inc Virtio network device
[root@unused ~]# lspci -vvv -t
-[0000:00]-+-00.0  Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
           +-01.0  Red Hat, Inc. QXL paravirtual graphic card
           +-02.0-[01]----00.0  Red Hat, Inc Virtio block device
           +-03.0-[02-06]----00.0-[03-06]--+-00.0-[04]----00.0  Red Hat, Inc Virtio block device
           |                               +-01.0-[05]----00.0  Red Hat, Inc Virtio block device
           |                               \-02.0-[06]----00.0  Red Hat, Inc Virtio network device
           +-04.0-[07]--
           +-1f.0  Intel Corporation 82801IB (ICH9) LPC Interface Controller
           +-1f.2  Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode]
           \-1f.3  Intel Corporation 82801I (ICH9 Family) SMBus Controller
[root@unused ~]# ifcnofifg
bash: ifcnofifg: command not found...
[root@unused ~]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.66.144.137  netmask 255.255.252.0  broadcast 10.66.147.255
        inet6 2620:52:0:4292:986a:6bff:fe6c:6d6e  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::986a:6bff:fe6c:6d6e  prefixlen 64  scopeid 0x20<link>
        ether 9a:6a:6b:6c:6d:6e  txqueuelen 1000  (Ethernet)
        RX packets 215  bytes 24922 (24.3 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 130  bytes 20004 (19.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 128  bytes 11780 (11.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 128  bytes 11780 (11.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 0.0.0.0
        ether 00:00:00:00:00:00  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0-nic: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 52:54:00:41:03:50  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0


Actual results:
Nic didn't be found in guest

Expected results:
Nic can be found in guest

Additional info:
Hit the same issue after attach to downstream port
Comment 1 jingzhao 2016-07-14 03:00:37 UTC 
The Nic info of host:
[root@hp-z800-01 home]# lspci |grep Ether
01:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5764M Gigabit Ethernet PCIe (rev 10)
02:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5764M Gigabit Ethernet PCIe (rev 10)
03:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
03:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
03:10.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
03:10.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
03:10.4 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
03:10.6 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
Comment 3 Alex Williamson 2016-07-14 13:04:53 UTC 
For any device behind a downstream port, ie. root port or downstream switch port, the only valid slot address is 0.  Using 0xa is very much non-standard.  If this cannot be reproduced with addr=0x0.0 and addr=0x0.1, then there's no bug here.  I'm going to assume that case, re-open otherwise.
Comment 4 jingzhao 2016-07-15 01:26:47 UTC 
Hi Alex

  Nic really can be found in guest, but I hit another problem, and I want to confrim with you

  Only 1 Nic can be found in guest, but I think we can found 2 nic normally, and  actually we can found 2 nic in hmp

Following is the info of guest:
[root@unused ~]# lspci 
00:00.0 Host bridge: Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
00:01.0 VGA compatible controller: Red Hat, Inc. QXL paravirtual graphic card (rev 04)
00:02.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
00:03.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
00:04.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
00:1f.0 ISA bridge: Intel Corporation 82801IB (ICH9) LPC Interface Controller (rev 02)
00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] (rev 02)
00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 02)
01:00.0 SCSI storage controller: Red Hat, Inc Virtio block device (rev 01)
02:00.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Upstream) (rev 02)
03:00.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01)
03:01.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01)
03:02.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01)
04:00.0 SCSI storage controller: Red Hat, Inc Virtio block device (rev 01)
05:00.0 SCSI storage controller: Red Hat, Inc Virtio block device (rev 01)
07:00.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
[root@unused ~]# lspci -vvv -t
-[0000:00]-+-00.0  Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
           +-01.0  Red Hat, Inc. QXL paravirtual graphic card
           +-02.0-[01]----00.0  Red Hat, Inc Virtio block device
           +-03.0-[02-06]----00.0-[03-06]--+-00.0-[04]----00.0  Red Hat, Inc Virtio block device
           |                               +-01.0-[05]----00.0  Red Hat, Inc Virtio block device
           |                               \-02.0-[06]--
           +-04.0-[07]----00.0  Intel Corporation 82576 Virtual Function
           +-1f.0  Intel Corporation 82801IB (ICH9) LPC Interface Controller
           +-1f.2  Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode]
           \-1f.3  Intel Corporation 82801I (ICH9 Family) SMBus Controller
[root@unused ~]# ifconfig
ens3f0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.66.152.155  netmask 255.255.255.0  broadcast 10.66.152.255
        inet6 2620:52:0:4298:8c9f:d0ff:fe13:801d  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::8c9f:d0ff:fe13:801d  prefixlen 64  scopeid 0x20<link>
        ether 8e:9f:d0:13:80:1d  txqueuelen 1000  (Ethernet)
        RX packets 159  bytes 21199 (20.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 149  bytes 22478 (21.9 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 212  bytes 19172 (18.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 212  bytes 19172 (18.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 0.0.0.0
        ether 00:00:00:00:00:00  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0-nic: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 52:54:00:41:03:50  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0


Following is the info of hmp:

(qemu) info qtree
bus: main-system-bus
  type System
  dev: kvm-ioapic, id ""
    gpio-in "" 24
    gsi_base = 0 (0x0)
    mmio 00000000fec00000/0000000000001000
  dev: q35-pcihost, id ""
    MCFG = 2952790016 (0xb0000000)
    pci-hole64-size = 0 (0 B)
    short_root_bus = 0 (0x0)
    bus: pcie.0
      type PCIE
      dev: ioh3420, id "root.2"
        power_controller_present = true
        chassis = 0 (0x0)
        slot = 3 (0x3)
        port = 0 (0x0)
        aer_log_max = 8 (0x8)
        addr = 04.0
        romfile = ""
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        class PCI bridge, addr 00:04.0, pci id 8086:3420 (sub 0000:0000)
        bus: root.2
          type PCIE
          dev: vfio-pci, id "vf-10.2"
            host = "0000:03:10.2"
            sysfsdev = "/sys/bus/pci/devices/0000:03:10.2"
            x-intx-mmap-timeout-ms = 1100 (0x44c)
            x-vga = false
            x-req = true
            x-no-mmap = false
            x-no-kvm-intx = false
            x-no-kvm-msi = false
            x-no-kvm-msix = false
            x-pci-vendor-id = 32902 (0x8086)
            x-pci-device-id = 4298 (0x10ca)
            x-pci-sub-vendor-id = 4294967295 (0xffffffff)
            x-pci-sub-device-id = 4294967295 (0xffffffff)
            addr = 00.1
            romfile = ""
            rombar = 1 (0x1)
            multifunction = false
            command_serr_enable = true
            class Ethernet controller, addr 07:00.1, pci id 8086:10ca (sub 8086:a03c)
            bar 0: mem at 0xf8608000 [0xf860bfff]
            bar 3: mem at 0xf860c000 [0xf860ffff]
          dev: vfio-pci, id "vf-10.0"
            host = "0000:03:10.0"
            sysfsdev = "/sys/bus/pci/devices/0000:03:10.0"
            x-intx-mmap-timeout-ms = 1100 (0x44c)
            x-vga = false
            x-req = true
            x-no-mmap = false
            x-no-kvm-intx = false
            x-no-kvm-msi = false
            x-no-kvm-msix = false
            x-pci-vendor-id = 32902 (0x8086)
            x-pci-device-id = 4298 (0x10ca)
            x-pci-sub-vendor-id = 4294967295 (0xffffffff)
            x-pci-sub-device-id = 4294967295 (0xffffffff)
            addr = 00.0
            romfile = ""
            rombar = 1 (0x1)
            multifunction = true
            command_serr_enable = true
            class Ethernet controller, addr 07:00.0, pci id 8086:10ca (sub 8086:a03c)
            bar 0: mem at 0xf8600000 [0xf8603fff]
            bar 3: mem at 0xf8604000 [0xf8607fff]
..................

(qemu) info pci
  Bus  0, device   0, function 0:
    Host bridge: PCI device 8086:29c0
      id ""
  Bus  0, device   1, function 0:
    VGA controller: PCI device 1b36:0100
      IRQ 10.
      BAR0: 32 bit memory at 0xf0000000 [0xf3ffffff].
      BAR1: 32 bit memory at 0xf4000000 [0xf7ffffff].
      BAR2: 32 bit memory at 0xf8a10000 [0xf8a11fff].
      BAR3: I/O at 0xc040 [0xc05f].
      BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe].
      id ""
  Bus  0, device   2, function 0:
    PCI bridge: PCI device 8086:3420
      BUS 0.
      secondary bus 1.
      subordinate bus 1.
      IO range [0x1000, 0x1fff]
      memory range [0xf8800000, 0xf89fffff]
      prefetchable memory range [0xfe000000, 0xfe7fffff]
      id "root.0"
  Bus  1, device   0, function 0:
    SCSI controller: PCI device 1af4:1042
      IRQ 11.
      BAR1: 32 bit memory at 0xf8800000 [0xf8800fff].
      BAR4: 64 bit prefetchable memory at 0xfe000000 [0xfe7fffff].
      id "virtio-disk0"
  Bus  0, device   3, function 0:
    PCI bridge: PCI device 8086:3420
      BUS 0.
      secondary bus 2.
      subordinate bus 6.
      IO range [0x2000, 0x4fff]
      memory range [0xf8000000, 0xf85fffff]
      prefetchable memory range [0xfc800000, 0xfdffffff]
      id "root.1"
  Bus  2, device   0, function 0:
    PCI bridge: PCI device 104c:8232
      BUS 2.
      secondary bus 3.
      subordinate bus 6.
      IO range [0x2000, 0x4fff]
      memory range [0xf8000000, 0xf85fffff]
      prefetchable memory range [0xfc800000, 0xfdffffff]
      id "upstream1"
  Bus  3, device   0, function 0:
    PCI bridge: PCI device 104c:8233
      BUS 3.
      secondary bus 4.
      subordinate bus 4.
      IO range [0x2000, 0x2fff]
      memory range [0xf8400000, 0xf85fffff]
      prefetchable memory range [0xfd000000, 0xfd7fffff]
      id "downstream1"
  Bus  4, device   0, function 0:
    SCSI controller: PCI device 1af4:1042
      IRQ 11.
      BAR1: 32 bit memory at 0xf8400000 [0xf8400fff].
      BAR4: 64 bit prefetchable memory at 0xfd000000 [0xfd7fffff].
      id "virtio-disk1"
  Bus  3, device   1, function 0:
    PCI bridge: PCI device 104c:8233
      BUS 3.
      secondary bus 5.
      subordinate bus 5.
      IO range [0x3000, 0x3fff]
      memory range [0xf8200000, 0xf83fffff]
      prefetchable memory range [0xfc800000, 0xfcffffff]
      id "downstream2"
  Bus  5, device   0, function 0:
    SCSI controller: PCI device 1af4:1042
      IRQ 10.
      BAR1: 32 bit memory at 0xf8200000 [0xf8200fff].
      BAR4: 64 bit prefetchable memory at 0xfc800000 [0xfcffffff].
      id "virtio-disk2"
  Bus  3, device   2, function 0:
    PCI bridge: PCI device 104c:8233
      BUS 3.
      secondary bus 6.
      subordinate bus 6.
      IO range [0x4000, 0x4fff]
      memory range [0xf8000000, 0xf81fffff]
      prefetchable memory range [0xfd800000, 0xfd9fffff]
      id "downstream3"
  Bus  0, device   4, function 0:
    PCI bridge: PCI device 8086:3420
      BUS 0.
      secondary bus 7.
      subordinate bus 7.
      IO range [0x5000, 0x5fff]
      memory range [0xf8600000, 0xf87fffff]
      prefetchable memory range [0xfe800000, 0xfe9fffff]
      id "root.2"
  Bus  7, device   0, function 0:
    Ethernet controller: PCI device 8086:10ca
      BAR0: 64 bit memory at 0xf8600000 [0xf8603fff].
      BAR3: 64 bit memory at 0xf8604000 [0xf8607fff].
      id "vf-10.0"
  Bus  7, device   0, function 1:
    Ethernet controller: PCI device 8086:10ca
      BAR0: 64 bit memory at 0xf8608000 [0xf860bfff].
      BAR3: 64 bit memory at 0xf860c000 [0xf860ffff].
      id "vf-10.2"
  Bus  0, device  31, function 0:
    ISA bridge: PCI device 8086:2918
      id ""
  Bus  0, device  31, function 2:
    SATA controller: PCI device 8086:2922
      IRQ 10.
      BAR4: I/O at 0xc060 [0xc07f].
      BAR5: 32 bit memory at 0xf8a12000 [0xf8a12fff].
      id ""
  Bus  0, device  31, function 3:
    SMBus: PCI device 8086:2930
      IRQ 10.
      BAR4: I/O at 0x0700 [0x073f].
      id ""

Could you help to confirm it, I will reopen it it's a issue

Thanks
Jing Zhao
Comment 5 Alex Williamson 2016-07-15 02:26:11 UTC 
It looks like the 2nd function should have been found, can it be reproduced with a non-assigned device, like an emulated e1000?  Include the complete new command line and guest dmesg when re-opening.  The 'info pci' output seems to indicate that resources were assigned to the device, so at some level the VM has seen and programmed resources of the 2nd function.  Thanks.
Comment 6 jingzhao 2016-07-15 03:21:14 UTC 
Re-open it because the nic of 2nd function didn't be found in the guest.

Also, checked it with emulated virtio net, but didn't hit the issue. Following is the detailed message of emulated virtio on guest

Guest info:
[root@openshift-190 ~]# lspci 
00:00.0 Host bridge: Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
00:01.0 VGA compatible controller: Red Hat, Inc. QXL paravirtual graphic card (rev 04)
00:02.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
00:03.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
00:04.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
00:1f.0 ISA bridge: Intel Corporation 82801IB (ICH9) LPC Interface Controller (rev 02)
00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] (rev 02)
00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 02)
01:00.0 SCSI storage controller: Red Hat, Inc Virtio block device (rev 01)
02:00.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Upstream) (rev 02)
03:00.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01)
03:01.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01)
03:02.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01)
04:00.0 SCSI storage controller: Red Hat, Inc Virtio block device (rev 01)
05:00.0 SCSI storage controller: Red Hat, Inc Virtio block device (rev 01)
07:00.0 Ethernet controller: Red Hat, Inc Virtio network device (rev 01)
07:00.1 Ethernet controller: Red Hat, Inc Virtio network device (rev 01)
[root@openshift-190 ~]# lspci -vvv -t
-[0000:00]-+-00.0  Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
           +-01.0  Red Hat, Inc. QXL paravirtual graphic card
           +-02.0-[01]----00.0  Red Hat, Inc Virtio block device
           +-03.0-[02-06]----00.0-[03-06]--+-00.0-[04]----00.0  Red Hat, Inc Virtio block device
           |                               +-01.0-[05]----00.0  Red Hat, Inc Virtio block device
           |                               \-02.0-[06]--
           +-04.0-[07]--+-00.0  Red Hat, Inc Virtio network device
           |            \-00.1  Red Hat, Inc Virtio network device
           +-1f.0  Intel Corporation 82801IB (ICH9) LPC Interface Controller
           +-1f.2  Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode]
           \-1f.3  Intel Corporation 82801I (ICH9 Family) SMBus Controller
[root@openshift-190 ~]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.66.144.137  netmask 255.255.252.0  broadcast 10.66.147.255
        inet6 2620:52:0:4292:986a:6bff:fe6c:6d6e  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::986a:6bff:fe6c:6d6e  prefixlen 64  scopeid 0x20<link>
        ether 9a:6a:6b:6c:6d:6e  txqueuelen 1000  (Ethernet)
        RX packets 92  bytes 10536 (10.2 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 34  bytes 5782 (5.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.66.147.190  netmask 255.255.252.0  broadcast 10.66.147.255
        inet6 2620:52:0:4292:986a:6bff:fe6c:6d6f  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::986a:6bff:fe6c:6d6f  prefixlen 64  scopeid 0x20<link>
        ether 9a:6a:6b:6c:6d:6f  txqueuelen 1000  (Ethernet)
        RX packets 197  bytes 27474 (26.8 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 106  bytes 17349 (16.9 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 460  bytes 41200 (40.2 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 460  bytes 41200 (40.2 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 0.0.0.0
        ether 00:00:00:00:00:00  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0-nic: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 52:54:00:41:03:50  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
Comment 7 jingzhao 2016-07-15 03:22:31 UTC 
Created attachment 1180015 [details]
dmesg for VF
Comment 8 jingzhao 2016-07-15 03:23:22 UTC 
Created attachment 1180016 [details]
dmesg of virtio-net
Comment 9 jingzhao 2016-07-15 03:24:48 UTC 
paste the command line of test:

/usr/libexec/qemu-kvm \
-M q35 \
-cpu SandyBridge \
-nodefaults -rtc base=utc \
-m 4G \
-smp 2,sockets=2,cores=1,threads=1 \
-enable-kvm \
-name rhel7.3 \
-uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
-smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \
-k en-us \
-serial unix:/tmp/serial0,server,nowait \
-boot menu=on \
-bios /usr/share/seabios/bios.bin \
-chardev file,path=/home/seabios.log,id=seabios \
-device isa-debugcon,chardev=seabios,iobase=0x402 \
-qmp tcp::8887,server,nowait \
-vga qxl \
-spice port=5932,disable-ticketing \
-device ioh3420,id=root.0,slot=1 \
-drive file=/home/q35-seabios.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=root.0,drive=drive-virtio-disk0,id=virtio-disk0,disable-legacy=on,disable-modern=off,bootindex=1 \
-device ioh3420,id=root.1,slot=2 \
-device x3130-upstream,bus=root.1,id=upstream1 \
-device xio3130-downstream,bus=upstream1,id=downstream1,chassis=1 \
-device xio3130-downstream,bus=upstream1,id=downstream2,chassis=2 \
-device xio3130-downstream,bus=upstream1,id=downstream3,chassis=3 \
-drive file=/home/block1.qcow2,if=none,id=drive-virtio-disk1,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=downstream1,drive=drive-virtio-disk1,id=virtio-disk1,disable-legacy=on,disable-modern=off \
-drive file=/home/block2.raw,if=none,id=drive-virtio-disk2,format=raw,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=downstream2,drive=drive-virtio-disk2,id=virtio-disk2,disable-legacy=on,disable-modern=off \
-device ioh3420,id=root.2,slot=3 \
-device vfio-pci,host=03:10.0,id=vf-10.0,bus=root.2,multifunction=on,addr=0x0.0 \
-device vfio-pci,host=03:10.2,id=vf-10.2,bus=root.2,addr=0x0.1 \
-monitor stdio \
Comment 10 Alex Williamson 2016-07-15 03:40:00 UTC 
Do you still see multifunction work with assigned devices in other contexts, for instance the devices placed on bus pcie.0 or a 440fx environment?  Is the failure unique to q35 or is this a regression?
Comment 11 jingzhao 2016-07-15 07:07:31 UTC 
(In reply to Alex Williamson from comment #10)
> Do you still see multifunction work with assigned devices in other contexts,
> for instance the devices placed on bus pcie.0 or a 440fx environment?  Is
> the failure unique to q35 or is this a regression?

Alex, 

  I think it's the failure unique to q35, because I tried it on 440fx machine type and didn't reproduce it

/usr/libexec/qemu-kvm \
-M pc \
-cpu Nehalem \
-nodefaults -rtc base=utc \
-m 4G \
-smp 2,sockets=2,cores=1,threads=1 \
-enable-kvm \
-name rhel7.3 \
-uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
-smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \
-k en-us \
-serial unix:/tmp/serial0,server,nowait \
-boot menu=on \
-bios /usr/share/seabios/bios.bin \
-chardev file,path=/home/seabios.log,id=seabios \
-device isa-debugcon,chardev=seabios,iobase=0x402 \
-qmp tcp::8887,server,nowait \
-vga qxl \
-spice port=5932,disable-ticketing \
-drive file=/home/q35-seabios.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,drive=drive-virtio-disk0,id=virtio-disk0,disable-legacy=on,disable-modern=off,bootindex=1 \
-device vfio-pci,host=03:10.0,id=vf-10.0,multifunction=on,addr=0xa.0 \
-device vfio-pci,host=03:10.2,id=vf-10.2,addr=0xa.1 \
-monitor stdio \


Also, I tried it with the devices placed on bus pcie.0 and didn't reproduce it.

About regression, I think I need to re-install the machine and then try it, will update the result later

Thanks
Jing Zhao
Comment 12 jingzhao 2016-07-15 07:46:03 UTC 
Didn't reproduce it with PF
/usr/libexec/qemu-kvm \
-M q35 \
-cpu SandyBridge \
-nodefaults -rtc base=utc \
-m 4G \
-smp 2,sockets=2,cores=1,threads=1 \
-enable-kvm \
-name rhel7.3 \
-uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
-smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \
-k en-us \
-serial unix:/tmp/serial0,server,nowait \
-boot menu=on \
-bios /usr/share/seabios/bios.bin \
-chardev file,path=/home/seabios.log,id=seabios \
-device isa-debugcon,chardev=seabios,iobase=0x402 \
-qmp tcp::8887,server,nowait \
-vga qxl \
-spice port=5932,disable-ticketing \
-device ioh3420,id=root.0,slot=1 \
-drive file=/home/q35-seabios.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=root.0,drive=drive-virtio-disk0,id=virtio-disk0,disable-legacy=on,disable-modern=off,bootindex=1 \
-device ioh3420,id=root.1,slot=2 \
-device x3130-upstream,bus=root.1,id=upstream1 \
-device xio3130-downstream,bus=upstream1,id=downstream1,chassis=1 \
-device xio3130-downstream,bus=upstream1,id=downstream2,chassis=2 \
-device xio3130-downstream,bus=upstream1,id=downstream3,chassis=3 \
-drive file=/home/block1.qcow2,if=none,id=drive-virtio-disk1,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=downstream1,drive=drive-virtio-disk1,id=virtio-disk1,disable-legacy=on,disable-modern=off \
-drive file=/home/block2.raw,if=none,id=drive-virtio-disk2,format=raw,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=downstream2,drive=drive-virtio-disk2,id=virtio-disk2,disable-legacy=on,disable-modern=off \
-device ioh3420,id=root.2,slot=3 \
-device vfio-pci,host=03:00.0,id=vf-00.0,bus=root.2,multifunction=on,addr=0x0.0 \
-device vfio-pci,host=03:00.1,id=vf-00.1,bus=root.2,addr=0x0.1 \
-monitor stdio \
Comment 13 jingzhao 2016-07-15 09:12:32 UTC 
Not a regression issue. 
Reproduced it with kernel-3.10.0-327.el7.x86_64 and qemu-kvm-rhev-2.3.0-31.el7_2.18.x86_64.

/usr/libexec/qemu-kvm \
-M q35 \
-cpu SandyBridge \
-nodefaults -rtc base=utc \
-m 4G \
-smp 2,sockets=2,cores=1,threads=1 \
-enable-kvm \
-name rhel7.3 \
-uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
-smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \
-k en-us \
-serial unix:/tmp/serial0,server,nowait \
-boot menu=on \
-bios /usr/share/seabios/bios.bin \
-chardev file,path=/home/seabios.log,id=seabios \
-device isa-debugcon,chardev=seabios,iobase=0x402 \
-qmp tcp::8887,server,nowait \
-vga qxl \
-spice port=5932,disable-ticketing \
-device ioh3420,id=root.0,slot=1 \
-drive file=/home/q35-seabios.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=root.0,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 \
-device ioh3420,id=root.1,slot=2 \
-device x3130-upstream,bus=root.1,id=upstream1 \
-device xio3130-downstream,bus=upstream1,id=downstream1,chassis=1 \
-device xio3130-downstream,bus=upstream1,id=downstream2,chassis=2 \
-device xio3130-downstream,bus=upstream1,id=downstream3,chassis=3 \
-device ioh3420,id=root.2,slot=3 \
-device vfio-pci,host=03:10.0,id=vf-10.0,bus=root.2,multifunction=on,addr=0x0.0 \
-device vfio-pci,host=03:10.2,id=vf-10.2,bus=root.2,addr=0x0.1 \
-monitor stdio \
Comment 14 Alex Williamson 2016-07-15 17:31:19 UTC 
Patch posted upstream:

http://lists.nongnu.org/archive/html/qemu-devel/2016-07/msg03521.html
Comment 15 Miroslav Rezanina 2016-07-22 09:12:31 UTC 
Fix included in qemu-kvm-rhev-2.6.0-15.el7
Comment 17 jingzhao 2016-08-02 08:40:39 UTC 
reproduced the issue with qemu-kvm-rhev-2.6.0-12.el7.x86_64, verified it with qemu-kvm-rhev-2.6.0-17.el7.x86_64. Following is the detailed verified steps

1.Boot guest with following cli:
/usr/libexec/qemu-kvm \
-M q35 \
-cpu SandyBridge \
-nodefaults -rtc base=utc \
-m 4G \
-smp 2,sockets=2,cores=1,threads=1 \
-enable-kvm \
-name rhel7.3 \
-uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
-smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \
-k en-us \
-serial unix:/tmp/console,server,nowait \
-boot menu=on \
-bios /usr/share/seabios/bios.bin \
-chardev file,path=/home/seabios.log,id=seabios \
-device isa-debugcon,chardev=seabios,iobase=0x402 \
-qmp tcp::8887,server,nowait \
-vga qxl \
-spice port=5932,disable-ticketing \
-device ioh3420,id=root.0,slot=1 \
-drive file=/home/q35-seabios.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=root.0,drive=drive-virtio-disk0,id=virtio-disk0,disable-legacy=on,disable-modern=off,bootindex=1 \
-device ioh3420,id=root.1,slot=2 \
-device x3130-upstream,bus=root.1,id=upstream1 \
-device xio3130-downstream,bus=upstream1,id=downstream1,chassis=1 \
-device xio3130-downstream,bus=upstream1,id=downstream2,chassis=2 \
-device xio3130-downstream,bus=upstream1,id=downstream3,chassis=3 \
-drive file=/home/block1.qcow2,if=none,id=drive-virtio-disk1,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=downstream1,drive=drive-virtio-disk1,id=virtio-disk1,disable-legacy=on,disable-modern=off \
-drive file=/home/block2.raw,if=none,id=drive-virtio-disk2,format=raw,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=downstream2,drive=drive-virtio-disk2,id=virtio-disk2,disable-legacy=on,disable-modern=off \
-device ioh3420,id=root.2,slot=3 \
-device vfio-pci,host=03:10.0,id=vf-10.0,bus=root.2,multifunction=on,addr=0x0.0 \
-device vfio-pci,host=03:10.2,id=vf-10.2,bus=root.2,addr=0x0.1 \
-monitor stdio \

2.check the info through hmp
(qemu) info pci
  
  Bus  7, device   0, function 0:
    Ethernet controller: PCI device 8086:10ca
      BAR0: 64 bit memory at 0xf8600000 [0xf8603fff].
      BAR3: 64 bit memory at 0xf8604000 [0xf8607fff].
      id "vf-10.0"
  Bus  7, device   0, function 1:
    Ethernet controller: PCI device 8086:10ca
      BAR0: 64 bit memory at 0xf8608000 [0xf860bfff].
      BAR3: 64 bit memory at 0xf860c000 [0xf860ffff].
      id "vf-10.2"

3.check the nic info in guest
[root@localhost ~]# lspci -vvv -t
lspci -vvv -t
-[0000:00]-+-00.0  Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
           +-01.0  Red Hat, Inc. QXL paravirtual graphic card
           +-02.0-[01]----00.0  Red Hat, Inc Virtio block device
           +-03.0-[02-06]----00.0-[03-06]--+-00.0-[04]----00.0  Red Hat, Inc Virtio block device
           |                               +-01.0-[05]----00.0  Red Hat, Inc Virtio block device
           |                               \-02.0-[06]--
           +-04.0-[07]--+-00.0  Intel Corporation 82576 Virtual Function
           |            \-00.1  Intel Corporation 82576 Virtual Function
           +-1f.0  Intel Corporation 82801IB (ICH9) LPC Interface Controller
           +-1f.2  Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode]
           \-1f.3  Intel Corporation 82801I (ICH9 Family) SMBus Controller
[root@localhost ~]# ifconfig
ifconfig
ens3f0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 6a:36:21:1e:d8:75  txqueuelen 1000  (Ethernet)
        RX packets 22  bytes 1346 (1.3 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens3f1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether ba:3e:a9:a2:d5:8d  txqueuelen 1000  (Ethernet)
        RX packets 22  bytes 1346 (1.3 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 364  bytes 32344 (31.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 364  bytes 32344 (31.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 0.0.0.0
        ether 00:00:00:00:00:00  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@unused home]# ping 10.66.4.211 -I ens3f0
PING 10.66.4.211 (10.66.4.211) from 10.66.152.180 ens3f0: 56(84) bytes of data.
64 bytes from 10.66.4.211: icmp_seq=1 ttl=61 time=0.603 ms
64 bytes from 10.66.4.211: icmp_seq=2 ttl=61 time=0.593 ms
^C
--- 10.66.4.211 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.593/0.598/0.603/0.005 ms
[root@unused home]# ping 10.66.4.211 -I ens3f1
PING 10.66.4.211 (10.66.4.211) from 10.66.152.179 ens3f1: 56(84) bytes of data.
64 bytes from 10.66.4.211: icmp_seq=1 ttl=61 time=0.526 ms
64 bytes from 10.66.4.211: icmp_seq=2 ttl=61 time=0.573 ms
64 bytes from 10.66.4.211: icmp_seq=3 ttl=61 time=0.560 ms
64 bytes from 10.66.4.211: icmp_seq=4 ttl=61 time=0.566 ms


Thanks
Jing Zhao
Comment 20 errata-xmlrpc 2016-11-07 21:24:28 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2673.html