Bug 1363768

Summary: [Q35] Support pci-bridge
Product: Red Hat Enterprise Linux 7 Reporter: Marcel Apfelbaum <marcel>
Component: qemu-kvm-rhevAssignee: Marcel Apfelbaum <marcel>
Status: CLOSED WONTFIX QA Contact: jingzhao <jinzhao>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.3CC: chayang, juzhang, knoel, laine, marcel, virt-maint, yduan
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
PCI bridge is usable in Q35, but not tested yet.
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-26 13:11:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Marcel Apfelbaum 2016-08-03 13:58:41 UTC
PCI bridge is supported for i440fx, but not for Q35. Mainly we need to add it to test plan.

Comment 1 jingzhao 2016-08-04 03:21:40 UTC
Hi Marcel

  As QE understanding, PCI-Bridge should be used on q35 with following scenario:

   [1] pcie bus -- i82801b11-bridge -- pci bridge -- devices

   [2] pcie bus --- root port  -- pci bridge -- devices

   [3] pcie bus --- root port --upstream port -- downstream port  -- pci bridge -- devices

and I think the [1] is the common connected method for the PCI devices

Am I right?  Could you share more usage info with QE ?

I tested it on Q35 with above test scenario, and found the disk didn't be found in guest (bug1358665)

Thanks
Jing Zhao

Comment 2 Marcel Apfelbaum 2016-08-04 10:36:50 UTC
> --- Comment #1 from jingzhao <jinzhao> ---
> Hi Marcel
>
>   As QE understanding, PCI-Bridge should be used on q35 with following
> scenario:
>
>    [1] pcie bus -- i82801b11-bridge -- pci bridge -- devices

this is the only supported way. if you need more pci-bridges
use:
 a. pcie bus -- i82801b11-bridge -- pci bridge -- pci bridge ...
 b. use several i82801b11-bridges

>
>    [2] pcie bus --- root port  -- pci bridge -- devices
>

it may work, but not supported. theoretically you need a pcie-to-pci bridge for this.
we don't have an emulation for it.

>    [3] pcie bus --- root port --upstream port -- downstream port  -- pci bridge
> -- devices
>

same as [2] - not supported

> and I think the [1] is the common connected method for the PCI devices
>
> Am I right?  Could you share more usage info with QE ?
>

You are right.
Please try to think like this: we don't want to mix pci and pcie hierarchies.


> I tested it on Q35 with above test scenario, and found the disk didn't be found
> in guest (bug1358665)
>

If with [1] works, please close the BZ and *verify* libvirt doesn't allow [2] and [3].
If it does, please open a libvirt bz and cc me.

Thanks,
Marcel

> Thanks
> Jing Zhao
>

Comment 3 jingzhao 2016-08-10 08:04:18 UTC
Hi Marcel

  As your comments, I tried it with qemu-kvm-rhev-2.6.0-19.el7.x86_64 and it works, following is the detailed steps

1. boot guest with following cli:
/usr/libexec/qemu-kvm \
-M q35 \
-cpu SandyBridge \
-nodefaults -rtc base=utc \
-m 4G \
-smp 2,sockets=2,cores=1,threads=1 \
-enable-kvm \
-name rhel7.3 \
-uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
-smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \
-k en-us \
-serial unix:/tmp/serial0,server,nowait \
-boot menu=on \
-bios /usr/share/seabios/bios.bin \
-chardev file,path=/home/seabios.log,id=seabios \
-device isa-debugcon,chardev=seabios,iobase=0x402 \
-qmp tcp::8887,server,nowait \
-vga qxl \
-spice port=5932,disable-ticketing \
-device ioh3420,id=root.0,slot=1 \
-drive file=/home/q35-seabios.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=root.0,drive=drive-virtio-disk0,id=virtio-disk0,disable-legacy=on,disable-modern=off,bootindex=1 \
-device i82801b11-bridge,bus=pcie.0,id=bridge1 \
-device pci-bridge,bus=bridge1,id=bridge2,chassis_nr=1 \
-drive file=/home/pci-block1.qcow2,if=none,id=pci-block1,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=bridge2,drive=pci-block1,id=pciblock1,addr=0x6.0 \
-drive file=/home/pci-block2.qcow2,if=none,id=pci-block2,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=bridge2,drive=pci-block2,id=pciblock2,addr=0x7.0 \
-device i82801b11-bridge,bus=pcie.0,id=bridge3 \
-device pci-bridge,bus=bridge3,id=bridge4,chassis_nr=2 \
-drive file=/home/pci-block3.qcow2,if=none,id=pci-block3,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=bridge3,drive=pci-block3,id=pciblock3,addr=0x8.0 \
-device ioh3420,id=root.2,slot=3 \
-device virtio-net-pci,bus=root.2,netdev=tap10,mac=9a:6a:6b:6c:6d:6e,disable-legacy=on,disable-modern=off -netdev tap,id=tap10 \
-monitor stdio \

2.check the info through hmp:

....
 dev: i82801b11-bridge, id "bridge3"
        addr = 04.0
        romfile = ""
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        x-pcie-lnksta-dllla = true
        class PCI bridge, addr 00:04.0, pci id 8086:244e (sub 0000:0000)
        bus: bridge3
          type PCI
          dev: virtio-blk-pci, id "pciblock3"
            class = 0 (0x0)
            ioeventfd = true
            vectors = 2 (0x2)
            virtio-pci-bus-master-bug-migration = false
            disable-legacy = "off"
            disable-modern = false
            migrate-extra = true
            modern-pio-notify = false
            x-disable-pcie = false
            addr = 08.0
            romfile = ""
            rombar = 1 (0x1)
            multifunction = false
            command_serr_enable = true
            x-pcie-lnksta-dllla = true
            class SCSI controller, addr 04:08.0, pci id 1af4:1001 (sub 1af4:0002)
            bar 0: i/o at 0xd000 [0xd03f]
            bar 1: mem at 0xf8201000 [0xf8201fff]
            bar 4: mem at 0xfb800000 [0xfbffffff]
            bus: virtio-bus
              type virtio-pci-bus
              dev: virtio-blk-device, id ""
                drive = "pci-block3"
                logical_block_size = 512 (0x200)
                physical_block_size = 512 (0x200)
                min_io_size = 0 (0x0)
                opt_io_size = 0 (0x0)
                discard_granularity = 4294967295 (0xffffffff)
                cyls = 16383 (0x3fff)
                heads = 16 (0x10)
                secs = 63 (0x3f)
                serial = ""
                config-wce = true
                scsi = false
                request-merging = true
                indirect_desc = true
                event_idx = true
                notify_on_empty = true
                any_layout = true
          dev: pci-bridge, id "bridge4"
            chassis_nr = 2 (0x2)
            msi = true
            shpc = true
            addr = 00.0
            romfile = ""
            rombar = 1 (0x1)
            multifunction = false
            command_serr_enable = true
            x-pcie-lnksta-dllla = true
            class PCI bridge, addr 04:00.0, pci id 1b36:0001 (sub 0000:0000)
            bar 0: mem at 0xf8200000 [0xf82000ff]
            bus: bridge4
              type PCI
      dev: i82801b11-bridge, id "bridge1"
        addr = 03.0
        romfile = ""
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        x-pcie-lnksta-dllla = true
        class PCI bridge, addr 00:03.0, pci id 8086:244e (sub 0000:0000)
        bus: bridge1
          type PCI
          dev: pci-bridge, id "bridge2"
            chassis_nr = 1 (0x1)
            msi = true
            shpc = true
            addr = 00.0
            romfile = ""
            rombar = 1 (0x1)
            multifunction = false
            command_serr_enable = true
            x-pcie-lnksta-dllla = true
            class PCI bridge, addr 02:00.0, pci id 1b36:0001 (sub 0000:0000)
            bar 0: mem at 0xf8600000 [0xf86000ff]
            bus: bridge2
              type PCI
              dev: virtio-blk-pci, id "pciblock2"
                class = 0 (0x0)
                ioeventfd = true
                vectors = 2 (0x2)
                virtio-pci-bus-master-bug-migration = false
                disable-legacy = "off"
                disable-modern = false
                migrate-extra = true
                modern-pio-notify = false
                x-disable-pcie = false
                addr = 07.0
                romfile = ""
                rombar = 1 (0x1)
                multifunction = false
                command_serr_enable = true
                x-pcie-lnksta-dllla = true
                class SCSI controller, addr 03:07.0, pci id 1af4:1001 (sub 1af4:0002)
                bar 0: i/o at 0xe040 [0xe07f]
                bar 1: mem at 0xf8401000 [0xf8401fff]
                bar 4: mem at 0xfd000000 [0xfd7fffff]
                bus: virtio-bus
                  type virtio-pci-bus
                  dev: virtio-blk-device, id ""
                    drive = "pci-block2"
                    logical_block_size = 512 (0x200)
                    physical_block_size = 512 (0x200)
                    min_io_size = 0 (0x0)
                    opt_io_size = 0 (0x0)
                    discard_granularity = 4294967295 (0xffffffff)
                    cyls = 10402 (0x28a2)
                    heads = 16 (0x10)
                    secs = 63 (0x3f)
                    serial = ""
                    config-wce = true
                    scsi = false
                    request-merging = true
                    indirect_desc = true
                    event_idx = true
                    notify_on_empty = true
                    any_layout = true
              dev: virtio-blk-pci, id "pciblock1"
                class = 0 (0x0)
                ioeventfd = true
                vectors = 2 (0x2)
                virtio-pci-bus-master-bug-migration = false
                disable-legacy = "off"
                disable-modern = false
                migrate-extra = true
                modern-pio-notify = false
                x-disable-pcie = false
                addr = 06.0
                romfile = ""
                rombar = 1 (0x1)
                multifunction = false
                command_serr_enable = true
                x-pcie-lnksta-dllla = true
                class SCSI controller, addr 03:06.0, pci id 1af4:1001 (sub 1af4:0002)
                bar 0: i/o at 0xe000 [0xe03f]
                bar 1: mem at 0xf8400000 [0xf8400fff]
                bar 4: mem at 0xfc800000 [0xfcffffff]
                bus: virtio-bus
                  type virtio-pci-bus
                  dev: virtio-blk-device, id ""
                    drive = "pci-block1"
                    logical_block_size = 512 (0x200)
                    physical_block_size = 512 (0x200)
                    min_io_size = 0 (0x0)
                    opt_io_size = 0 (0x0)
                    discard_granularity = 4294967295 (0xffffffff)
                    cyls = 16383 (0x3fff)
                    heads = 16 (0x10)
                    secs = 63 (0x3f)
                    serial = ""
                    config-wce = true
                    scsi = false
                    request-merging = true
                    indirect_desc = true
                    event_idx = true
                    notify_on_empty = true
                    any_layout = true

3. check the block info in guest

[root@localhost ~]# lsblk
NAME          MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
vda           252:0    0  30G  0 disk 
├─vda1        252:1    0   1G  0 part /boot
└─vda2        252:2    0  29G  0 part 
  ├─rhel-root 253:0    0  26G  0 lvm  /
  └─rhel-swap 253:1    0   3G  0 lvm  [SWAP]
vdb           252:16   0  10G  0 disk 
vdc           252:32   0   5G  0 disk 
vdd           252:48   0   8G  0 disk 
[root@localhost ~]# lspci -vvv -t
-[0000:00]-+-00.0  Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
           +-01.0  Red Hat, Inc. QXL paravirtual graphic card
           +-02.0-[01]----00.0  Red Hat, Inc Virtio block device
           +-03.0-[02-03]----00.0-[03]--+-06.0  Red Hat, Inc Virtio block device
           |                            \-07.0  Red Hat, Inc Virtio block device
           +-04.0-[04-05]--+-00.0-[05]--
           |               \-08.0  Red Hat, Inc Virtio block device
           +-05.0-[06]----00.0  Red Hat, Inc Virtio network device
           +-1f.0  Intel Corporation 82801IB (ICH9) LPC Interface Controller
           +-1f.2  Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode]
           \-1f.3  Intel Corporation 82801I (ICH9 Family) SMBus Controller
[root@localhost ~]# lspci 
00:00.0 Host bridge: Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
00:01.0 VGA compatible controller: Red Hat, Inc. QXL paravirtual graphic card (rev 04)
00:02.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
00:03.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 92)
00:04.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 92)
00:05.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
00:1f.0 ISA bridge: Intel Corporation 82801IB (ICH9) LPC Interface Controller (rev 02)
00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] (rev 02)
00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 02)
01:00.0 SCSI storage controller: Red Hat, Inc Virtio block device (rev 01)
02:00.0 PCI bridge: Red Hat, Inc. QEMU PCI-PCI bridge
03:06.0 SCSI storage controller: Red Hat, Inc Virtio block device
03:07.0 SCSI storage controller: Red Hat, Inc Virtio block device
04:00.0 PCI bridge: Red Hat, Inc. QEMU PCI-PCI bridge
04:08.0 SCSI storage controller: Red Hat, Inc Virtio block device
06:00.0 Ethernet controller: Red Hat, Inc Virtio network device (rev 01)

Is it enough ?

Thanks
Jing Zhao

Comment 4 jingzhao 2016-08-10 09:23:25 UTC
(In reply to Marcel Apfelbaum from comment #2)
> > --- Comment #1 from jingzhao <jinzhao> ---
> > Hi Marcel
> >
> >   As QE understanding, PCI-Bridge should be used on q35 with following
> > scenario:
> >
> >    [1] pcie bus -- i82801b11-bridge -- pci bridge -- devices
> 
> this is the only supported way. if you need more pci-bridges
> use:
>  a. pcie bus -- i82801b11-bridge -- pci bridge -- pci bridge ...
>  b. use several i82801b11-bridges
> 
> >
> >    [2] pcie bus --- root port  -- pci bridge -- devices
> >
> 
> it may work, but not supported. theoretically you need a pcie-to-pci bridge
> for this.
> we don't have an emulation for it.
> 
> >    [3] pcie bus --- root port --upstream port -- downstream port  -- pci bridge
> > -- devices
> >
> 
> same as [2] - not supported
> 
> > and I think the [1] is the common connected method for the PCI devices
> >
> > Am I right?  Could you share more usage info with QE ?
> >
> 
> You are right.
> Please try to think like this: we don't want to mix pci and pcie hierarchies.
> 
> 
> > I tested it on Q35 with above test scenario, and found the disk didn't be found
> > in guest (bug1358665)
> >
> 
> If with [1] works, please close the BZ and *verify* libvirt doesn't allow
> [2] and [3].
> If it does, please open a libvirt bz and cc me.

about [2] and [3], bz1362648 can track the issue

Thanks
Jing Zhao

> 
> Thanks,
> Marcel
> 
> > Thanks
> > Jing Zhao
> >

Comment 5 Laine Stump 2016-08-10 18:48:53 UTC
Just as it does with legacy PCI endpoint devices, libvirt will never auto-assign a pci-bridge to plug into anything except a dmi-to-pci-bridge or another pci-bridge. However, there are existing configurations in the wild that, e.g., plug a pci-bridge directly into pcie-root, and this reportedly does work. For that reason, if you specifically ask for it, libvirt will not deny you. This is not considered a bug, but is rather left in place both for compatibility with existing configs, and for a fallback in case a dmi-to-pci-bridge isn't a possibility.

In the end, if it works, libvirt allows it but doesn't suggest it (we've discussed the possibility of "tainting" the domain in those cases, for whatever good that might do); if it doesn't work, libvirt will prohibit it when you ask for it.

Comment 6 Marcel Apfelbaum 2016-08-14 13:48:43 UTC
(In reply to jingzhao from comment #3)
> Hi Marcel
> 
>   As your comments, I tried it with qemu-kvm-rhev-2.6.0-19.el7.x86_64 and it
> works, following is the detailed steps
> 
> 1. boot guest with following cli:
> /usr/libexec/qemu-kvm \
> -M q35 \
> -cpu SandyBridge \
> -nodefaults -rtc base=utc \
> -m 4G \
> -smp 2,sockets=2,cores=1,threads=1 \
> -enable-kvm \
> -name rhel7.3 \
> -uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
> -smbios type=1,manufacturer='Red Hat',product='RHEV
> Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-
> e63fcdb67170 \
> -k en-us \
> -serial unix:/tmp/serial0,server,nowait \
> -boot menu=on \
> -bios /usr/share/seabios/bios.bin \
> -chardev file,path=/home/seabios.log,id=seabios \
> -device isa-debugcon,chardev=seabios,iobase=0x402 \
> -qmp tcp::8887,server,nowait \
> -vga qxl \
> -spice port=5932,disable-ticketing \
> -device ioh3420,id=root.0,slot=1 \
> -drive
> file=/home/q35-seabios.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,
> cache=none,werror=stop,rerror=stop \
> -device
> virtio-blk-pci,bus=root.0,drive=drive-virtio-disk0,id=virtio-disk0,disable-
> legacy=on,disable-modern=off,bootindex=1 \
> -device i82801b11-bridge,bus=pcie.0,id=bridge1 \
> -device pci-bridge,bus=bridge1,id=bridge2,chassis_nr=1 \
> -drive
> file=/home/pci-block1.qcow2,if=none,id=pci-block1,format=qcow2,cache=none,
> werror=stop,rerror=stop \
> -device virtio-blk-pci,bus=bridge2,drive=pci-block1,id=pciblock1,addr=0x6.0 \
> -drive
> file=/home/pci-block2.qcow2,if=none,id=pci-block2,format=qcow2,cache=none,
> werror=stop,rerror=stop \
> -device virtio-blk-pci,bus=bridge2,drive=pci-block2,id=pciblock2,addr=0x7.0 \
> -device i82801b11-bridge,bus=pcie.0,id=bridge3 \
> -device pci-bridge,bus=bridge3,id=bridge4,chassis_nr=2 \
> -drive
> file=/home/pci-block3.qcow2,if=none,id=pci-block3,format=qcow2,cache=none,
> werror=stop,rerror=stop \
> -device virtio-blk-pci,bus=bridge3,drive=pci-block3,id=pciblock3,addr=0x8.0 \
> -device ioh3420,id=root.2,slot=3 \
> -device
> virtio-net-pci,bus=root.2,netdev=tap10,mac=9a:6a:6b:6c:6d:6e,disable-
> legacy=on,disable-modern=off -netdev tap,id=tap10 \
> -monitor stdio \
> 
> 2.check the info through hmp:
> 
> ....
>  dev: i82801b11-bridge, id "bridge3"
>         addr = 04.0
>         romfile = ""
>         rombar = 1 (0x1)
>         multifunction = false
>         command_serr_enable = true
>         x-pcie-lnksta-dllla = true
>         class PCI bridge, addr 00:04.0, pci id 8086:244e (sub 0000:0000)
>         bus: bridge3
>           type PCI
>           dev: virtio-blk-pci, id "pciblock3"
>             class = 0 (0x0)
>             ioeventfd = true
>             vectors = 2 (0x2)
>             virtio-pci-bus-master-bug-migration = false
>             disable-legacy = "off"
>             disable-modern = false
>             migrate-extra = true
>             modern-pio-notify = false
>             x-disable-pcie = false
>             addr = 08.0
>             romfile = ""
>             rombar = 1 (0x1)
>             multifunction = false
>             command_serr_enable = true
>             x-pcie-lnksta-dllla = true
>             class SCSI controller, addr 04:08.0, pci id 1af4:1001 (sub
> 1af4:0002)
>             bar 0: i/o at 0xd000 [0xd03f]
>             bar 1: mem at 0xf8201000 [0xf8201fff]
>             bar 4: mem at 0xfb800000 [0xfbffffff]
>             bus: virtio-bus
>               type virtio-pci-bus
>               dev: virtio-blk-device, id ""
>                 drive = "pci-block3"
>                 logical_block_size = 512 (0x200)
>                 physical_block_size = 512 (0x200)
>                 min_io_size = 0 (0x0)
>                 opt_io_size = 0 (0x0)
>                 discard_granularity = 4294967295 (0xffffffff)
>                 cyls = 16383 (0x3fff)
>                 heads = 16 (0x10)
>                 secs = 63 (0x3f)
>                 serial = ""
>                 config-wce = true
>                 scsi = false
>                 request-merging = true
>                 indirect_desc = true
>                 event_idx = true
>                 notify_on_empty = true
>                 any_layout = true
>           dev: pci-bridge, id "bridge4"
>             chassis_nr = 2 (0x2)
>             msi = true
>             shpc = true
>             addr = 00.0
>             romfile = ""
>             rombar = 1 (0x1)
>             multifunction = false
>             command_serr_enable = true
>             x-pcie-lnksta-dllla = true
>             class PCI bridge, addr 04:00.0, pci id 1b36:0001 (sub 0000:0000)
>             bar 0: mem at 0xf8200000 [0xf82000ff]
>             bus: bridge4
>               type PCI
>       dev: i82801b11-bridge, id "bridge1"
>         addr = 03.0
>         romfile = ""
>         rombar = 1 (0x1)
>         multifunction = false
>         command_serr_enable = true
>         x-pcie-lnksta-dllla = true
>         class PCI bridge, addr 00:03.0, pci id 8086:244e (sub 0000:0000)
>         bus: bridge1
>           type PCI
>           dev: pci-bridge, id "bridge2"
>             chassis_nr = 1 (0x1)
>             msi = true
>             shpc = true
>             addr = 00.0
>             romfile = ""
>             rombar = 1 (0x1)
>             multifunction = false
>             command_serr_enable = true
>             x-pcie-lnksta-dllla = true
>             class PCI bridge, addr 02:00.0, pci id 1b36:0001 (sub 0000:0000)
>             bar 0: mem at 0xf8600000 [0xf86000ff]
>             bus: bridge2
>               type PCI
>               dev: virtio-blk-pci, id "pciblock2"
>                 class = 0 (0x0)
>                 ioeventfd = true
>                 vectors = 2 (0x2)
>                 virtio-pci-bus-master-bug-migration = false
>                 disable-legacy = "off"
>                 disable-modern = false
>                 migrate-extra = true
>                 modern-pio-notify = false
>                 x-disable-pcie = false
>                 addr = 07.0
>                 romfile = ""
>                 rombar = 1 (0x1)
>                 multifunction = false
>                 command_serr_enable = true
>                 x-pcie-lnksta-dllla = true
>                 class SCSI controller, addr 03:07.0, pci id 1af4:1001 (sub
> 1af4:0002)
>                 bar 0: i/o at 0xe040 [0xe07f]
>                 bar 1: mem at 0xf8401000 [0xf8401fff]
>                 bar 4: mem at 0xfd000000 [0xfd7fffff]
>                 bus: virtio-bus
>                   type virtio-pci-bus
>                   dev: virtio-blk-device, id ""
>                     drive = "pci-block2"
>                     logical_block_size = 512 (0x200)
>                     physical_block_size = 512 (0x200)
>                     min_io_size = 0 (0x0)
>                     opt_io_size = 0 (0x0)
>                     discard_granularity = 4294967295 (0xffffffff)
>                     cyls = 10402 (0x28a2)
>                     heads = 16 (0x10)
>                     secs = 63 (0x3f)
>                     serial = ""
>                     config-wce = true
>                     scsi = false
>                     request-merging = true
>                     indirect_desc = true
>                     event_idx = true
>                     notify_on_empty = true
>                     any_layout = true
>               dev: virtio-blk-pci, id "pciblock1"
>                 class = 0 (0x0)
>                 ioeventfd = true
>                 vectors = 2 (0x2)
>                 virtio-pci-bus-master-bug-migration = false
>                 disable-legacy = "off"
>                 disable-modern = false
>                 migrate-extra = true
>                 modern-pio-notify = false
>                 x-disable-pcie = false
>                 addr = 06.0
>                 romfile = ""
>                 rombar = 1 (0x1)
>                 multifunction = false
>                 command_serr_enable = true
>                 x-pcie-lnksta-dllla = true
>                 class SCSI controller, addr 03:06.0, pci id 1af4:1001 (sub
> 1af4:0002)
>                 bar 0: i/o at 0xe000 [0xe03f]
>                 bar 1: mem at 0xf8400000 [0xf8400fff]
>                 bar 4: mem at 0xfc800000 [0xfcffffff]
>                 bus: virtio-bus
>                   type virtio-pci-bus
>                   dev: virtio-blk-device, id ""
>                     drive = "pci-block1"
>                     logical_block_size = 512 (0x200)
>                     physical_block_size = 512 (0x200)
>                     min_io_size = 0 (0x0)
>                     opt_io_size = 0 (0x0)
>                     discard_granularity = 4294967295 (0xffffffff)
>                     cyls = 16383 (0x3fff)
>                     heads = 16 (0x10)
>                     secs = 63 (0x3f)
>                     serial = ""
>                     config-wce = true
>                     scsi = false
>                     request-merging = true
>                     indirect_desc = true
>                     event_idx = true
>                     notify_on_empty = true
>                     any_layout = true
> 
> 3. check the block info in guest
> 
> [root@localhost ~]# lsblk
> NAME          MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> vda           252:0    0  30G  0 disk 
> ├─vda1        252:1    0   1G  0 part /boot
> └─vda2        252:2    0  29G  0 part 
>   ├─rhel-root 253:0    0  26G  0 lvm  /
>   └─rhel-swap 253:1    0   3G  0 lvm  [SWAP]
> vdb           252:16   0  10G  0 disk 
> vdc           252:32   0   5G  0 disk 
> vdd           252:48   0   8G  0 disk 
> [root@localhost ~]# lspci -vvv -t
> -[0000:00]-+-00.0  Intel Corporation 82G33/G31/P35/P31 Express DRAM
> Controller
>            +-01.0  Red Hat, Inc. QXL paravirtual graphic card
>            +-02.0-[01]----00.0  Red Hat, Inc Virtio block device
>            +-03.0-[02-03]----00.0-[03]--+-06.0  Red Hat, Inc Virtio block
> device
>            |                            \-07.0  Red Hat, Inc Virtio block
> device
>            +-04.0-[04-05]--+-00.0-[05]--
>            |               \-08.0  Red Hat, Inc Virtio block device
>            +-05.0-[06]----00.0  Red Hat, Inc Virtio network device
>            +-1f.0  Intel Corporation 82801IB (ICH9) LPC Interface Controller
>            +-1f.2  Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA
> Controller [AHCI mode]
>            \-1f.3  Intel Corporation 82801I (ICH9 Family) SMBus Controller
> [root@localhost ~]# lspci 
> 00:00.0 Host bridge: Intel Corporation 82G33/G31/P35/P31 Express DRAM
> Controller
> 00:01.0 VGA compatible controller: Red Hat, Inc. QXL paravirtual graphic
> card (rev 04)
> 00:02.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express
> Root Port 0 (rev 02)
> 00:03.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 92)
> 00:04.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 92)
> 00:05.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express
> Root Port 0 (rev 02)
> 00:1f.0 ISA bridge: Intel Corporation 82801IB (ICH9) LPC Interface
> Controller (rev 02)
> 00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6
> port SATA Controller [AHCI mode] (rev 02)
> 00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev
> 02)
> 01:00.0 SCSI storage controller: Red Hat, Inc Virtio block device (rev 01)
> 02:00.0 PCI bridge: Red Hat, Inc. QEMU PCI-PCI bridge
> 03:06.0 SCSI storage controller: Red Hat, Inc Virtio block device
> 03:07.0 SCSI storage controller: Red Hat, Inc Virtio block device
> 04:00.0 PCI bridge: Red Hat, Inc. QEMU PCI-PCI bridge
> 04:08.0 SCSI storage controller: Red Hat, Inc Virtio block device
> 06:00.0 Ethernet controller: Red Hat, Inc Virtio network device (rev 01)
> 
> Is it enough ?
>

Yes,

Thanks,
Marcel
 
> Thanks
> Jing Zhao