RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1272759 - [PCIe] Make virtio devices pci-express on Q35 machines
Summary: [PCIe] Make virtio devices pci-express on Q35 machines
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.3
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Marcel Apfelbaum
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks: 1227278 1305606 1313485
TreeView+ depends on / blocked
 
Reported: 2015-10-18 14:23 UTC by Marcel Apfelbaum
Modified: 2016-11-07 20:45 UTC (History)
11 users (show)

Fixed In Version: qemu-kvm-rhev-2.5.0-1.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-11-07 20:45:50 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:2673 0 normal SHIPPED_LIVE qemu-kvm-rhev bug fix and enhancement update 2016-11-08 01:06:13 UTC

Description Marcel Apfelbaum 2015-10-18 14:23:28 UTC 
Under Q35 virtio devices should be PCIe devices.
Comment 2 Marcel Apfelbaum 2015-10-19 09:09:59 UTC 
Breaking it down to tasks:
 - Add PCIe capability and the minimum required caps as required by PCIe spec. 
 - Virtio PCIe devices should have no bars in io address space
Comment 3 Marcel Apfelbaum 2015-12-22 10:08:36 UTC 
The solution is part of QEMU 2.5, once we rebase it can be passed to QE.
Comment 5 jingzhao 2016-05-18 04:18:57 UTC 
Hi Marcel

  How about the test scenario for the feature? Following is the steps which I tested with the latest build, any idea for this feature.

kernel-3.10.0-402.el7.x86_64
qemu-kvm-rhev-2.6.0-1.el7.x86_64
seabios-1.9.1-4.el7.x86_64

Test steps:
1.Boot vm with following command:

(add the "disable-modern=off" in the command)

/usr/libexec/qemu-kvm \
-M q35 \
-cpu SandyBridge \
-nodefaults -rtc base=utc \
-m 4G \
-smp 2,sockets=2,cores=1,threads=1 \
-enable-kvm \
-name rhel7 \
-uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
-smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \
-k en-us \
-serial unix:/tmp/serial0,server,nowait \
-boot menu=on \
-bios /usr/share/seabios/bios.bin \
-vga std \
-vnc :1 \
-qmp tcp:0:6666,server,nowait \
-chardev file,path=/home/bug/seabios.log,id=seabios \
-device isa-debugcon,chardev=seabios,iobase=0x402 \
-device ioh3420,bus=pcie.0,id=root.0,slot=1 \
-device x3130-upstream,bus=root.0,id=upstream1 \
-device xio3130-downstream,bus=upstream1,id=downstream1,chassis=1 \
-drive file=/home/bug/1272939/pxb.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop,aio=threads \
-device virtio-blk-pci,bus=downstream1,drive=drive-virtio-disk0,id=virtio-disk0,disable-modern=off,bootindex=0 \
-device xio3130-downstream,bus=upstream1,id=downstream2,chassis=2 \
-device virtio-scsi-pci,bus=downstream2,id=scsi_pci_bus0,disable-modern=off \
-drive file=/home/bug/1272759/test.qcow2,if=none,id=virtio-scsi,format=qcow2,cache=none,werror=stop,rerror=stop,aio=threads \
-device scsi-hd,bus=scsi_pci_bus0.0,drive=virtio-scsi,id=virtio-disk1 \
-device xio3130-downstream,bus=upstream1,id=downstream3,chassis=3 \
-device virtio-net-pci,bus=downstream3,netdev=tap10,mac=9a:6a:6b:6c:6d:6e,disable-modern=off \
-netdev tap,id=tap10 \
-monitor stdio \

2.Check the info in hmp

(qemu) info qtree
bus: main-system-bus
  type System
  dev: kvm-ioapic, id ""
    gpio-in "" 24
    gsi_base = 0 (0x0)
    mmio 00000000fec00000/0000000000001000
  dev: q35-pcihost, id ""
    MCFG = 2952790016 (0xb0000000)
    pci-hole64-size = 0 (0 B)
    short_root_bus = 0 (0x0)
    bus: pcie.0
      type PCIE
      dev: ioh3420, id "root.0"
        power_controller_present = true
        chassis = 0 (0x0)
        slot = 1 (0x1)
        port = 0 (0x0)
        aer_log_max = 8 (0x8)
        addr = 02.0
        romfile = ""
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        class PCI bridge, addr 00:02.0, pci id 8086:3420 (sub 0000:0000)
        bus: root.0
          type PCIE
          dev: x3130-upstream, id "upstream1"
            port = 0 (0x0)
            aer_log_max = 8 (0x8)
            addr = 00.0
            romfile = ""
            rombar = 1 (0x1)
            multifunction = false
            command_serr_enable = true
            class PCI bridge, addr 01:00.0, pci id 104c:8232 (sub 0000:0000)
            bus: upstream1
              type PCIE
              dev: xio3130-downstream, id "downstream3"
                power_controller_present = true
                chassis = 3 (0x3)
                slot = 0 (0x0)
                port = 0 (0x0)
                aer_log_max = 8 (0x8)
                addr = 02.0
                romfile = ""
                rombar = 1 (0x1)
                multifunction = false
                command_serr_enable = true
                class PCI bridge, addr 02:02.0, pci id 104c:8233 (sub 0000:0000)
                bus: downstream3
                  type PCIE
                  dev: virtio-net-pci, id ""
                    ioeventfd = false
                    vectors = 3 (0x3)
                    virtio-pci-bus-master-bug-migration = true
                    disable-legacy = false
                    disable-modern = false
                    migrate-extra = true
                    modern-pio-notify = false
                    x-disable-pcie = false
                    addr = 00.0
                    romfile = "pxe-virtio.rom"
                    rombar = 1 (0x1)
                    multifunction = false
                    command_serr_enable = true
                    class Ethernet controller, addr 05:00.0, pci id 1af4:1000 (sub 1af4:0001)
                    bar 0: i/o at 0xc000 [0xc01f]
                    bar 1: mem at 0xfe440000 [0xfe440fff]
                    bar 4: mem at 0xfc000000 [0xfc7fffff]
                    bar 6: mem at 0xffffffffffffffff [0x3fffe]
                    bus: virtio-bus
                      type virtio-pci-bus
                      dev: virtio-net-device, id ""
                        csum = true
                        guest_csum = true
                        gso = true
                        guest_tso4 = true
                        guest_tso6 = true
                        guest_ecn = true
                        guest_ufo = true
                        guest_announce = true
                        host_tso4 = true
                        host_tso6 = true
                        host_ecn = true
                        host_ufo = true
                        mrg_rxbuf = true
                        status = true
                        ctrl_vq = true
                        ctrl_rx = true
                        ctrl_vlan = true
                        ctrl_rx_extra = true
                        ctrl_mac_addr = true
                        ctrl_guest_offloads = true
                        mq = false
                        mac = "9a:6a:6b:6c:6d:6e"
                        vlan = <null>
                        netdev = "tap10"
                        x-txtimer = 150000 (0x249f0)
                        x-txburst = 256 (0x100)
                        tx = ""
                        indirect_desc = true
                        event_idx = true
                        notify_on_empty = true
                        any_layout = true
              dev: xio3130-downstream, id "downstream2"
                power_controller_present = true
                chassis = 2 (0x2)
                slot = 0 (0x0)
                port = 0 (0x0)
                aer_log_max = 8 (0x8)
                addr = 01.0
                romfile = ""
                rombar = 1 (0x1)
                multifunction = false
                command_serr_enable = true
                class PCI bridge, addr 02:01.0, pci id 104c:8233 (sub 0000:0000)
                bus: downstream2
                  type PCIE
                  dev: virtio-scsi-pci, id "scsi_pci_bus0"
                    ioeventfd = true
                    vectors = 4 (0x4)
                    virtio-pci-bus-master-bug-migration = true
                    disable-legacy = false
                    disable-modern = false
                    migrate-extra = true
                    modern-pio-notify = false
                    x-disable-pcie = false
                    addr = 00.0
                    romfile = ""
                    rombar = 1 (0x1)
                    multifunction = false
                    command_serr_enable = true
                    class SCSI controller, addr 04:00.0, pci id 1af4:1004 (sub 1af4:0008)
                    bar 0: i/o at 0xd000 [0xd03f]
                    bar 1: mem at 0xfe600000 [0xfe600fff]
                    bar 4: mem at 0xfc800000 [0xfcffffff]
                    bus: virtio-bus
                      type virtio-pci-bus
                      dev: virtio-scsi-device, id ""
                        num_queues = 1 (0x1)
                        max_sectors = 65535 (0xffff)
                        cmd_per_lun = 128 (0x80)
                        hotplug = true
                        param_change = true
                        indirect_desc = true
                        event_idx = true
                        notify_on_empty = true
                        any_layout = true
                        bus: scsi_pci_bus0.0
                          type SCSI
                          dev: scsi-hd, id "virtio-disk1"
                            drive = "virtio-scsi"
                            logical_block_size = 512 (0x200)
                            physical_block_size = 512 (0x200)
                            min_io_size = 0 (0x0)
                            opt_io_size = 0 (0x0)
                            discard_granularity = 4096 (0x1000)
                            ver = "2.5+"
                            serial = ""
                            vendor = "QEMU"
                            product = "QEMU HARDDISK"
                            removable = false
                            dpofua = false
                            wwn = 0 (0x0)
                            port_wwn = 0 (0x0)
                            port_index = 0 (0x0)
                            max_unmap_size = 1073741824 (0x40000000)
                            max_io_size = 2147483647 (0x7fffffff)
                            cyls = 2080 (0x820)
                            heads = 16 (0x10)
                            secs = 63 (0x3f)
                            channel = 0 (0x0)
                            scsi-id = 0 (0x0)
                            lun = 0 (0x0)
              dev: xio3130-downstream, id "downstream1"
                power_controller_present = true
                chassis = 1 (0x1)
                slot = 0 (0x0)
                port = 0 (0x0)
                aer_log_max = 8 (0x8)
                addr = 00.0
                romfile = ""
                rombar = 1 (0x1)
                multifunction = false
                command_serr_enable = true
                class PCI bridge, addr 02:00.0, pci id 104c:8233 (sub 0000:0000)
                bus: downstream1
                  type PCIE
                  dev: virtio-blk-pci, id "virtio-disk0"
                    class = 0 (0x0)
                    ioeventfd = true
                    vectors = 2 (0x2)
                    virtio-pci-bus-master-bug-migration = true
                    disable-legacy = false
                    disable-modern = false
                    migrate-extra = true
                    modern-pio-notify = false
                    x-disable-pcie = false
                    addr = 00.0
                    romfile = ""
                    rombar = 1 (0x1)
                    multifunction = false
                    command_serr_enable = true
                    class SCSI controller, addr 03:00.0, pci id 1af4:1001 (sub 1af4:0002)
                    bar 0: i/o at 0xe000 [0xe03f]
                    bar 1: mem at 0xfe800000 [0xfe800fff]
                    bar 4: mem at 0xfd000000 [0xfd7fffff]
                    bus: virtio-bus
                      type virtio-pci-bus
                      dev: virtio-blk-device, id ""
                        drive = "drive-virtio-disk0"
                        logical_block_size = 512 (0x200)
                        physical_block_size = 512 (0x200)
                        min_io_size = 0 (0x0)
                        opt_io_size = 0 (0x0)
                        discard_granularity = 4294967295 (0xffffffff)
                        cyls = 16383 (0x3fff)
                        heads = 16 (0x10)
                        secs = 63 (0x3f)
                        serial = ""
                        config-wce = true
                        scsi = false
                        request-merging = true
                        indirect_desc = true
                        event_idx = true
                        notify_on_empty = true
                        any_layout = true
      dev: VGA, id ""
        vgamem_mb = 16 (0x10)
        mmio = true
        qemu-extended-regs = false
        addr = 01.0
        romfile = "vgabios-stdvga.bin"
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        class VGA controller, addr 00:01.0, pci id 1234:1111 (sub 1af4:1100)
        bar 0: mem at 0xfb000000 [0xfbffffff]
        bar 2: mem at 0xfea10000 [0xfea10fff]
        bar 6: mem at 0xffffffffffffffff [0xfffe]



3. Check the capability in the guest through "lspci -vnn" and we can got the info like "Capabilities: [40] Express Endpoint, MSI 00"

03:00.0 SCSI storage controller [0100]: Red Hat, Inc Virtio block device [1af4:1001]
	Subsystem: Red Hat, Inc Device [1af4:0002]
	Physical Slot: 0
	Flags: bus master, fast devsel, latency 0, IRQ 22
	I/O ports at e000 [size=64]
	Memory at fe800000 (32-bit, non-prefetchable) [size=4K]
	Memory at fd000000 (64-bit, prefetchable) [size=8M]
	Capabilities: [dc] MSI-X: Enable+ Count=2 Masked-
	Capabilities: [c8] Vendor Specific Information: Len=14 <?>
	Capabilities: [b4] Vendor Specific Information: Len=14 <?>
	Capabilities: [a4] Vendor Specific Information: Len=10 <?>
	Capabilities: [94] Vendor Specific Information: Len=10 <?>
	Capabilities: [84] Vendor Specific Information: Len=10 <?>
	Capabilities: [7c] Power Management version 3
	Capabilities: [40] Express Endpoint, MSI 00
	Kernel driver in use: virtio-pci

04:00.0 SCSI storage controller [0100]: Red Hat, Inc Virtio SCSI [1af4:1004]
	Subsystem: Red Hat, Inc Device [1af4:0008]
	Physical Slot: 0-1
	Flags: bus master, fast devsel, latency 0, IRQ 23
	I/O ports at d000 [size=64]
	Memory at fe600000 (32-bit, non-prefetchable) [size=4K]
	Memory at fc800000 (64-bit, prefetchable) [size=8M]
	Capabilities: [dc] MSI-X: Enable+ Count=4 Masked-
	Capabilities: [c8] Vendor Specific Information: Len=14 <?>
	Capabilities: [b4] Vendor Specific Information: Len=14 <?>
	Capabilities: [a4] Vendor Specific Information: Len=10 <?>
	Capabilities: [94] Vendor Specific Information: Len=10 <?>
	Capabilities: [84] Vendor Specific Information: Len=10 <?>
	Capabilities: [7c] Power Management version 3
	Capabilities: [40] Express Endpoint, MSI 00
	Kernel driver in use: virtio-pci

05:00.0 Ethernet controller [0200]: Red Hat, Inc Virtio network device [1af4:1000]
	Subsystem: Red Hat, Inc Device [1af4:0001]
	Physical Slot: 0-2
	Flags: bus master, fast devsel, latency 0, IRQ 20
	I/O ports at c000 [size=32]
	Memory at fe440000 (32-bit, non-prefetchable) [size=4K]
	Memory at fc000000 (64-bit, prefetchable) [size=8M]
	Expansion ROM at fe400000 [disabled] [size=256K]
	Capabilities: [dc] MSI-X: Enable+ Count=3 Masked-
	Capabilities: [c8] Vendor Specific Information: Len=14 <?>
	Capabilities: [b4] Vendor Specific Information: Len=14 <?>
	Capabilities: [a4] Vendor Specific Information: Len=10 <?>
	Capabilities: [94] Vendor Specific Information: Len=10 <?>
	Capabilities: [84] Vendor Specific Information: Len=10 <?>
	Capabilities: [7c] Power Management version 3
	Capabilities: [40] Express Endpoint, MSI 00
	Kernel driver in use: virtio-pci

4. In guest, make sure the virtio disk can work correctly
Comment 6 Marcel Apfelbaum 2016-05-19 18:54:32 UTC 
(In reply to jingzhao from comment #5)
> Hi Marcel
> 
>   How about the test scenario for the feature? Following is the steps which
> I tested with the latest build, any idea for this feature.
> 
> kernel-3.10.0-402.el7.x86_64
> qemu-kvm-rhev-2.6.0-1.el7.x86_64
> seabios-1.9.1-4.el7.x86_64
> 
> Test steps:
> 1.Boot vm with following command:
> 
> (add the "disable-modern=off" in the command)
> 
> /usr/libexec/qemu-kvm \
> -M q35 \
> -cpu SandyBridge \
> -nodefaults -rtc base=utc \
> -m 4G \
> -smp 2,sockets=2,cores=1,threads=1 \
> -enable-kvm \
> -name rhel7 \
> -uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
> -smbios type=1,manufacturer='Red Hat',product='RHEV
> Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-
> e63fcdb67170 \
> -k en-us \
> -serial unix:/tmp/serial0,server,nowait \
> -boot menu=on \
> -bios /usr/share/seabios/bios.bin \
> -vga std \
> -vnc :1 \
> -qmp tcp:0:6666,server,nowait \
> -chardev file,path=/home/bug/seabios.log,id=seabios \
> -device isa-debugcon,chardev=seabios,iobase=0x402 \
> -device ioh3420,bus=pcie.0,id=root.0,slot=1 \
> -device x3130-upstream,bus=root.0,id=upstream1 \
> -device xio3130-downstream,bus=upstream1,id=downstream1,chassis=1 \
> -drive
> file=/home/bug/1272939/pxb.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,
> cache=none,werror=stop,rerror=stop,aio=threads \
> -device
> virtio-blk-pci,bus=downstream1,drive=drive-virtio-disk0,id=virtio-disk0,
> disable-modern=off,bootindex=0 \
> -device xio3130-downstream,bus=upstream1,id=downstream2,chassis=2 \
> -device virtio-scsi-pci,bus=downstream2,id=scsi_pci_bus0,disable-modern=off \
> -drive
> file=/home/bug/1272759/test.qcow2,if=none,id=virtio-scsi,format=qcow2,
> cache=none,werror=stop,rerror=stop,aio=threads \
> -device scsi-hd,bus=scsi_pci_bus0.0,drive=virtio-scsi,id=virtio-disk1 \
> -device xio3130-downstream,bus=upstream1,id=downstream3,chassis=3 \
> -device
> virtio-net-pci,bus=downstream3,netdev=tap10,mac=9a:6a:6b:6c:6d:6e,disable-
> modern=off \
> -netdev tap,id=tap10 \
> -monitor stdio \
> 
> 2.Check the info in hmp
> 
> (qemu) info qtree
> bus: main-system-bus
>   type System
>   dev: kvm-ioapic, id ""
>     gpio-in "" 24
>     gsi_base = 0 (0x0)
>     mmio 00000000fec00000/0000000000001000
>   dev: q35-pcihost, id ""
>     MCFG = 2952790016 (0xb0000000)
>     pci-hole64-size = 0 (0 B)
>     short_root_bus = 0 (0x0)
>     bus: pcie.0
>       type PCIE
>       dev: ioh3420, id "root.0"
>         power_controller_present = true
>         chassis = 0 (0x0)
>         slot = 1 (0x1)
>         port = 0 (0x0)
>         aer_log_max = 8 (0x8)
>         addr = 02.0
>         romfile = ""
>         rombar = 1 (0x1)
>         multifunction = false
>         command_serr_enable = true
>         class PCI bridge, addr 00:02.0, pci id 8086:3420 (sub 0000:0000)
>         bus: root.0
>           type PCIE
>           dev: x3130-upstream, id "upstream1"
>             port = 0 (0x0)
>             aer_log_max = 8 (0x8)
>             addr = 00.0
>             romfile = ""
>             rombar = 1 (0x1)
>             multifunction = false
>             command_serr_enable = true
>             class PCI bridge, addr 01:00.0, pci id 104c:8232 (sub 0000:0000)
>             bus: upstream1
>               type PCIE
>               dev: xio3130-downstream, id "downstream3"
>                 power_controller_present = true
>                 chassis = 3 (0x3)
>                 slot = 0 (0x0)
>                 port = 0 (0x0)
>                 aer_log_max = 8 (0x8)
>                 addr = 02.0
>                 romfile = ""
>                 rombar = 1 (0x1)
>                 multifunction = false
>                 command_serr_enable = true
>                 class PCI bridge, addr 02:02.0, pci id 104c:8233 (sub
> 0000:0000)
>                 bus: downstream3
>                   type PCIE
>                   dev: virtio-net-pci, id ""
>                     ioeventfd = false
>                     vectors = 3 (0x3)
>                     virtio-pci-bus-master-bug-migration = true
>                     disable-legacy = false
>                     disable-modern = false
>                     migrate-extra = true
>                     modern-pio-notify = false
>                     x-disable-pcie = false
>                     addr = 00.0
>                     romfile = "pxe-virtio.rom"
>                     rombar = 1 (0x1)
>                     multifunction = false
>                     command_serr_enable = true
>                     class Ethernet controller, addr 05:00.0, pci id
> 1af4:1000 (sub 1af4:0001)
>                     bar 0: i/o at 0xc000 [0xc01f]
>                     bar 1: mem at 0xfe440000 [0xfe440fff]
>                     bar 4: mem at 0xfc000000 [0xfc7fffff]
>                     bar 6: mem at 0xffffffffffffffff [0x3fffe]
>                     bus: virtio-bus
>                       type virtio-pci-bus
>                       dev: virtio-net-device, id ""
>                         csum = true
>                         guest_csum = true
>                         gso = true
>                         guest_tso4 = true
>                         guest_tso6 = true
>                         guest_ecn = true
>                         guest_ufo = true
>                         guest_announce = true
>                         host_tso4 = true
>                         host_tso6 = true
>                         host_ecn = true
>                         host_ufo = true
>                         mrg_rxbuf = true
>                         status = true
>                         ctrl_vq = true
>                         ctrl_rx = true
>                         ctrl_vlan = true
>                         ctrl_rx_extra = true
>                         ctrl_mac_addr = true
>                         ctrl_guest_offloads = true
>                         mq = false
>                         mac = "9a:6a:6b:6c:6d:6e"
>                         vlan = <null>
>                         netdev = "tap10"
>                         x-txtimer = 150000 (0x249f0)
>                         x-txburst = 256 (0x100)
>                         tx = ""
>                         indirect_desc = true
>                         event_idx = true
>                         notify_on_empty = true
>                         any_layout = true
>               dev: xio3130-downstream, id "downstream2"
>                 power_controller_present = true
>                 chassis = 2 (0x2)
>                 slot = 0 (0x0)
>                 port = 0 (0x0)
>                 aer_log_max = 8 (0x8)
>                 addr = 01.0
>                 romfile = ""
>                 rombar = 1 (0x1)
>                 multifunction = false
>                 command_serr_enable = true
>                 class PCI bridge, addr 02:01.0, pci id 104c:8233 (sub
> 0000:0000)
>                 bus: downstream2
>                   type PCIE
>                   dev: virtio-scsi-pci, id "scsi_pci_bus0"
>                     ioeventfd = true
>                     vectors = 4 (0x4)
>                     virtio-pci-bus-master-bug-migration = true
>                     disable-legacy = false
>                     disable-modern = false
>                     migrate-extra = true
>                     modern-pio-notify = false
>                     x-disable-pcie = false
>                     addr = 00.0
>                     romfile = ""
>                     rombar = 1 (0x1)
>                     multifunction = false
>                     command_serr_enable = true
>                     class SCSI controller, addr 04:00.0, pci id 1af4:1004
> (sub 1af4:0008)
>                     bar 0: i/o at 0xd000 [0xd03f]
>                     bar 1: mem at 0xfe600000 [0xfe600fff]
>                     bar 4: mem at 0xfc800000 [0xfcffffff]
>                     bus: virtio-bus
>                       type virtio-pci-bus
>                       dev: virtio-scsi-device, id ""
>                         num_queues = 1 (0x1)
>                         max_sectors = 65535 (0xffff)
>                         cmd_per_lun = 128 (0x80)
>                         hotplug = true
>                         param_change = true
>                         indirect_desc = true
>                         event_idx = true
>                         notify_on_empty = true
>                         any_layout = true
>                         bus: scsi_pci_bus0.0
>                           type SCSI
>                           dev: scsi-hd, id "virtio-disk1"
>                             drive = "virtio-scsi"
>                             logical_block_size = 512 (0x200)
>                             physical_block_size = 512 (0x200)
>                             min_io_size = 0 (0x0)
>                             opt_io_size = 0 (0x0)
>                             discard_granularity = 4096 (0x1000)
>                             ver = "2.5+"
>                             serial = ""
>                             vendor = "QEMU"
>                             product = "QEMU HARDDISK"
>                             removable = false
>                             dpofua = false
>                             wwn = 0 (0x0)
>                             port_wwn = 0 (0x0)
>                             port_index = 0 (0x0)
>                             max_unmap_size = 1073741824 (0x40000000)
>                             max_io_size = 2147483647 (0x7fffffff)
>                             cyls = 2080 (0x820)
>                             heads = 16 (0x10)
>                             secs = 63 (0x3f)
>                             channel = 0 (0x0)
>                             scsi-id = 0 (0x0)
>                             lun = 0 (0x0)
>               dev: xio3130-downstream, id "downstream1"
>                 power_controller_present = true
>                 chassis = 1 (0x1)
>                 slot = 0 (0x0)
>                 port = 0 (0x0)
>                 aer_log_max = 8 (0x8)
>                 addr = 00.0
>                 romfile = ""
>                 rombar = 1 (0x1)
>                 multifunction = false
>                 command_serr_enable = true
>                 class PCI bridge, addr 02:00.0, pci id 104c:8233 (sub
> 0000:0000)
>                 bus: downstream1
>                   type PCIE
>                   dev: virtio-blk-pci, id "virtio-disk0"
>                     class = 0 (0x0)
>                     ioeventfd = true
>                     vectors = 2 (0x2)
>                     virtio-pci-bus-master-bug-migration = true
>                     disable-legacy = false
>                     disable-modern = false
>                     migrate-extra = true
>                     modern-pio-notify = false
>                     x-disable-pcie = false
>                     addr = 00.0
>                     romfile = ""
>                     rombar = 1 (0x1)
>                     multifunction = false
>                     command_serr_enable = true
>                     class SCSI controller, addr 03:00.0, pci id 1af4:1001
> (sub 1af4:0002)
>                     bar 0: i/o at 0xe000 [0xe03f]
>                     bar 1: mem at 0xfe800000 [0xfe800fff]
>                     bar 4: mem at 0xfd000000 [0xfd7fffff]
>                     bus: virtio-bus
>                       type virtio-pci-bus
>                       dev: virtio-blk-device, id ""
>                         drive = "drive-virtio-disk0"
>                         logical_block_size = 512 (0x200)
>                         physical_block_size = 512 (0x200)
>                         min_io_size = 0 (0x0)
>                         opt_io_size = 0 (0x0)
>                         discard_granularity = 4294967295 (0xffffffff)
>                         cyls = 16383 (0x3fff)
>                         heads = 16 (0x10)
>                         secs = 63 (0x3f)
>                         serial = ""
>                         config-wce = true
>                         scsi = false
>                         request-merging = true
>                         indirect_desc = true
>                         event_idx = true
>                         notify_on_empty = true
>                         any_layout = true
>       dev: VGA, id ""
>         vgamem_mb = 16 (0x10)
>         mmio = true
>         qemu-extended-regs = false
>         addr = 01.0
>         romfile = "vgabios-stdvga.bin"
>         rombar = 1 (0x1)
>         multifunction = false
>         command_serr_enable = true
>         class VGA controller, addr 00:01.0, pci id 1234:1111 (sub 1af4:1100)
>         bar 0: mem at 0xfb000000 [0xfbffffff]
>         bar 2: mem at 0xfea10000 [0xfea10fff]
>         bar 6: mem at 0xffffffffffffffff [0xfffe]
> 
> 
> 
> 3. Check the capability in the guest through "lspci -vnn" and we can got the
> info like "Capabilities: [40] Express Endpoint, MSI 00"
> 
> 03:00.0 SCSI storage controller [0100]: Red Hat, Inc Virtio block device
> [1af4:1001]
> 	Subsystem: Red Hat, Inc Device [1af4:0002]
> 	Physical Slot: 0
> 	Flags: bus master, fast devsel, latency 0, IRQ 22
> 	I/O ports at e000 [size=64]
> 	Memory at fe800000 (32-bit, non-prefetchable) [size=4K]
> 	Memory at fd000000 (64-bit, prefetchable) [size=8M]
> 	Capabilities: [dc] MSI-X: Enable+ Count=2 Masked-
> 	Capabilities: [c8] Vendor Specific Information: Len=14 <?>
> 	Capabilities: [b4] Vendor Specific Information: Len=14 <?>
> 	Capabilities: [a4] Vendor Specific Information: Len=10 <?>
> 	Capabilities: [94] Vendor Specific Information: Len=10 <?>
> 	Capabilities: [84] Vendor Specific Information: Len=10 <?>
> 	Capabilities: [7c] Power Management version 3
> 	Capabilities: [40] Express Endpoint, MSI 00
> 	Kernel driver in use: virtio-pci
> 
> 04:00.0 SCSI storage controller [0100]: Red Hat, Inc Virtio SCSI [1af4:1004]
> 	Subsystem: Red Hat, Inc Device [1af4:0008]
> 	Physical Slot: 0-1
> 	Flags: bus master, fast devsel, latency 0, IRQ 23
> 	I/O ports at d000 [size=64]
> 	Memory at fe600000 (32-bit, non-prefetchable) [size=4K]
> 	Memory at fc800000 (64-bit, prefetchable) [size=8M]
> 	Capabilities: [dc] MSI-X: Enable+ Count=4 Masked-
> 	Capabilities: [c8] Vendor Specific Information: Len=14 <?>
> 	Capabilities: [b4] Vendor Specific Information: Len=14 <?>
> 	Capabilities: [a4] Vendor Specific Information: Len=10 <?>
> 	Capabilities: [94] Vendor Specific Information: Len=10 <?>
> 	Capabilities: [84] Vendor Specific Information: Len=10 <?>
> 	Capabilities: [7c] Power Management version 3
> 	Capabilities: [40] Express Endpoint, MSI 00
> 	Kernel driver in use: virtio-pci
> 
> 05:00.0 Ethernet controller [0200]: Red Hat, Inc Virtio network device
> [1af4:1000]
> 	Subsystem: Red Hat, Inc Device [1af4:0001]
> 	Physical Slot: 0-2
> 	Flags: bus master, fast devsel, latency 0, IRQ 20
> 	I/O ports at c000 [size=32]
> 	Memory at fe440000 (32-bit, non-prefetchable) [size=4K]
> 	Memory at fc000000 (64-bit, prefetchable) [size=8M]
> 	Expansion ROM at fe400000 [disabled] [size=256K]
> 	Capabilities: [dc] MSI-X: Enable+ Count=3 Masked-
> 	Capabilities: [c8] Vendor Specific Information: Len=14 <?>
> 	Capabilities: [b4] Vendor Specific Information: Len=14 <?>
> 	Capabilities: [a4] Vendor Specific Information: Len=10 <?>
> 	Capabilities: [94] Vendor Specific Information: Len=10 <?>
> 	Capabilities: [84] Vendor Specific Information: Len=10 <?>
> 	Capabilities: [7c] Power Management version 3
> 	Capabilities: [40] Express Endpoint, MSI 00
> 	Kernel driver in use: virtio-pci
> 
> 4. In guest, make sure the virtio disk can work correctly

Looks good! As we can see, virtio devices have the PCIe capability if connected to switch downstream ports. You should also check the devices connected directly to a root port.

Thanks,
Marcel
Comment 7 jingzhao 2016-05-23 02:31:21 UTC 
As comment 5, why we need to check the devices connected to a root port directly with "disable-modern=off"? I remember enable the PCIe capability when connected to switch downstream ports with "disable-modern=off". Also, I check the devices connected to a root port with "disable-modern=off" and there are no difference with connect to switch downstream. It's right?

Thanks
Jing
Comment 8 Marcel Apfelbaum 2016-05-23 14:27:33 UTC 
(In reply to jingzhao from comment #7)
> As comment 5, why we need to check the devices connected to a root port
> directly with "disable-modern=off"? I remember enable the PCIe capability
> when connected to switch downstream ports with "disable-modern=off". Also, I
> check the devices connected to a root port with "disable-modern=off" and
> there are no difference with connect to switch downstream. It's right?
> 

Indeed there is no difference between connecting a device to a downstream port
or a root port. So we should check and if there is a difference we should file a BZ.

> Thanks
> Jing
Comment 9 jingzhao 2016-05-24 01:00:14 UTC 
(In reply to Marcel Apfelbaum from comment #8)
> (In reply to jingzhao from comment #7)
> > As comment 5, why we need to check the devices connected to a root port
> > directly with "disable-modern=off"? I remember enable the PCIe capability
> > when connected to switch downstream ports with "disable-modern=off". Also, I
> > check the devices connected to a root port with "disable-modern=off" and
> > there are no difference with connect to switch downstream. It's right?
> > 
> 
> Indeed there is no difference between connecting a device to a downstream
> port
> or a root port. So we should check and if there is a difference we should
> file a BZ.
> 
As I understanding, we can connect to switch downstream or root port with "disable-modern=off" when we enable the PCI-e capability, and it's the better choice when connect to switch downstream with "disable-modern=off". it's right?


> > Thanks
> > Jing
Comment 10 Marcel Apfelbaum 2016-05-26 14:08:48 UTC 
(In reply to jingzhao from comment #9)
> (In reply to Marcel Apfelbaum from comment #8)
> > (In reply to jingzhao from comment #7)
> > > As comment 5, why we need to check the devices connected to a root port
> > > directly with "disable-modern=off"? I remember enable the PCIe capability
> > > when connected to switch downstream ports with "disable-modern=off". Also, I
> > > check the devices connected to a root port with "disable-modern=off" and
> > > there are no difference with connect to switch downstream. It's right?
> > > 
> > 
> > Indeed there is no difference between connecting a device to a downstream
> > port
> > or a root port. So we should check and if there is a difference we should
> > file a BZ.
> > 
> As I understanding, we can connect to switch downstream or root port with
> "disable-modern=off" when we enable the PCI-e capability, and it's the
> better choice when connect to switch downstream with "disable-modern=off".
> it's right?
> 

Correct. You enable virtio 1.0 that way.
Thanks,
Marcel

> 
> > > Thanks
> > > Jing
Comment 13 errata-xmlrpc 2016-11-07 20:45:50 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2673.html
Note You need to log in before you can comment on or make changes to this bug.