Bug 1242479

Testing: sanity checking is sufficient. The same testing as for bug 1103313 is enough; ie. just check that this backport doesn't regress the fix for bug 1103313.

(I'm obviously checking these patches more extensively, but that requires such OVMF patches on the guest side that are pending review even for upstream.)

Fix included in qemu-kvm-rhev-2.3.0-11.el7

Hi Laszlo,


as the OVMF still not recognizes devices behind PXB, so I'm trying booting device on pci.0 and check pci roots

ls -l /sys/devices/

could see pci0000:00 pci:0000:07 pci:0000:0b pci0000:0f etc

so the qemu emulation working correctly
and before this OVMF bug gets fixed, I think there's not possible to boot from such configuration(device behind PXB)
Bug 1193080 - RFE: Support multiple PCI root buses


Since this bug is about qemu-kvm-rhev emulation, as it's now working, as below cli, is that suffice to VERIFIED this bug ?
if any further test needed, please let me know


cat pxb-ovmf.sh 
ISO=/home/kvm_autotest_root/iso/linux/RHEL7.2-Server-x86_64.iso
CODE=/usr/share/OVMF/OVMF_CODE.fd
TMPL=/root/ovmf-pxb.fd
TFTP=/var/lib/dnsmasq
BF=/boot/efi/EFI/redhat/shim.efi

cp $TMPL vars.fd

/usr/libexec/qemu-kvm \
  -S -vnc :0 \
  -m 2048 \
  -M pc \
  -enable-kvm \
  -device qxl-vga \
  -drive if=pflash,readonly,format=raw,file=$CODE \
  -drive if=pflash,format=raw,file=vars.fd \
  -drive id=cdrom,if=none,readonly,format=raw,cache=writethrough,file=$ISO \
  \
  -device pxb,bus=pci.0,id=bridge1,bus_nr=11 \
  \
  -device virtio-scsi-pci,id=scsi0,bus=pci.0 \
# use bus=bridge1 is not bootable , the OVMF do not recognize devices behind pxb
\
\
\
\
\
\

  -device scsi-cd,bus=scsi0.0,drive=cdrom,bootindex=1 \
  -debugcon file:debug.log \
  -global isa-debugcon.iobase=0x402 \
  \
  -monitor stdio \
  \
  -netdev user,id=netdev0,hostfwd=tcp:127.0.0.1:2222-:22,tftp=$TFTP,bootfile=$BF \
  -device virtio-net-pci,netdev=netdev0,bus=bridge1,addr=2,romfile=,bootindex=0 \
  \
  -device pxb,bus=pci.0,id=bridge2,bus_nr=7 \
  -netdev user,id=netdev1,hostfwd=tcp:127.0.0.1:2223-:22 \
  -device virtio-net-pci,netdev=netdev1,bus=bridge2,addr=3,romfile=,bootindex=2 \
  \
  -device pxb,bus=pci.0,id=bridge3,bus_nr=15 \
  -netdev user,id=netdev2,hostfwd=tcp:127.0.0.1:2224-:22 \
  -device virtio-net-pci,netdev=netdev2,bus=bridge3,addr=4,romfile=,bootindex=3 \

Hi Xiaoqing,

so there are two possibilities for testing this bug (see comment 5 too):

(1) Upstream OVMF contains the feature tracked by bug 1193080 now. And, it so happens that for completely unrelated reasons, I uploaded a fresh upstream OVMF build to this location yesterday:

http://people.redhat.com/~lersek/ovmf-r18063-ad-hoc-4530eb75/

If you wish, you can verify qemu-kvm-rhev-2.3.0-11.el7 with that OVMF binary functionally (using the test script that you already found & customized from comment 6 --> bug 1193080 comment 15).

(2) The other possibility is just regression testing / sanity testing. If that's what you prefer, then this bug should not be tested separately. Instead, bug 1103313 should be verified, however not against qemu-kvm-rhev-2.3.0-9.el7, but against qemu-kvm-rhev-2.3.0-11.el7. That would prove that these patches have not regressed the fixes for bug 1103313, and for sanity testing, that suffices.

I'll let you decide which option you should pick. Thanks!

Hi Laszlo,

Thanks for the explain and the scratch OVMF build, I've done the (1) in C#11,
and it works fine

on host with:
kernel-3.10.0-297.el7.x86_64
qemu-kvm-rhev-2.3.0-12.el7.x86_64
and guest with:
kernel-3.10.0-229.el7.x86_64

plus the OVMF build you provided( http://people.redhat.com/~lersek/ovmf-r18063-ad-hoc-4530eb75/ )

guest boot into installer successfully, and also sees multiple pci roots,

So this bug has been fixed correctly.

I'll post the 'info qtree' 'info pci' and cli in next comment

# cat pxb-ovmf.sh 
ISO=/home/kvm_autotest_root/iso/linux/RHEL7.2-Server-x86_64.iso
CODE="OVMF_CODE.fd"
TMPL="OVMF_VARS.fd"
TFTP=/var/lib/dnsmasq
BF=/boot/efi/EFI/redhat/shim.efi

cp $TMPL vars.fd

/usr/libexec/qemu-kvm \
  -S -vnc :0 \
  -m 2048 \
  -M pc \
  -enable-kvm \
  -device qxl-vga \
  -drive if=pflash,readonly,format=raw,file=$CODE \
  -drive if=pflash,format=raw,file=vars.fd \
  -drive id=cdrom,if=none,readonly,format=raw,cache=writethrough,file=$ISO \
  \
  -device pxb,bus=pci.0,id=bridge1,bus_nr=11 \
  \
  -device virtio-scsi-pci,id=scsi0,bus=bridge1 \
  -device scsi-cd,bus=scsi0.0,drive=cdrom,bootindex=1 \
  -debugcon file:debug.log \
  -global isa-debugcon.iobase=0x402 \
  \
  -monitor stdio \
  \
  -netdev user,id=netdev0,hostfwd=tcp:127.0.0.1:2222-:22,tftp=$TFTP,bootfile=$BF \
  -device virtio-net-pci,netdev=netdev0,bus=bridge1,addr=2,romfile=,bootindex=0 \
  \
  -device pxb,bus=pci.0,id=bridge2,bus_nr=7 \
  -netdev user,id=netdev1,hostfwd=tcp:127.0.0.1:2223-:22 \
  -device virtio-net-pci,netdev=netdev1,bus=bridge2,addr=3,romfile=,bootindex=2 \
  \
  -device pxb,bus=pci.0,id=bridge3,bus_nr=15 \
  -netdev user,id=netdev2,hostfwd=tcp:127.0.0.1:2224-:22 \
  -device virtio-net-pci,netdev=netdev2,bus=bridge3,addr=4,romfile=,bootindex=3 

#########################


(qemu) info block
pflash0: OVMF_CODE.fd (raw, read-only)
    Cache mode:       writeback

pflash1: vars.fd (raw)
    Cache mode:       writeback

cdrom: /home/kvm_autotest_root/iso/linux/RHEL7.2-Server-x86_64.iso (raw, read-only)
    Removable device: not locked, tray closed
    Cache mode:       writethrough

floppy0: [not inserted]
    Removable device: not locked, tray closed

sd0: [not inserted]
    Removable device: not locked, tray closed



-----------------------------------------
(qemu) info pci
  Bus 15, device   0, function 0:
    PCI bridge: PCI device 1b36:0001
      BUS 0.
      secondary bus 0.
      subordinate bus 0.
      IO range [0x0000, 0x0fff]
      memory range [0x00000000, 0x000fffff]
      prefetchable memory range [0x00000000, 0x000fffff]
      id "bridge3"
  Bus  7, device   0, function 0:
    PCI bridge: PCI device 1b36:0001
      BUS 0.
      secondary bus 0.
      subordinate bus 0.
      IO range [0x0000, 0x0fff]
      memory range [0x00000000, 0x000fffff]
      prefetchable memory range [0x00000000, 0x000fffff]
      id "bridge2"
  Bus 11, device   0, function 0:
    PCI bridge: PCI device 1b36:0001
      BUS 0.
      secondary bus 0.
      subordinate bus 0.
      IO range [0x0000, 0x0fff]
      memory range [0x00000000, 0x000fffff]
      prefetchable memory range [0x00000000, 0x000fffff]
      id "bridge1"
  Bus  0, device   0, function 0:
    Host bridge: PCI device 8086:1237
      id ""
  Bus  0, device   1, function 0:
    ISA bridge: PCI device 8086:7000
      id ""
  Bus  0, device   1, function 1:
    IDE controller: PCI device 8086:7010
      BAR4: I/O at 0xffffffffffffffff [0x000e].
      id ""
  Bus  0, device   1, function 3:
    Bridge: PCI device 8086:7113
      IRQ 0.
      id ""
  Bus  0, device   2, function 0:
    VGA controller: PCI device 1b36:0100
      IRQ 0.
      BAR0: 32 bit memory at 0xffffffffffffffff [0x03fffffe].
      BAR1: 32 bit memory at 0xffffffffffffffff [0x03fffffe].
      BAR2: 32 bit memory at 0xffffffffffffffff [0x00001ffe].
      BAR3: I/O at 0xffffffffffffffff [0x001e].
      BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe].
      id ""
  Bus  0, device   3, function 0:
    Host bridge: PCI device 1b36:0009
      id "bridge1"
  Bus  0, device   4, function 0:
    Host bridge: PCI device 1b36:0009
      id "bridge2"
  Bus  0, device   5, function 0:
    Host bridge: PCI device 1b36:0009
      id "bridge3"
==========================================


(qemu) info qtree 
bus: main-system-bus
  type System
  dev: pxb-host, id ""
    bus: pxb-internal
      type pxb-bus
      dev: pci-bridge, id "bridge3"
        chassis_nr = 15 (0xf)
        msi = false
        shpc = false
        addr = 00.0
        romfile = ""
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        class PCI bridge, addr 0f:00.0, pci id 1b36:0001 (sub 0000:0000)
        bus: bridge3
          type PCI
          dev: virtio-net-pci, id ""
            ioeventfd = false
            vectors = 3 (0x3)
            virtio-pci-bus-master-bug-migration = false
            disable-legacy = false
            disable-modern = true
            addr = 04.0
            romfile = ""
            rombar = 1 (0x1)
            multifunction = false
            command_serr_enable = true
            class Ethernet controller, addr 00:04.0, pci id 1af4:1000 (sub 1af4:0001)
            bar 0: i/o at 0xffffffffffffffff [0x1e]
            bar 1: mem at 0xffffffffffffffff [0xffe]
            bus: virtio-bus
              type virtio-pci-bus
              dev: virtio-net-device, id ""
                any_layout = true
                csum = true
                guest_csum = true
                gso = true
                guest_tso4 = true
                guest_tso6 = true
                guest_ecn = true
                guest_ufo = true
                guest_announce = true
                host_tso4 = true
                host_tso6 = true
                host_ecn = true
                host_ufo = true
                mrg_rxbuf = true
                status = true
                ctrl_vq = true
                ctrl_rx = true
                ctrl_vlan = true
                ctrl_rx_extra = true
                ctrl_mac_addr = true
                ctrl_guest_offloads = true
                mq = false
                mac = "52:54:00:12:34:58"
                vlan = <null>
                netdev = "netdev2"
                x-txtimer = 150000 (0x249f0)
                x-txburst = 256 (0x100)
                tx = ""
                indirect_desc = true
                event_idx = true
                notify_on_empty = true
  dev: pxb-host, id ""
    bus: pxb-internal
      type pxb-bus
      dev: pci-bridge, id "bridge2"
        chassis_nr = 7 (0x7)
        msi = false
        shpc = false
        addr = 00.0
        romfile = ""
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        class PCI bridge, addr 07:00.0, pci id 1b36:0001 (sub 0000:0000)
        bus: bridge2
          type PCI
          dev: virtio-net-pci, id ""
            ioeventfd = false
            vectors = 3 (0x3)
            virtio-pci-bus-master-bug-migration = false
            disable-legacy = false
            disable-modern = true
            addr = 03.0
            romfile = ""
            rombar = 1 (0x1)
            multifunction = false
            command_serr_enable = true
            class Ethernet controller, addr 00:03.0, pci id 1af4:1000 (sub 1af4:0001)
            bar 0: i/o at 0xffffffffffffffff [0x1e]
            bar 1: mem at 0xffffffffffffffff [0xffe]
            bus: virtio-bus
              type virtio-pci-bus
              dev: virtio-net-device, id ""
                any_layout = true
                csum = true
                guest_csum = true
                gso = true
                guest_tso4 = true
                guest_tso6 = true
                guest_ecn = true
                guest_ufo = true
                guest_announce = true
                host_tso4 = true
                host_tso6 = true
                host_ecn = true
                host_ufo = true
                mrg_rxbuf = true
                status = true
                ctrl_vq = true
                ctrl_rx = true
                ctrl_vlan = true
                ctrl_rx_extra = true
                ctrl_mac_addr = true
                ctrl_guest_offloads = true
                mq = false
                mac = "52:54:00:12:34:57"
                vlan = <null>
                netdev = "netdev1"
                x-txtimer = 150000 (0x249f0)
                x-txburst = 256 (0x100)
                tx = ""
                indirect_desc = true
                event_idx = true
                notify_on_empty = true
  dev: pxb-host, id ""
    bus: pxb-internal
      type pxb-bus
      dev: pci-bridge, id "bridge1"
        chassis_nr = 11 (0xb)
        msi = false
        shpc = false
        addr = 00.0
        romfile = ""
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        class PCI bridge, addr 0b:00.0, pci id 1b36:0001 (sub 0000:0000)
        bus: bridge1
          type PCI
          dev: virtio-net-pci, id ""
            ioeventfd = false
            vectors = 3 (0x3)
            virtio-pci-bus-master-bug-migration = false
            disable-legacy = false
            disable-modern = true
            addr = 02.0
            romfile = ""
            rombar = 1 (0x1)
            multifunction = false
            command_serr_enable = true
            class Ethernet controller, addr 00:02.0, pci id 1af4:1000 (sub 1af4:0001)
            bar 0: i/o at 0xffffffffffffffff [0x1e]
            bar 1: mem at 0xffffffffffffffff [0xffe]
            bus: virtio-bus
              type virtio-pci-bus
              dev: virtio-net-device, id ""
                any_layout = true
                csum = true
                guest_csum = true
                gso = true
                guest_tso4 = true
                guest_tso6 = true
                guest_ecn = true
                guest_ufo = true
                guest_announce = true
                host_tso4 = true
                host_tso6 = true
                host_ecn = true
                host_ufo = true
                mrg_rxbuf = true
                status = true
                ctrl_vq = true
                ctrl_rx = true
                ctrl_vlan = true
                ctrl_rx_extra = true
                ctrl_mac_addr = true
                ctrl_guest_offloads = true
                mq = false
                mac = "52:54:00:12:34:56"
                vlan = <null>
                netdev = "netdev0"
                x-txtimer = 150000 (0x249f0)
                x-txburst = 256 (0x100)
                tx = ""
                indirect_desc = true
                event_idx = true
                notify_on_empty = true
          dev: virtio-scsi-pci, id "scsi0"
            ioeventfd = true
            vectors = 4 (0x4)
            virtio-pci-bus-master-bug-migration = false
            disable-legacy = false
            disable-modern = true
            addr = 00.0
            romfile = ""
            rombar = 1 (0x1)
            multifunction = false
            command_serr_enable = true
            class SCSI controller, addr 00:00.0, pci id 1af4:1004 (sub 1af4:0008)
            bar 0: i/o at 0xffffffffffffffff [0x3e]
            bar 1: mem at 0xffffffffffffffff [0xffe]
            bus: virtio-bus
              type virtio-pci-bus
              dev: virtio-scsi-device, id ""
                num_queues = 1 (0x1)
                max_sectors = 65535 (0xffff)
                cmd_per_lun = 128 (0x80)
                any_layout = true
                hotplug = true
                param_change = true
                indirect_desc = true
                event_idx = true
                notify_on_empty = true
                bus: scsi0.0
                  type SCSI
                  dev: scsi-cd, id ""
                    drive = "cdrom"
                    logical_block_size = 512 (0x200)
                    physical_block_size = 512 (0x200)
                    min_io_size = 0 (0x0)
                    opt_io_size = 0 (0x0)
                    discard_granularity = 4096 (0x1000)
                    ver = "2.3.0"
                    serial = ""
                    vendor = "QEMU"
                    product = "QEMU CD-ROM"
                    wwn = 0 (0x0)
                    port_wwn = 0 (0x0)
                    port_index = 0 (0x0)
                    max_io_size = 2147483647 (0x7fffffff)
                    channel = 0 (0x0)
                    scsi-id = 0 (0x0)
                    lun = 0 (0x0)
  dev: kvm-ioapic, id ""
    gpio-in "" 24
    gsi_base = 0 (0x0)
    mmio 00000000fec00000/0000000000001000
  dev: i440FX-pcihost, id ""
    pci-hole64-size = 18446744073709551615 (16 EiB)
    short_root_bus = 0 (0x0)
    bus: pci.0
      type PCI
      dev: pxb, id "bridge3"
        bus_nr = 15 (0xf)
        numa_node = 128 (0x80)
        addr = 05.0
        romfile = ""
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        class Host bridge, addr 00:05.0, pci id 1b36:0009 (sub 1af4:1100)
      dev: pxb, id "bridge2"
        bus_nr = 7 (0x7)
        numa_node = 128 (0x80)
        addr = 04.0
        romfile = ""
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        class Host bridge, addr 00:04.0, pci id 1b36:0009 (sub 1af4:1100)
      dev: pxb, id "bridge1"
        bus_nr = 11 (0xb)
        numa_node = 128 (0x80)
        addr = 03.0
        romfile = ""
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        class Host bridge, addr 00:03.0, pci id 1b36:0009 (sub 1af4:1100)
      dev: qxl-vga, id ""
        ram_size = 67108864 (0x4000000)
        vram_size = 67108864 (0x4000000)
        revision = 4 (0x4)
        debug = 0 (0x0)
        guestdebug = 0 (0x0)
        cmdlog = 0 (0x0)
        ram_size_mb = 4294967295 (0xffffffff)
        vram_size_mb = 4294967295 (0xffffffff)
        vram64_size_mb = 4294967295 (0xffffffff)
        vgamem_mb = 16 (0x10)
        surfaces = 1024 (0x400)
        addr = 02.0
        romfile = "vgabios-qxl.bin"
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        class VGA controller, addr 00:02.0, pci id 1b36:0100 (sub 1af4:1100)
        bar 0: mem at 0xffffffffffffffff [0x3fffffe]
        bar 1: mem at 0xffffffffffffffff [0x3fffffe]
        bar 2: mem at 0xffffffffffffffff [0x1ffe]
        bar 3: i/o at 0xffffffffffffffff [0x1e]
        bar 6: mem at 0xffffffffffffffff [0xfffe]
      dev: PIIX4_PM, id ""
        smb_io_base = 45312 (0xb100)
        disable_s3 = 1 (0x1)
        disable_s4 = 1 (0x1)
        s4_val = 2 (0x2)
        acpi-pci-hotplug-with-bridge-support = true
        memory-hotplug-support = true
        addr = 01.3
        romfile = ""
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        class Bridge, addr 00:01.3, pci id 8086:7113 (sub 1af4:1100)
        bus: i2c
          type i2c-bus
          dev: smbus-eeprom, id ""
            address = 87 (0x57)
          dev: smbus-eeprom, id ""
            address = 86 (0x56)
          dev: smbus-eeprom, id ""
            address = 85 (0x55)
          dev: smbus-eeprom, id ""
            address = 84 (0x54)
          dev: smbus-eeprom, id ""
            address = 83 (0x53)
          dev: smbus-eeprom, id ""
            address = 82 (0x52)
          dev: smbus-eeprom, id ""
            address = 81 (0x51)
          dev: smbus-eeprom, id ""
            address = 80 (0x50)
      dev: piix3-ide, id ""
        addr = 01.1
        romfile = ""
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        class IDE controller, addr 00:01.1, pci id 8086:7010 (sub 1af4:1100)
        bar 4: i/o at 0xffffffffffffffff [0xe]
        bus: ide.1
          type IDE
        bus: ide.0
          type IDE
      dev: PIIX3, id ""
        addr = 01.0
        romfile = ""
        rombar = 1 (0x1)
        multifunction = true
        command_serr_enable = true
        class ISA bridge, addr 00:01.0, pci id 8086:7000 (sub 1af4:1100)
        bus: isa.0
          type ISA
          dev: isa-debugcon, id "debugcon"
            iobase = 1026 (0x402)
            chardev = "debugcon"
            readback = 233 (0xe9)
          dev: isa-fdc, id ""
            iobase = 1008 (0x3f0)
            irq = 6 (0x6)
            dma = 2 (0x2)
            driveA = "floppy0"
            driveB = ""
            check_media_rate = true
            isa irq 6
          dev: port92, id ""
          dev: vmmouse, id ""
          dev: vmport, id ""
          dev: i8042, id ""
            isa irqs 1,12
          dev: isa-serial, id ""
            index = 0 (0x0)
            iobase = 1016 (0x3f8)
            irq = 4 (0x4)
            chardev = "serial0"
            wakeup = 0 (0x0)
            isa irq 4
          dev: isa-pcspk, id ""
            iobase = 97 (0x61)
          dev: kvm-pit, id ""
            gpio-in "" 1
            iobase = 64 (0x40)
            lost_tick_policy = "delay"
          dev: mc146818rtc, id ""
            base_year = 0 (0x0)
            lost_tick_policy = "discard"
            isa irq 8
          dev: kvm-i8259, id ""
            iobase = 160 (0xa0)
            elcr_addr = 1233 (0x4d1)
            elcr_mask = 222 (0xde)
            master = false
          dev: kvm-i8259, id ""
            iobase = 32 (0x20)
            elcr_addr = 1232 (0x4d0)
            elcr_mask = 248 (0xf8)
            master = true
      dev: i440FX, id ""
        addr = 00.0
        romfile = ""
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        class Host bridge, addr 00:00.0, pci id 8086:1237 (sub 1af4:1100)
  dev: fw_cfg_io, id ""
    iobase = 1296 (0x510)
  dev: cfi.pflash01, id ""
    drive = "pflash1"
    num-blocks = 32 (0x20)
    sector-length = 4096 (0x1000)
    width = 1 (0x1)
    device-width = 0 (0x0)
    max-device-width = 0 (0x0)
    big-endian = 0 (0x0)
    id0 = 0 (0x0)
    id1 = 0 (0x0)
    id2 = 0 (0x0)
    id3 = 0 (0x0)
    name = "system.flash1"
    mmio 00000000ffe00000/0000000000020000
  dev: cfi.pflash01, id ""
    drive = "pflash0"
    num-blocks = 480 (0x1e0)
    sector-length = 4096 (0x1000)
    width = 1 (0x1)
    device-width = 0 (0x0)
    max-device-width = 0 (0x0)
    big-endian = 0 (0x0)
    id0 = 0 (0x0)
    id1 = 0 (0x0)
    id2 = 0 (0x0)
    id3 = 0 (0x0)
    name = "system.flash0"
    mmio 00000000ffe20000/00000000001e0000
  dev: kvmclock, id ""
  dev: kvmvapic, id ""
  dev: icc-bridge, id ""
    mmio 00000000fee00000/0000000000100000
    bus: icc
      type icc-bus
      dev: kvm-apic, id ""
        id = 0 (0x0)
        version = 20 (0x14)
        vapic = true
      dev: qemu64-x86_64-cpu, id ""
        pmu = false
        hv-spinlocks = -1 (0xffffffffffffffff)
        hv-relaxed = false
        hv-vapic = false
        hv-time = false
        check = false
        enforce = false
        kvm = true

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2546.html