Bug 1671902 - Out of I/O address space when boot up with 15 e1000e NICs via pcie-root-port
Summary: Out of I/O address space when boot up with 15 e1000e NICs via pcie-root-port
Status: NEW
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: seabios
Version: 8.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: Michael S. Tsirkin
QA Contact: Pei Zhang
URL:
Whiteboard:
Keywords:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-02-02 04:04 UTC by Yiqian Wei
Modified: 2019-06-11 21:32 UTC (History)
9 users (show)

(edit)
Clone Of:
(edit)
Last Closed:


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Bugzilla 1271457 None CLOSED seabios has no output if boot guest with more than 8 pci-bridge disks. 2019-06-12 01:26 UTC
Red Hat Bugzilla 1344299 None CLOSED PCIe: Add an option to PCIe ports to disable IO port space support 2019-06-12 01:26 UTC

Description Yiqian Wei 2019-02-02 04:04:01 UTC
Description of problem:
Out of I/O address space when boot up with 15 e1000e NICs via pcie-root-port

Version-Release number of selected component (if applicable):
host version:
qemu-kvm-3.1.0-10.module+el8+2732+3228f155.x86_64
kernel-4.18.0-64.el8.x86_64
seabios-1.12.0-1.module+el8+2706+3c6581b6.x86_64
guest:rhel8

How reproducible:
1/1

Steps to Reproduce:
1.boot guest as the following via nics.sh shell script
# sh nics.sh e1000e
QEMU 3.1.0 monitor - type 'help' for more information
(qemu)

cat nics.sh
#!/bin/sh
if [ "$1" == "e1000e" ]
then
	NIC_TYPE=e1000e
elif [ "$1" == "virtio" ]
then
	NIC_TYPE=virtio-net-pci
else
        echo "nics.sh < e1000e | virtio >"
        exit 0
fi

MEM=4G
SMP=4
VGA=qxl
FMT=qcow2
CPU=Broadwell-noTSX
SYSDISK=/home/sca/rhel8.qcow2
CLI="	/usr/libexec/qemu-kvm \
	-M q35,accel=kvm,kernel-irqchip=split \
	-m $MEM   \
	-cpu $CPU,enforce \
	-smp $SMP,sockets=2,cores=2,threads=1 \
	-vga $VGA \
	-device intel-iommu,intremap=on,caching-mode=on,device-iotlb=on \
	-blockdev driver=$FMT,file.driver=file,node-name=drive_sysdisk,cache.no-flush=off,cache.direct=on,file.filename=$SYSDISK \
	-device ide-drive,bus=ide.0,unit=0,drive=drive_sysdisk,id=device_sysdisk,bootindex=1 \
	-rtc base=utc \
	-enable-kvm \
	-nodefaults \
	-k en-us \
	-chardev file,path=/home/seabios.log,id=seabios \
	-device isa-debugcon,chardev=seabios,iobase=0x402 \
	-boot menu=on \
	-serial unix:/tmp/console,server,nowait \
	-qmp tcp:0:6668,server,nowait \
	-monitor stdio \
	-spice port=5901,disable-ticketing"

w=1
while [ $w -lt 16 ]
	do
		CLI="$CLI -device pcie-root-port,bus=pcie.0,id=root.$w,slot=$w"
            	CLI="$CLI -netdev tap,id=tap$w,vhost=on -device $NIC_TYPE,netdev=tap$w,mac=9a:6a:6b:6c:6d:$(printf "%02x" $w),id=$NIC_TYPE-$w,bus=root.$w"
		w=$(($w+1))
	done	
$CLI

Actual results:
guest didn't boot up and using isa-debugcon debug, got the following messages:
[root@lenovo-sr630-04 ~]# cat /home/seabios.log 
SeaBIOS (version 1.12.0-1.module+el8+2706+3c6581b6)
BUILD: gcc: (GCC) 8.2.1 20180905 (Red Hat 8.2.1-3) binutils: version 2.30-49.el8
No Xen hypervisor found.
Running on QEMU (q35)
Running on KVM
RamSize: 0x80000000 [cmos]
Relocating init from 0x000d71e0 to 0x7ffac3c0 (size 80800)
Found QEMU fw_cfg
QEMU fw_cfg DMA interface supported
RamBlock: addr 0x0000000000000000 len 0x0000000080000000 [e820]
RamBlock: addr 0x0000000100000000 len 0x0000000080000000 [e820]
Moving pm_base to 0x600
boot order:
1: /pci@i0cf8/pci8086,2922@1f,2/drive@0/disk@0
=== PCI bus & bridge init ===
PCI: pci_bios_init_bus_rec bus = 0x0
PCI: pci_bios_init_bus_rec bdf = 0x8
PCI: primary bus = 0x0
PCI: secondary bus = 0xff -> 0x1
PCI: pci_bios_init_bus_rec bus = 0x1
PCI: QEMU resource reserve cap not found
PCI: subordinate bus = 0x0 -> 0x1
PCI: pci_bios_init_bus_rec bdf = 0x10
PCI: primary bus = 0x0
PCI: secondary bus = 0xff -> 0x2
PCI: pci_bios_init_bus_rec bus = 0x2
PCI: QEMU resource reserve cap not found
PCI: subordinate bus = 0x0 -> 0x2
PCI: pci_bios_init_bus_rec bdf = 0x18
PCI: primary bus = 0x0
PCI: secondary bus = 0xff -> 0x3
PCI: pci_bios_init_bus_rec bus = 0x3
PCI: QEMU resource reserve cap not found
PCI: subordinate bus = 0x0 -> 0x3
PCI: pci_bios_init_bus_rec bdf = 0x20
PCI: primary bus = 0x0
PCI: secondary bus = 0xff -> 0x4
PCI: pci_bios_init_bus_rec bus = 0x4
PCI: QEMU resource reserve cap not found
PCI: subordinate bus = 0x0 -> 0x4
PCI: pci_bios_init_bus_rec bdf = 0x28
PCI: primary bus = 0x0
PCI: secondary bus = 0xff -> 0x5
PCI: pci_bios_init_bus_rec bus = 0x5
PCI: QEMU resource reserve cap not found
PCI: subordinate bus = 0x0 -> 0x5
PCI: pci_bios_init_bus_rec bdf = 0x30
PCI: primary bus = 0x0
PCI: secondary bus = 0xff -> 0x6
PCI: pci_bios_init_bus_rec bus = 0x6
PCI: QEMU resource reserve cap not found
PCI: subordinate bus = 0x0 -> 0x6
PCI: pci_bios_init_bus_rec bdf = 0x38
PCI: primary bus = 0x0
PCI: secondary bus = 0xff -> 0x7
PCI: pci_bios_init_bus_rec bus = 0x7
PCI: QEMU resource reserve cap not found
PCI: subordinate bus = 0x0 -> 0x7
PCI: pci_bios_init_bus_rec bdf = 0x40
PCI: primary bus = 0x0
PCI: secondary bus = 0xff -> 0x8
PCI: pci_bios_init_bus_rec bus = 0x8
PCI: QEMU resource reserve cap not found
PCI: subordinate bus = 0x0 -> 0x8
PCI: pci_bios_init_bus_rec bdf = 0x48
PCI: primary bus = 0x0
PCI: secondary bus = 0xff -> 0x9
PCI: pci_bios_init_bus_rec bus = 0x9
PCI: QEMU resource reserve cap not found
PCI: subordinate bus = 0x0 -> 0x9
PCI: pci_bios_init_bus_rec bdf = 0x50
PCI: primary bus = 0x0
PCI: secondary bus = 0xff -> 0xa
PCI: pci_bios_init_bus_rec bus = 0xa
PCI: QEMU resource reserve cap not found
PCI: subordinate bus = 0x0 -> 0xa
PCI: pci_bios_init_bus_rec bdf = 0x58
PCI: primary bus = 0x0
PCI: secondary bus = 0xff -> 0xb
PCI: pci_bios_init_bus_rec bus = 0xb
PCI: QEMU resource reserve cap not found
PCI: subordinate bus = 0x0 -> 0xb
PCI: pci_bios_init_bus_rec bdf = 0x60
PCI: primary bus = 0x0
PCI: secondary bus = 0xff -> 0xc
PCI: pci_bios_init_bus_rec bus = 0xc
PCI: QEMU resource reserve cap not found
PCI: subordinate bus = 0x0 -> 0xc
PCI: pci_bios_init_bus_rec bdf = 0x68
PCI: primary bus = 0x0
PCI: secondary bus = 0xff -> 0xd
PCI: pci_bios_init_bus_rec bus = 0xd
PCI: QEMU resource reserve cap not found
PCI: subordinate bus = 0x0 -> 0xd
PCI: pci_bios_init_bus_rec bdf = 0x70
PCI: primary bus = 0x0
PCI: secondary bus = 0xff -> 0xe
PCI: pci_bios_init_bus_rec bus = 0xe
PCI: QEMU resource reserve cap not found
PCI: subordinate bus = 0x0 -> 0xe
PCI: pci_bios_init_bus_rec bdf = 0x78
PCI: primary bus = 0x0
PCI: secondary bus = 0xff -> 0xf
PCI: pci_bios_init_bus_rec bus = 0xf
PCI: QEMU resource reserve cap not found
PCI: subordinate bus = 0x0 -> 0xf
=== PCI device probing ===
Found 34 PCI devices (max PCI bus is 0f)
=== PCI new allocation pass #1 ===
PCI: check devices
PCI: QEMU resource reserve cap not found
PCI: secondary bus 15 size 00001000 type io
PCI: secondary bus 15 size 00200000 type mem
PCI: secondary bus 15 size 00200000 type prefmem
PCI: QEMU resource reserve cap not found
PCI: secondary bus 14 size 00001000 type io
PCI: secondary bus 14 size 00200000 type mem
PCI: secondary bus 14 size 00200000 type prefmem
PCI: QEMU resource reserve cap not found
PCI: secondary bus 13 size 00001000 type io
PCI: secondary bus 13 size 00200000 type mem
PCI: secondary bus 13 size 00200000 type prefmem
PCI: QEMU resource reserve cap not found
PCI: secondary bus 12 size 00001000 type io
PCI: secondary bus 12 size 00200000 type mem
PCI: secondary bus 12 size 00200000 type prefmem
PCI: QEMU resource reserve cap not found
PCI: secondary bus 11 size 00001000 type io
PCI: secondary bus 11 size 00200000 type mem
PCI: secondary bus 11 size 00200000 type prefmem
PCI: QEMU resource reserve cap not found
PCI: secondary bus 10 size 00001000 type io
PCI: secondary bus 10 size 00200000 type mem
PCI: secondary bus 10 size 00200000 type prefmem
PCI: QEMU resource reserve cap not found
PCI: secondary bus 9 size 00001000 type io
PCI: secondary bus 9 size 00200000 type mem
PCI: secondary bus 9 size 00200000 type prefmem
PCI: QEMU resource reserve cap not found
PCI: secondary bus 8 size 00001000 type io
PCI: secondary bus 8 size 00200000 type mem
PCI: secondary bus 8 size 00200000 type prefmem
PCI: QEMU resource reserve cap not found
PCI: secondary bus 7 size 00001000 type io
PCI: secondary bus 7 size 00200000 type mem
PCI: secondary bus 7 size 00200000 type prefmem
PCI: QEMU resource reserve cap not found
PCI: secondary bus 6 size 00001000 type io
PCI: secondary bus 6 size 00200000 type mem
PCI: secondary bus 6 size 00200000 type prefmem
PCI: QEMU resource reserve cap not found
PCI: secondary bus 5 size 00001000 type io
PCI: secondary bus 5 size 00200000 type mem
PCI: secondary bus 5 size 00200000 type prefmem
PCI: QEMU resource reserve cap not found
PCI: secondary bus 4 size 00001000 type io
PCI: secondary bus 4 size 00200000 type mem
PCI: secondary bus 4 size 00200000 type prefmem
PCI: QEMU resource reserve cap not found
PCI: secondary bus 3 size 00001000 type io
PCI: secondary bus 3 size 00200000 type mem
PCI: secondary bus 3 size 00200000 type prefmem
PCI: QEMU resource reserve cap not found
PCI: secondary bus 2 size 00001000 type io
PCI: secondary bus 2 size 00200000 type mem
PCI: secondary bus 2 size 00200000 type prefmem
PCI: QEMU resource reserve cap not found
PCI: secondary bus 1 size 00001000 type io
PCI: secondary bus 1 size 00200000 type mem
PCI: secondary bus 1 size 00200000 type prefmem
=== PCI new allocation pass #2 ===
PCI: out of I/O address space

Expected results:
guest boot up and seabios didn't have error message

Additional info:
1)guest boot up with 14 e1000e NICs via pcie-root-port
2)with virtio-net-pci NICs,no have hit this issue

Comment 2 Alex Williamson 2019-02-04 15:10:39 UTC
This is a SeaBIOS bug, it is possible to create QEMU configurations that exceed the available I/O port space.  Failure to assign I/O port space to a PCI Express hierarchy should not be considered a fatal error.  The guest would likely work fine if SeaBIOS had left the bridge I/O port apertures unprogrammed.

Comment 3 Laszlo Ersek 2019-02-04 20:53:42 UTC
- This BZ seems to be a duplicate (or "reincarnation") of
  <https://bugzilla.redhat.com/show_bug.cgi?id=1271457>, from 2015/2016.

- The BZ specifies "e1000e", that is, a PCI Express device, which is
  supposed to work fine without IO BARs. Therefore what I think is
  missing is the "io-reserve=0" property from the domain configuration
  (for the "pcie-root-port" devices), which we introduced for
  <https://bugzilla.redhat.com/show_bug.cgi?id=1344299>.


Note You need to log in before you can comment on or make changes to this bug.