RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 990418 - Provide option to enable/disable 64-bit PCI hole
Summary: Provide option to enable/disable 64-bit PCI hole
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.1
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Ján Tomko
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks: 1058602
TreeView+ depends on / blocked
 
Reported: 2013-07-31 07:26 UTC by Igor Mammedov
Modified: 2016-04-26 15:30 UTC (History)
12 users (show)

Fixed In Version: libvirt-1.2.7-1.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1058602 (view as bug list)
Environment:
Last Closed: 2015-03-05 07:21:10 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2015:0323 0 normal SHIPPED_LIVE Low: libvirt security, bug fix, and enhancement update 2015-03-05 12:10:54 UTC

Description Igor Mammedov 2013-07-31 07:26:47 UTC
Description of problem:

Following M$ OSes BSOD at early boot if 64-bit PCI hole is reported to guest:
 * Windows server 2003 x32|x64
 * Windows server 2003 R2 x32|x64
 * Windows XP x32|x64

QEMU 1.6 will have ability to pass on PCI hole configuration to Seabios (merged)
Seabios will read it (pending patches), instead of setting it to hard-coded values (including hiding 64-bit PCI hole if none of devices use it).
New PCI hole initialization will lead to above mentioned OSes BSOD, when guest started with qemu-system-x86_64 binary.

It's possible to explicitly disable 64-bit PCI hole in QEMU 1.6 by passing
  -global i440FX-pcihost.pci-hole64-size=0
option for affected OSes.

How reproducible:
100%

Comment 2 Dave Allan 2013-08-02 15:50:50 UTC
Can you explain what the effect of the hole is any why some guests might want it and some might not?

Comment 3 Michael S. Tsirkin 2013-08-05 16:03:05 UTC
pci hole is where guests place PCI BARs for devices.
Some devices (assigned devices, and ivshmem) might
have large BARs so they don't fit in the 32
bit hole, which is normally 
-#define ACPI_BUILD_PCIMEM_START        0xe0000000
-#define ACPI_BUILD_PCIMEM_END          0xfec00000    /* IOAPIC is mapped at */
so 256 megabyte.

To use these devices guests need a larger 64 bit hole.
Unfortunately some windows guests crash on boot if you make
64 hole too large in ACPI. How large - depends on windows
version, 32 or 64 bit.

Comment 4 Michael S. Tsirkin 2013-08-05 20:46:50 UTC
and to clarify, a bios change that we have to introduce
to fix another bug will cause exactly this crash.

Comment 6 Ján Tomko 2013-08-12 13:13:03 UTC
Upstream patches adding the <pcihole64 size='x'/> element have been proposed:
https://www.redhat.com/archives/libvir-list/2013-August/msg00510.html

Comment 7 Ján Tomko 2013-08-28 06:59:44 UTC
The version adding the element under the PCI root controller has been pushed upstream:
https://www.redhat.com/archives/libvir-list/2013-August/msg00674.html

commit 63ee776f8cd876b7d1a618bf4ca1cba1b62c35bd
Author:     Ján Tomko <jtomko>
AuthorDate: 2013-08-12 13:48:34 +0200
Commit:     Ján Tomko <jtomko>
CommitDate: 2013-08-27 17:42:29 +0200

    Build QEMU command line for pcihole64
    
    QEMU commit 3984890 introduced the "pci-hole64-size" property,
    to i440FX-pcihost and q35-pcihost with a default setting of 2 GB.
    
    Translate <pcihole64>x<pcihole64/> to:
    -global q35-pcihost.pci-hole64-size=x for q35 machines and
    -global i440FX-pcihost.pci-hole64-size=x for i440FX-based machines.
    
    Error out on other machine types or if the size was specified
    but the pcihost device lacks 'pci-hole64-size' property.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=990418

commit 01cda91809751f82ed24ae0eda11ae168d562d7e
Author:     Ján Tomko <jtomko>
AuthorDate: 2013-08-12 13:39:04 +0200
Commit:     Ján Tomko <jtomko>
CommitDate: 2013-08-27 17:42:29 +0200

    Add pcihole64 element to root PCI controllers
    
    <controller type='pci' index='0' model='pci-root'>
      <pcihole64 unit='KiB'>1048576</pcihole64>
    </controller>
    
    It can be used to adjust (or disable) the size of the 64-bit
    PCI hole. The size attribute is in kilobytes (different unit
    can be specified on input), but it gets rounded up to
    the nearest GB by QEMU.
    
    Disabling it will be needed for guests that crash with the
    64-bit PCI hole (like Windows XP), see:
    https://bugzilla.redhat.com/show_bug.cgi?id=990418

Comment 11 Hu Jianwei 2014-12-03 09:45:45 UTC
Do a sanity testing on libvirt side.

[root@ibm-x3850x5-06 ~]# rpm -q libvirt qemu-kvm-rhev
libvirt-1.2.8-10.el7.x86_64
qemu-kvm-rhev-2.1.2-14.el7.x86_64
 
<1> for i440FX-based machines
[root@ibm-x3850x5-06 ~]# virsh dumpxml r7 | grep -E "machine|pci-root" -b3
128-  <currentMemory unit='KiB'>1048576</currentMemory>
180-  <vcpu placement='static'>2</vcpu>
216-  <os>
223:    <type arch='x86_64' machine='pc-i440fx-rhel7.1.0'>hvm</type>
288-    <boot dev='hd'/>
309-  </os>
317-  <features>
--
706-      <target dev='vda' bus='virtio'/>
745-      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
827-    </disk>
839:    <controller type='pci' index='0' model='pci-root'>
894-      <pcihole64 unit='KiB'>1024</pcihole64>
939-    </controller>
957-    <controller type='ide' index='0'>
[root@ibm-x3850x5-06 ~]# virsh start r7
Domain r7 started

[root@ibm-x3850x5-06 ~]# ps aux | grep qemu-kvm
qemu     10387 88.8  0.5 2356040 187944 ?      Sl   13:51   0:09 /usr/libexec/qemu-kvm -name r7 -S -machine pc-i440fx-rhel7.1.0,accel=kvm,usb=off -m 1024 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid 974e8b98-e530-4a95-a6ba-c65951bccb60 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/r7.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -global i440FX-pcihost.pci-hole64-size=1024K ...

[root@ibm-x3850x5-06 ~]# virsh destroy r7
Domain r7 destroyed

<2> for q35 machines
[root@ibm-x3850x5-06 ~]# virsh dumpxml r7 | grep -E "machine|pcie-root" -b3
129-  <currentMemory unit='KiB'>20971520</currentMemory>
182-  <vcpu placement='static'>2</vcpu>
218-  <os>
225:    <type arch='x86_64' machine='pc-q35-rhel7.1.0'>hvm</type>
287-    <boot dev='hd'/>
308-  </os>
316-  <features>
--
705-      <target dev='vda' bus='virtio'/>
744-      <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/>
826-    </disk>
838:    <controller type='pci' index='0' model='pcie-root'>
894-      <pcihole64 unit='KiB'>1048576</pcihole64>
942-    </controller>
960-    <controller type='pci' index='1' model='dmi-to-pci-bridge'>
[root@ibm-x3850x5-06 ~]# virsh start r7
Domain r7 started

[root@ibm-x3850x5-06 ~]# ps aux | grep qemu-kvm
qemu     28427  134  0.1 23787676 47988 ?      Sl   17:36   0:02 /usr/libexec/qemu-kvm -name r7 -S -machine pc-q35-rhel7.1.0,accel=kvm,usb=off -m 20480 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid 974e8b98-e530-4a95-a6ba-c65951bccb60 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/r7.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -global q35-pcihost.pci-hole64-size=1048576K ...

<3> disable pcihole64 on pcie-root
[root@ibm-x3850x5-06 ~]# virsh dumpxml r7 | grep -E "machine|pcie-root" -b3
129-  <currentMemory unit='KiB'>20971520</currentMemory>
182-  <vcpu placement='static'>2</vcpu>
218-  <os>
225:    <type arch='x86_64' machine='pc-q35-rhel7.1.0'>hvm</type>
287-    <boot dev='hd'/>
308-  </os>
316-  <features>
--
705-      <target dev='vda' bus='virtio'/>
744-      <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/>
826-    </disk>
838:    <controller type='pci' index='0' model='pcie-root'>
894-      <pcihole64 unit='KiB'>0</pcihole64>
936-    </controller>
954-    <controller type='pci' index='1' model='dmi-to-pci-bridge'>
[root@ibm-x3850x5-06 ~]# virsh start r7
Domain r7 started

[root@ibm-x3850x5-06 ~]# ps aux | grep qemu-kvm
qemu     21721 99.8  0.1 23787676 50236 ?      Sl   17:33   0:06 /usr/libexec/qemu-kvm -name r7 -S -machine pc-q35-rhel7.1.0,accel=kvm,usb=off -m 20480 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid 974e8b98-e530-4a95-a6ba-c65951bccb60 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/r7.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -global q35-pcihost.pci-hole64-size=0K ...

I don't know whether it is enough for verifying the bug, could you give me some suggestion? Need I check the status in guest OS? Also please tell me how to do if you know about it.

Thanks.

Comment 12 Ján Tomko 2014-12-05 13:42:24 UTC
It seems QEMU solved the possible quest crash another way, because I can't see the setting being used anywhere in QEMU source code.

Since it has no effect on the guest, the only change is on the command line.

Comment 13 Hu Jianwei 2014-12-08 02:45:46 UTC
Thanks, according to comment 11 and 12, move the bug to Verified.

Comment 14 Charlie 2015-02-04 04:46:01 UTC
Hi Jan,
I've come across a similar bug to this in Bug 1058602. from what I can see this has already been documented in:

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7-Beta/html-single/Virtualization_Deployment_and_Administration_Guide/index.html.  

In section 20.3. Configuring device controllers. About half way down there is a section starting: 

" The root controllers (pci-root and pcie-root) have an optional pcihole64 element specifying how big (in kilobytes, or in the unit specified by pcihole64's unit attribute) the 64-bit PCI hole should be. Some guest virtual machines (such as Windows XP or Windows Server 2003) may cause a crash, unless unit is disabled (set to 0 unit='0').
For machine types which provide an implicit PCI bus, the pci-root controller with index='0' is auto-added and required to use PCI devices. pci-root has no address. PCI bridges are auto-added if there are too many devices to fit on the one bus provided by model='pci-root', or a PCI bus number greater than zero was specified. PCI bridges can also be specified manually, but their addresses should only refer to PCI buses provided by already specified PCI controllers. Leaving gaps in the PCI controller indexes might lead to an invalid configuration. The following XML example can be added to the <devices> section..." and goes on a bit more from there.

Just wondering if you could confirm if this adequately documents the feature. Much appreciated.

Charles.

Comment 15 Ján Tomko 2015-02-04 14:10:37 UTC
The option has no effect (it seems only development versions of qemu were affected).

As for the wording:
Unit cannot be disabled, unit is the unit of the size of the hole. The hole could be disabled by setting it to 0, if the option worked.

Comment 17 errata-xmlrpc 2015-03-05 07:21:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0323.html


Note You need to log in before you can comment on or make changes to this bug.