Red Hat Bugzilla – Bug 638468
[qemu-kvm] bochs vga lfb @ 0xe0000000 causes trouble for hot-plug
Last modified: 2013-01-09 18:11:51 EST
Description: Boot up a RHEL 6.0 guest in a RHEL 6.0 host, then hotplug pass-through device to guest(82576 NIC). Qemu-kvm will quit with this message: create_userspace_phys_mem: File exists assigned_dev_iomem_map: Error: create new mapping failed If using the same command line with "-device pci-assign,host=xx:xx.xx", the guest can boot up successfully. Version-Release number of selected component (if applicable): Host: 2.6.32-71.1.1.el6_0.x86_64 Guest: 2.6.32-71 Qemu: rpm -qa |grep qemu qemu-kvm-tools-0.12.1.2-2.113.el6_0.1.x86_64 qemu-kvm-0.12.1.2-2.113.el6_0.1.x86_64 gpxe-roms-qemu-0.9.7-6.3.el6.noarch qemu-img-0.12.1.2-2.113.el6_0.1.x86_64 qemu-kvm-debuginfo-0.12.1.2-2.113.el6_0.1.x86_64 How reproducible: Always Steps to Reproduce: 1. Boot up a RHEL 6.0 guest # /usr/libexec/qemu-kvm -M rhel6.0.0 -enable-kvm -m 11000 -smp 4,sockets=4,cores=4,threads=1 -name test -uuid 27c443dc-190e-ecbd-7b19-187bcae3f26d -nodefconfig -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/test.monitor,server,nowait -mon chardev=monitor,mode=readline -rtc base=utc -boot c -drive file=/home/rhel6RC1.0,if=none,id=drive-ide0-0-0,boot=on,format=raw,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,werror=stop,rerror=stop,cache=none -chardev pty,id=serial0 -device isa-serial,chardev=serial0 -usb -device usb-tablet,id=input0 -vnc :0 -vga std 2. Unbind the pci device # echo "8086 10ca" > /sys/bus/pci/drivers/pci-stub/new_id # echo 0000:28:10.1 > /sys/bus/pci/devices/0000\:28\:10.1/driver/unbind # echo 0000:28:10.1 > /sys/bus/pci/drivers/pci-stub/bind 2. Use device_add to hot add a pass-through device, then qemu-kvm will quit. # device_add pci-assign,host=28:10.1,id=test Actual results: Qemu-kvm quit Expected results: Can assign a pass-through device to guest Additional info: 1.Host cpuinfo processor : 7 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Xeon(R) CPU E5530 @ 2.40GHz stepping : 5 cpu MHz : 1596.000 cache size : 8192 KB physical id : 1 siblings : 4 core id : 3 cpu cores : 4 apicid : 22 initial apicid : 22 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid bogomips : 4787.20 clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: 2. Error message create_userspace_phys_mem: File exists assigned_dev_iomem_map: Error: create new mapping failed 3. Nic info 28:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) 28:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
Looks like device pass-through is just a victim here. When using the standard vga device, 16MB of memory is setup for VBE at fixed address 0xe0000000. The guest kernel knows nothing about this, so tries to assign device resources overlapping this range when the 82576 VF is hot added. When the range is passed into the KVM ioctl, it fails since the range has already been assigned to a different slot, causing the VM to exit. For the moment, there are several potential workarounds: 1. Add the option memmap=16M$0xe0000000 to the guest command line. This forces the guest to treat that range as reserved. 2. Use -vga cirrus rather than std. 3. Boot with less memory in the guest, effectively making the I/O window larger, allowing the guest to map the hotplug device below 0xe0000000.
TODO: backport http://patchwork.ozlabs.org/patch/67919/ + friends vgabios needs an update too.
Committed upstream: http://git.qemu.org/qemu.git/commit/?id=543f8e3468e6df32bfde8f84ac36d05a7604e082 http://git.qemu.org/qemu.git/commit/?id=4eccfec4943db1106c79a01069e18dd4f463219b Additional fix needed for migration compatibility: http://patchwork.ozlabs.org/patch/71545/ That puts the lfb @ 0xe0000000 back for older qemu versions. We will have to do the same for -M rhel6.0.0 as we can't just zap the mapping while the guest might use it. Only for -M rhel6.1.0+ we can really kill it. So we might additionally backport the e820 bits? Alex? Jes? Opinions?
Ignore Comment #13. According to Comment #12, this bug has been fixed. Moving Status to VERIFIED.
with vgabios-0.6b-3.5.el6.noarch.rpm installed, boot a guest and issue lspci -vs2: 00:02.0 VGA compatible controller: Technical Corp. Device 1111 (prog-if 00 [VGA controller]) Subsystem: Red Hat, Inc Device 1100 Physical Slot: 2 Flags: fast devsel Memory at f0000000 (32-bit, prefetchable) [size=16M] Expansion ROM at f1000000 [disabled] [size=64K]
Technical note added. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. New Contents: see bug 654639
Technical note updated. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. Diffed Contents: @@ -1 +1 @@ -see bug 654639+Previously, when a pass-through device was plugged into an operational guest on qemu-kvn, the process failed with "create_userspace_phys_mem: File exists. assigned_dev_iomem_map: Error: create new mapping failed" error. This error occurred because qemu-kvn attempted to use the memory address 0xe0000000, which was reserved for another process. This is now fixed and qemu-kvn no longer attempts to use the reserved memory at the location 0xe0000000.
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2011-0534.html