Description of problem: When trying to pxe boot kvm vm's using kernel-3.5.3-1.fc17.x86_64 they pause immediately after the iPXE (http://ipxe.org)... line. Version-Release number of selected component (if applicable): Fedora 17, libvirt-0.9.11.5-3.fc17.x86_64, kernel-3.5.3-1.fc17.x86_64 How reproducible: Always. Appears to always happen on multiple systems Steps to Reproduce: 1. Set up a kvm vm to pxe boot 2. Start the vm Actual results: vm pauses Expected results: vm should pxe boot and install Additional info: Booting back to kernel-3.5.2-3.fc17.x86_64 and prior works just fine. Booting to disk works fine with kernel-3.5.3-1.fc17.x86_64 - it appears to just be pxe booting that is affected. /var/log/libvirt/qemu/test.log ends with: KVM: entry failed, hardware error 0x80000021 If you're running a guest on an Intel machine without unrestricted mode support, the failure can be most likely due to the guest entering an invalid state for Intel VT. For example, the guest maybe running in big real mode which is not supported on less recent Intel processors. EAX=00000011 EBX=00000000 ECX=00000030 EDX=00007baa ESI=c007deba EDI=00038110 EBP=00000000 ESP=00007baa EIP=00000370 EFL=00010006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0030 0009aef0 ffffffff 0000f300 DPL=3 DS16 [-WA] CS =9a60 0009a630 0000ffff 00009b00 DPL=0 CS16 [-RA] SS =0000 00000000 0000ffff 00009300 DPL=0 DS16 [-WA] DS =0030 0009aef0 ffffffff 0000f300 DPL=3 DS16 [-WA] FS =0030 0009aef0 ffffffff 0000f300 DPL=3 DS16 [-WA] GS =0030 0009aef0 ffffffff 0000f300 DPL=3 DS16 [-WA] LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS32-busy GDT= 0009af30 00000037 IDT= 00000000 0000ffff CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000000 Code=66 0f 01 16 40 00 66 0f 01 1e 78 00 0f 20 c0 0c 01 0f 22 c0 <66> ea a4 00 00 00 08 00 0f 20 c0 24 fe 0f 22 c0 ff 2e 7e 00 2e a1 14 08 8e d8 8e c0 8e e0 handle_dev_stop: stop
*** Bug 855700 has been marked as a duplicate of this bug. ***
The subject is a bit off. KVM does not pause but crashes which causes virt-manager to show the VM as paused. More info and discussion about this bug here: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1045027
Avi, looks like a regression caused by something that got pulled into the 3.5.3 stable tree for KVM. Would upstream commit a27685c33acccce91268ddef88d7896e3205fda5 possibly fix this?
It seems this problem is not limited to PXE booting. I installed a Win7 x86_64 VM on a laptop which has a COA. Since KVM still does not pass the laptop's SLIC on to the seabios (sigh) I had to create a new seabios with the laptop's SLIC added and boot the Win7 VM with that new seabios. In order to boot the Win7 VM with the new seabios I do the following: 0) build new seabios with SLIC 1) cp seabios-slic.bin /usr/share/seabios && restorecon ... 2) change Win7.xml config: Change: <domain type='kvm'> to <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> And add to the end of Win7.xml after </devices>: <qemu:commandline> <qemu:arg value='-bios'/> <qemu:arg value='/usr/share/seabios/seabios-slic.bin'/> </qemu:commandline> The results: - booting Win7 x86_64 VM on kernel 3.5.2 with new seabios with SLIC fails - booting Win7 x86_64 VM on kernel 3.5.2 with new seabios with SLIC and kvm_intel module loaded with emulate_invalid_guest_state=y fails - booting Win7 x86_64 VM on kernel 3.5.3 with new seabios with SLIC fails - booting Win7 x86_64 VM on kernel 3.5.3 with new seabios with SLIC and kvm_intel module loaded with emulate_invalid_guest_state=y fails Fails means: KVM: entry failed, hardware error 0x80000021 Hope this helps. Please let me know if you have any questions.
FYI: I just tried kernel-3.5.4-2.fc17.x86_64 from koji and the problem is still there. It's impossible to do a PXE install of CentOS 6. I still get KVM: entry failed, hardware error 0x80000021 in the log file of the VM. If you need more information or would like me to test another kernel then please let me know. I'll be happy to help. Thanks!
FWIW here is the cpuinfo of the box where I see this issue. It's a quadcore cpu but only the first processor is pasted here. $ cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Core(TM) i7 CPU 960 @ 3.20GHz stepping : 5 microcode : 0x11 cpu MHz : 1596.000 cache size : 8192 KB physical id : 0 siblings : 8 core id : 0 cpu cores : 4 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid bogomips : 6395.26 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management:
I'm seeing the same issue, there was a change between vanilla kernel 3.5.2 and 3.5.3 that seemed to break things for me.
*** Bug 859830 has been marked as a duplicate of this bug. ***
*** Bug 863082 has been marked as a duplicate of this bug. ***
Reproduced with qemu-system-x86_64 -enable-kvm -net nic -net user,bootfile=resolv.conf,tftp=/etc
Sorry, that was an operator error due to invalid pxe configuration. With a valid config I can boot on 3.5.5. Can you share the command line which fails? Preferably with -net user (and the internal tftp server) so there is less setup effort involved.
I'm not sure what you mean with "-net user (and the internal tftp server)". I only use virt-manager to create VMs. From the logs in /var/log/libvirt/qemu/ here's a command line which failed: LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=spice /usr/bin/qemu-kvm -S -M pc-0.15 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -name Test -uuid eee9787c-d9b4-a7c7-672a-a052c6a8e72b -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/Test.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/home/test/VM/Test.img,if=none,id=drive-virtio-disk0,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=25,id=hostnet0,vhost=on,vhostfd=26 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:56:58:60:62,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -device usb-tablet,id=input0 -spice port=5900,addr=127.0.0.1,disable-ticketing -vga qxl -global qxl-vga.vram_size=67108864 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7
I downloaded 3.5.5-2 from http://kojipkgs.fedoraproject.org/packages/kernel/3.5.5/2.fc17/x86_64/ and did a local install I was now able to pxe boot the node without the VM pausing.
Interesting, more information... the above kernel worked on one of my systems. cpuinfo from the one that worked. processor : 3 vendor_id : GenuineIntel cpu family : 6 model : 37 model name : Intel(R) Core(TM) i5 CPU M 540 @ 2.53GHz stepping : 2 microcode : 0xd cpu MHz : 1199.000 cache size : 3072 KB physical id : 0 siblings : 4 core id : 2 cpu cores : 2 apicid : 5 initial apicid : 5 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt aes lahf_lm ida arat dtherm tpr_shadow vnmi flexpriority ept vpid bogomips : 5054.19 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: cpuinfo from the one that didn't processor : 7 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Core(TM) i7 CPU 930 @ 2.80GHz stepping : 5 microcode : 0xf cpu MHz : 1600.000 cache size : 8192 KB physical id : 0 siblings : 8 core id : 3 cpu cores : 4 apicid : 7 initial apicid : 7 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid bogomips : 5613.06 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: trying newer kernel on the i7 that didn't work.
I did a mock rebuild of the kernel-3.6.0-2.fc18.src.rpm on my f17 system (after getting one of the dependencies missing). I've got the RPMs I used here. http://dmlb2000.homelinux.org/packages/kvm-bug-854983/ This allows the i7 to boot with pxe enabled.
Avi, I was using your commandline for qemu-kvm when I was testing on my systems at least. So to recap seems the problem affected my i5 and my i7 systems. However when I updated to 3.5.5 that fixed the i5 I have but not the i7. Updating to 3.6 fixed the i7 system.
(In reply to comment #11) > Can you share the command line which fails? Preferably with -net user (and > the internal tftp server) so there is less setup effort involved. No need for any qemu-kvm options. Fails with defaults too: qemu-kvm -enable-kvm Guest results: SeaBIOS (version 1.7.0-20120528_224116-x86-16.phx2.fedoraproject.org) iPXE (http://ipxe.org) 00:03.0 C900 PCI2.10 PnP PMM+07FC9CA0+07F89CA0 C900 Booting from Hard Disk... Boot failed: could not read the boot disk Booting from DVD/CD... Boot failed: Could not read from CDROM (code 0003) Booting from ROM... iPXE (PCI 00:03.0) starting execution... Host results: KVM: entry failed, hardware error 0x80000021 If you're running a guest on an Intel machine without unrestricted mode support, the failure can be most likely due to the guest entering an invalid state for Intel VT. For example, the guest maybe running in big real mode which is not supported on less recent Intel processors. EAX=00000011 EBX=00000000 ECX=00000030 EDX=00007baa ESI=f807df0a EDI=000385c8 EBP=00000000 ESP=00007baa EIP=00000370 EFL=00010006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0030 0009c6f0 ffffffff 0000f300 DPL=3 DS16 [-WA] CS =9be0 0009be30 0000ffff 00009b00 DPL=0 CS16 [-RA] SS =0000 00000000 0000ffff 00009300 DPL=0 DS16 [-WA] DS =0030 0009c6f0 ffffffff 0000f300 DPL=3 DS16 [-WA] FS =0030 0009c6f0 ffffffff 0000f300 DPL=3 DS16 [-WA] GS =0030 0009c6f0 ffffffff 0000f300 DPL=3 DS16 [-WA] LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS32-busy GDT= 0009c730 00000037 IDT= 00000000 0000ffff CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000700000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000000 Code=66 0f 01 16 40 00 66 0f 01 1e 78 00 0f 20 c0 0c 01 0f 22 c0 <66> ea a4 00 00 00 08 00 0f 20 c0 24 fe 0f 22 c0 ff 2e 7e 00 2e a1 14 08 8e d8 8e c0 8e e0 Fail on my notebook (Genuine Intel(R) CPU T2400 @ 1.83GHz) and desktop too (Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz).
I too have this same problem and have been tracking this bug, I have just installed kernel-3.5.5-1.fc17 but it hasn't fixed the problem. I have an older PC with CPU: Intel(R) Core(TM)2 Duo CPU E8200. virt-manager: SeaBIOS (version 1.7.0-20120528-224116-x86-16.phx2.fedoraproject.org) iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+3FFC9980+3FF89980 CA00) Press F12 for boot menu. Booting from ROM.. iPXE (PCI 00:03.0) starting execution... /var/log/libvirt/qemu/pridns.log: KVM: entry failed, hardware error 0x80000021 Not sure if I need to update SeaBIO? This is a recent/fresh install of fc17. Colin - Stevenage/UK
Have just installed kernel-3.6.0-2.fc17.src.rpm and it has fixed the problem. Thank you, :-) Rgds- Colin
kernel-3.5.5-2.fc17 - same problem here I can't find any kernel-3.6.x.fc17 in koji/bodhi. Is it possible to backport this fix to any fc17 kernel?
Kernel 3.5.5-2.fc17 still has the PXE issue on my i7 box too (specs in comment 6). @Jan ONDREJ: the 3.6.0 kernel packages for F17 can be found at the url in comment 15: http://dmlb2000.homelinux.org/packages/kvm-bug-854983/
F17 will get an official 3.6.x kernel update sometime this week.
I see it with F17's ipxe/seabios. I'll revert the offending patch in 3.5.
(In reply to comment #23) > I see it with F17's ipxe/seabios. I'll revert the offending patch in 3.5. Thanks Avi. F17 in git has been moved to 3.6.1, so marking this as MODIFIED.
kernel-3.6.1-1.fc17 has been submitted as an update for Fedora 17. https://admin.fedoraproject.org/updates/kernel-3.6.1-1.fc17
Package kernel-3.6.1-1.fc17: * should fix your issue, * was pushed to the Fedora 17 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing kernel-3.6.1-1.fc17' as soon as you are able to, then reboot. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2012-15855/kernel-3.6.1-1.fc17 then log in and leave karma (feedback).
3.6.3-1 does NOT fix this bug. KVM: entry failed, hardware error 0x80000021 If you're running a guest on an Intel machine without unrestricted mode support, the failure can be most likely due to the guest entering an invalid state for Intel VT. For example, the guest maybe running in big real mode which is not supported on less recent Intel processors. EAX=00000000 EBX=00195eba ECX=00000000 EDX=fffff000 ESI=def00014 EDI=00000001 EBP=de855fa4 ESP=de855f64 EIP=c04288a3 EFL=00010246 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =007b 00000000 ffffffff 00c0f300 DPL=3 DS [-WA] CS =0060 00000000 ffffffff 00c09b00 DPL=0 CS32 [-RA] SS =0068 00000000 ffffffff 00c09300 DPL=0 DS [-WA] DS =007b 00000000 ffffffff 00c0f300 DPL=3 DS [-WA] FS =00d8 1e26c000 ffffffff 00809300 DPL=0 DS16 [-WA] GS =00e0 def06480 00000018 00409100 DPL=0 DS [--A] LDT=0000 ffff0000 f0000fff 00f0ff00 DPL=3 CS64 [CRA] TR =0080 def04300 0000206b 00008b00 DPL=0 TSS32-busy GDT= deeff000 000000ff IDT= c0b7f000 000007ff CR0=8005003b CR2=ffffffff CR3=00ca4000 CR4=000006d0 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000700000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000000 Code=00 8b 15 04 ff b8 c0 55 89 e5 5d 8d 84 10 00 c0 ff ff 8b 00 <c3> 8d b6 00 00 00 00 8d bf 00 00 00 00 8b 15 a0 f6 b8 c0 55 89 e5 53 89 c3 b8 30 00 00 00 ^C $ uname -a Linux thinkpad.home.annexia.org 3.6.3-1.fc17.i686 #1 SMP Mon Oct 22 16:10:29 UTC 2012 i686 i686 i386 GNU/Linux (Same kernel on guest and host) $ cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 14 model name : Intel(R) Core(TM) Duo CPU L2400 @ 1.66GHz stepping : 12 microcode : 0x54 cpu MHz : 1000.000 cache size : 2048 KB physical id : 0 siblings : 2 core id : 0 cpu cores : 2 apicid : 0 initial apicid : 0 fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx constant_tsc arch_perfmon bts aperfmperf pni monitor vmx est tm2 xtpr pdcm dtherm bogomips : 3325.30 clflush size : 64 cache_alignment : 64 address sizes : 32 bits physical, 32 bits virtual power management: processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 14 model name : Intel(R) Core(TM) Duo CPU L2400 @ 1.66GHz stepping : 12 microcode : 0x54 cpu MHz : 1000.000 cache size : 2048 KB physical id : 0 siblings : 2 core id : 1 cpu cores : 2 apicid : 1 initial apicid : 1 fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx constant_tsc arch_perfmon bts aperfmperf pni monitor vmx est tm2 xtpr pdcm dtherm bogomips : 3325.30 clflush size : 64 cache_alignment : 64 address sizes : 32 bits physical, 32 bits virtual power management: This hardware is ancient (and rare) early 32 bit Intel VT laptop.
kernel-3.6.1-1.fc17 has been pushed to the Fedora 17 stable repository. If problems still persist, please make note of it in this bug report.