Bug 927032
Summary: | guest crash when installing RHEL6.3: KVM internal error. Suberror: 1 | ||||||||
---|---|---|---|---|---|---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Dan Callaghan <dcallagh> | ||||||
Component: | qemu | Assignee: | Fedora Virtualization Maintainers <virt-maint> | ||||||
Status: | CLOSED WORKSFORME | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||||
Severity: | unspecified | Docs Contact: | |||||||
Priority: | unspecified | ||||||||
Version: | 21 | CC: | amit.shah, berrange, cfergeau, crobinso, dcallagh, dwmw2, itamar, pbonzini, rjones, scottt.tw, virt-maint | ||||||
Target Milestone: | --- | Keywords: | Reopened | ||||||
Target Release: | --- | ||||||||
Hardware: | Unspecified | ||||||||
OS: | Unspecified | ||||||||
Whiteboard: | |||||||||
Fixed In Version: | Doc Type: | Bug Fix | |||||||
Doc Text: | Story Points: | --- | |||||||
Clone Of: | Environment: | ||||||||
Last Closed: | 2015-01-12 07:05:04 UTC | Type: | Bug | ||||||
Regression: | --- | Mount Type: | --- | ||||||
Documentation: | --- | CRM: | |||||||
Verified Versions: | Category: | --- | |||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||
Embargoed: | |||||||||
Attachments: |
|
Description
Dan Callaghan
2013-03-24 23:47:05 UTC
Can you pull the latest f17 kernel from updates-testing and try to reproduce? After upgrading to kernel-3.8.4-102.fc17.x86_64 I haven't been able to reproduce this. So I guess we can consider it fixed. I will re-open the bug if I see the error again. This is still happening with: kernel-3.10.10-200.fc19.x86_64 qemu-kvm-1.4.2-7.fc19.x86_64 libvirt-1.0.5.5-1.fc19.x86_64 It happens on about 50% of my installations now, which seems like a higher rate than previously. I can provide any core dumps, stack traces, or logs which might help to debug this. 2013-09-11 03:17:07.874+0000: starting up LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/qemu-kvm -name beefyguest1 -S -machine pc-0.14,accel=kvm,usb=off -m 2048 -smp 4,sockets=4,cores=1,threads=1 -uuid 3e35960b-3b9b-c6f5-4c7e-f6df56308e34 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/beefyguest1.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot menu=off -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/dev/vg_test2/beefyguest1,if=none,id=drive-virtio-disk0,format=raw,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=2 -netdev tap,fd=31,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:9a:62:b0,bus=pci.0,addr=0x3,bootindex=1 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:5 -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 char device redirected to /dev/pts/5 (label charserial0) KVM internal error. Suberror: 1 emulation failure RAX=ffffffff81000122 RBX=0000000001f8b000 RCX=0000000001d55000 RDX=0000000001000000 RSI=0000000000093780 RDI=0000000001a8c000 RBP=0000000000000000 RSP=000000000236e140 R8 =0000000001a8c000 R9 =0000000000000001 R10=0000000000000038 R11=0000000000000038 R12=0000000000000000 R13=0000000000000000 R14=0000000000000000 R15=0000000000000000 RIP=ffffffff81000122 RFL=00010006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0000 0000000000000000 ffffffff 00c00000 CS =0010 0000000000000000 ffffffff 00a09b00 DPL=0 CS64 [-RA] SS =0000 0000000000000000 ffffffff 00c00000 DS =0000 0000000000000000 ffffffff 00c00000 FS =0000 0000000000000000 ffffffff 00c00000 GS =0000 0000000000000000 ffffffff 00c00000 LDT=0000 0000000000000000 ffffffff 00c00000 TR =0020 0000000000000000 00000fff 00808b00 DPL=0 TSS64-busy GDT= 00000000004d80d8 00000030 IDT= 0000000000000000 00000000 CR0=80000011 CR2=0000000000000000 CR3=000000009238e090 CR4=000000a0 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000500 Code=?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? <??> ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? Is there anything printed in the kernel log (dmesg) when this error happens? Created attachment 796665 [details] libvirt XML for guest (In reply to Richard W.M. Jones from comment #4) > Is there anything printed in the kernel log (dmesg) when > this error happens? Yes, it looks like these two errors appear from the kernel for each crash: [95864.764587] qemu-system-x86: sending ioctl 5326 to a partition! [95864.764859] qemu-system-x86: sending ioctl 80200204 to a partition! The guests all use LVM logical volumes for their virtual disks (virtio, format raw, cache none). Complete guest XML definition is attached. (In reply to Dan Callaghan from comment #5) > Yes, it looks like these two errors appear from the kernel for each crash: > > [95864.764587] qemu-system-x86: sending ioctl 5326 to a partition! > [95864.764859] qemu-system-x86: sending ioctl 80200204 to a partition! Looking closer, I'm not at all certain that these correspond to crashes. They might just be when the guests start normally. It's a bit hard to tell when the guests crashed because the "KVM internal error" in the libvirt logs isn't timestamped. But I can extrapolate from the Beaker logs roughly when the crashes happened and the kernel messages don't line up. There's nothing else of interest in kernel messages, only this type of thing which I'm sure is normal: [95844.450285] br0: port 6(vnet4) entered forwarding state In case it matters, I'm also seeing this crash on an F18 box with slightly different hardware but very similar guest setup, running kernel-3.10.10-100.fc18.x86_64. (In reply to Dan Callaghan from comment #5) > Created attachment 796665 [details] > libvirt XML for guest > > (In reply to Richard W.M. Jones from comment #4) > > Is there anything printed in the kernel log (dmesg) when > > this error happens? > > Yes, it looks like these two errors appear from the kernel for each crash: > > [95864.764587] qemu-system-x86: sending ioctl 5326 to a partition! > [95864.764859] qemu-system-x86: sending ioctl 80200204 to a partition! These wouldn't be connected to this crash. The error is 'KVM_INTERNAL_ERROR_EMULATION' which means different things on Intel & AMD host processors. Is the host processor Intel or AMD (and what precise model? /proc/cpuinfo would be useful here). Created attachment 797053 [details]
/proc/cpuinfo from host
The host CPU is an Intel Core i7 870. /proc/cpuinfo for the host is attached.
The other box where I am seeing the same crash has a Xeon W3550, I can attach its /proc/cpuinfo too if needed.
It's some internal error deep inside KVM. I have no idea why it happens, but you could help by adjusting the libvirt configuration to see if you can reliably make the error appear and go away (eg. by adding/removing a particular device). Otherwise I'd suggest posting a bug in the upstream qemu tracker. Dan, still seeing this with latest F19 packages? I'd also be interested to know if trying kernel 3.12 makes any difference, here's the command, but it may pull in other dependencies, if so don't worry: sudo yum install fedora-release-rawhide sudo yum --enablerepo=rawhide update kernel Latest rawhide kernel has quite a few emulation fixes for kvm, so I'm going to assume this is fixed in the 3.13 snapshots which will eventually end up in f19. Closing, please reopen if that's not the case. This is still happening with kernel-3.13.0-0.rc8.git0.1.fc21.x86_64. KVM internal error. Suberror: 1 emulation failure RAX=ffffffff81000122 RBX=0000000001f8b000 RCX=0000000001d55000 RDX=0000000001000000 RSI=0000000000093780 RDI=0000000001a8c000 RBP=0000000000000000 RSP=000000000236e140 R8 =0000000001a8c000 R9 =0000000000000001 R10=0000000000000038 R11=0000000000000038 R12=0000000000000000 R13=0000000000000000 R14=0000000000000000 R15=0000000000000000 RIP=ffffffff81000122 RFL=00010006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0000 0000000000000000 ffffffff 00c00000 CS =0010 0000000000000000 ffffffff 00a09b00 DPL=0 CS64 [-RA] SS =0000 0000000000000000 ffffffff 00c00000 DS =0000 0000000000000000 ffffffff 00c00000 FS =0000 0000000000000000 ffffffff 00c00000 GS =0000 0000000000000000 ffffffff 00c00000 LDT=0000 0000000000000000 ffffffff 00c00000 TR =0020 0000000000000000 00000fff 00808b00 DPL=0 TSS64-busy GDT= 00000000004d80d8 00000030 IDT= 0000000000000000 00000000 CR0=80000011 CR2=0000000000000000 CR3=000000009238e090 CR4=000000a0 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000500 Code=?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? <??> ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? qemu: terminating on signal 15 from pid 819 Anyone still hitting this with latest packages? If so, please list the versions I'm still seeing this occasionally with: kernel-3.15.6-200.fc20.x86_64 qemu-kvm-1.6.2-6.fc20.x86_64 paolo, is this a known issue? any extra info dan can provide? No, it's not known... This message is a notice that Fedora 19 is now at end of life. Fedora has stopped maintaining and issuing updates for Fedora 19. It is Fedora's policy to close all bug reports from releases that are no longer maintained. Approximately 4 (four) weeks from now this bug will be closed as EOL if it remains open with a Fedora 'version' of '19'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 19 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. I can't reproduce this anymore using kernel-3.17.8-300.fc21.x86_64 and qemu-kvm-2.1.2-7.fc21.x86_64. |