Bug 544339 - Segfaults logged from kvm (qemu-kvm) resulting in guest sudden crash and data loss
Segfaults logged from kvm (qemu-kvm) resulting in guest sudden crash and data...
Status: CLOSED WONTFIX
Product: Fedora
Classification: Fedora
Component: kvm (Show other bugs)
12
All Linux
low Severity medium
: ---
: ---
Assigned To: Glauber Costa
Fedora Extras Quality Assurance
: Triaged
Depends On:
Blocks: F13VirtImportant
  Show dependency treegraph
 
Reported: 2009-12-04 11:52 EST by alan
Modified: 2013-01-09 06:29 EST (History)
13 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-12-03 21:13:48 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)

  None (edit)
Description alan 2009-12-04 11:52:37 EST
Description of problem:

Randomly the following is logged and the guest explodes


qemu-kvm[28375]: segfault at 2dfd7e85f0 ip 00000034a9e81510 sp 00007fff5e3201e8 error 4 in libc-2.11.so[34a9e00000+16f000]


Version-Release number of selected component (if applicable):

qemu-kvm-0.11.0-11.fc12.x86_64

How reproducible:

Occurs at random with FC12, never seen with FC11

Steps to Reproduce:
1. Upgrade to FC12
2. Attempt to use virtualisation

Other maybe relevant info

64bit host, 64bit guest
8 CPU host, 4 cpu guest
Guest has virtual ide root and eight virtual i/o other disks (but not in use)
X session in use on guest
Networking active on guest

  
Actual results:
Kerboom

Expected results:
Stable system
Comment 1 Jon Ciesla 2009-12-07 15:47:24 EST
I've seen something very like this.  If I do an install with kvm via virt-manager, the machine reboots once I finish choosing options and boot the installer.  If I choose qemu, it's smooth sailing.  OS doesn't matter, but I tried Fedora 12, ReactOS 0.3.10 and AROS.  Also the most recent Haiku, IIRC.
Comment 2 Justin M. Forbes 2009-12-10 14:07:39 EST
Can you give some more details on the system, does it have a smolt profile, if not what is the CPU/model?  What host kernel is in use?  Also, is it possible to attach a backtrace?  abrt should make this easier to grab.
Comment 3 Jon Ciesla 2009-12-10 14:15:41 EST
kernel is 23.3136-162.fc12.i686, though this has happened on all the recent kernels. CPU is AMD Athlon 64X2 Dual Core TK-55.  No backtrace, or anything in abrt.  If there's a good way to get one, let me know.
Comment 4 Stephen Murray 2009-12-15 20:26:55 EST
I have the identical problem. It started recently under F12, it never happened under F11, and I don't recall seeing it with the early F12 environment. I am running a Vista 32 bit guest created under F11 64 bit.

Current kernel is 2.6.31.6-166.fc12.x86_64. 

Current QEMU is:
rpm -qa | grep qemu
qemu-img-0.11.0-12.fc12.x86_64
qemu-system-x86-0.11.0-12.fc12.x86_64
gpxe-roms-qemu-0.9.7-6.fc12.noarch
qemu-common-0.11.0-12.fc12.x86_64
qemu-kvm-0.11.0-12.fc12.x86_64

Machine is Dell Precision 390 with Intel Core2 Quad @ 2.66GHz, 8GB RAM, 3 SATA drives.

FWIW, I run a similar setup at the office, but I have *never* seen this problem. That's a Dell Precision 3500 with Intel Core2 Quad (Intel(R) Xeon(R) CPU W3540  @ 2.93GHz), 12GB RAM, 3 SATA drives. Same F12, same Vista release, same patch levels of kernel and QEMU. Though I don't run much video/music on it, and I have the nVidia graphics driver rather than nouveau, as nouveau stalls on the office machine.

Some stuff from "var/log/messages":

Dec 15 17:33:24 linux1 kernel: [drm] nouveau 0000:01:00.0: Setting dpms mode 0 on CRTC 0
Dec 15 17:33:24 linux1 kernel: [drm] nouveau 0000:01:00.0: Setting dpms mode 0 on vga encoder (output 0)
Dec 15 17:35:38 linux1 kernel: npviewer.bin[17431]: segfault at 0 ip (null) sp 00007fff0914d898 error 14 in npviewer.bin[400000+1e000]
Dec 15 17:35:38 linux1 abrtd: Directory 'ccpp-1260916538-17431' creation detected
Dec 15 17:35:38 linux1 abrtd: Lock file '/var/cache/abrt/ccpp-1260916538-17431.lock' is locked by process 2562
Dec 15 17:35:39 linux1 abrt: saved core dump of pid 17431 to /var/cache/abrt/ccpp-1260916538-17431/coredump (135667712 bytes)
Dec 15 17:35:39 linux1 abrtd: Blacklisted package
Dec 15 17:35:39 linux1 abrtd: Corrupted or bad crash, deleting
Dec 15 17:37:42 linux1 kernel: qemu-kvm[11964]: segfault at 2996d49750 ip 00000031e5281510 sp 00007fff4ba7dd48 error 4 in libc-2.11.so[31e5200000+16f000]
Dec 15 17:37:42 linux1 abrtd: Directory 'ccpp-1260916662-11964' creation detected
Dec 15 17:37:42 linux1 abrtd: Lock file '/var/cache/abrt/ccpp-1260916662-11964.lock' is locked by process 2751
<--snip--> (lots and lots of the above "locked" message)
Dec 15 17:37:55 linux1 abrt: saved core dump of pid 11964 to /var/cache/abrt/ccpp-1260916662-11964/coredump (2237153280 bytes)
Dec 15 17:37:55 linux1 kernel: br0: port 2(vnet0) entering disabled state
Dec 15 17:37:55 linux1 kernel: device vnet0 left promiscuous mode
Dec 15 17:37:55 linux1 kernel: br0: port 2(vnet0) entering disabled state
Dec 15 17:37:55 linux1 libvirtd: 17:37:55.716: error : qemudDomainGetMemoryBalloon:3518 : operation failed: could not query memory balloon allocation
Dec 15 17:37:56 linux1 ntpd[1644]: Deleting interface #24 vnet0, fe80::7893:f4ff:fe5d:c0d0#123, interface stats: received=0, sent=0, dropped=0, active_time=84719 secs
Dec 15 17:37:56 linux1 abrtd: Getting local universal unique identification...
Dec 15 17:38:00 linux1 abrtd: CDebugDump::CDebugDump(): /var/cache/abrt/ccpp-1260782780-32201 does not exist.
Dec 15 17:38:00 linux1 abrtd: Crash is in database already
Dec 15 17:38:00 linux1 abrtd: Already saved crash, just sending dbus signal
Dec 15 17:38:00 linux1 kernel: abrtd[1678]: segfault at 10 ip 000000000041cbe7 sp 00007fff343d8f40 error 4 in abrtd[400000+2a000]
Dec 15 17:38:00 linux1 abrt: pid 1678 is '/usr/sbin/abrtd', not dumping it to avoid abrt recursion

I have seen the npviewer crash just before other qemu crashes too. I don't know if it's related, might be a total coincidence, but I mention it here for completeness. I see the crash early in the morning when I'm not at the computer, I see it when I am using it. It happens, I'd say, about every couple of days. I once thought it was Outlook because it crashed while trying to open an email message, so I got into the habit of shutting Outlook down. The guest still went down though. I've patched kernel and qemu to current via "yum" many times, no improvement.
Comment 5 Jon Ciesla 2009-12-16 08:30:54 EST
I don't think it's video related, I'm on a radeon.  Is it just your guest or hour host machine crashing as well?  For me it's the host that hard reboots.
Comment 6 Stephen Murray 2009-12-16 09:33:56 EST
For me, it's just the guest that crashes. The host is fine.

I'm in the same position as the original poster of the bug, I see the message "qemu-kvm[....]: segfault at ..... error 4 in libc-2.11.so" in "/var/log/messages" as well as the other messages I posted following the guest crash.

I'm not convinced that it's anything to do with video, I just wanted to toss in everything that was happening around the time of the crash.

A thought, I haven't tried this with my RHEL5 guest. I'll bring up both Vista and RHEL5 and see if they both go down.
Comment 7 alan 2009-12-16 11:43:54 EST
Just the guest in my case

I'm now fairly sure its the emulated video that is the bug - if I have a guest running where the guest is in text mode I've yet to see this happen, in graphics mode it happens all the time to the point KVM is basically unusable in FC12 for anything with a Linux guest where the guest is in X11.
Comment 8 Stephen Murray 2009-12-19 09:07:53 EST
I just encountered another guest crash this morning. For a test, I had brought up the Vista guest and a RHEL5 guest with X11/GUI running. The RHEL5 guest had a terminal window open within the GUI where I would periodically issue a command to check it.

At 4:57am today the Vista guest crashed, but the RHEL5 guest stayed up. The RHEL5 logs show nothing at all, if there was an event external to it, it didn't see it. The Vista Event Logs show nothing after 3:01am, and even what is recorded is not interesting or useful.

One significant software difference between my not-crashing office machine and this crashing home one is the Video driver. At work I am using the nVidia driver (forced upon me, nouveau stalls too much with the very recent video adapter), at home I use the nouveau. My next test will be to install nVidia driver on my home machine to see if the Vista crashes stop. 

More later.


The "/var/log/messages" log shows this (all activity from 9pm last night):

Dec 18 21:07:00 linux1 kernel: [drm] nouveau 0000:01:00.0: Setting dpms mode 0 on CRTC 0
Dec 18 21:07:00 linux1 kernel: [drm] nouveau 0000:01:00.0: Setting dpms mode 0 on vga encoder (output 0)
Dec 18 21:10:15 linux1 kernel: npviewer.bin[26348]: segfault at 8f0 ip 00007f1b096f1117 sp 00007ffff0ef73e0 error 6 in libflashplayer.so[7f1b094a9000+8c7000]
Dec 18 21:10:15 linux1 abrt: abrt daemon is not running. If it crashed, /proc/sys/kernel/core_pattern contains a stale value, consider resetting it to 'core'
Dec 18 21:14:06 linux1 kernel: npviewer.bin[5289]: segfault at 0 ip (null) sp 00007fffd541f368 error 14 in npviewer.bin[400000+1e000]
Dec 18 21:14:06 linux1 abrt: abrt daemon is not running. If it crashed, /proc/sys/kernel/core_pattern contains a stale value, consider resetting it to 'core'
Dec 18 22:00:31 linux1 kernel: [drm] nouveau 0000:01:00.0: Setting dpms mode 3 on vga encoder (output 0)
Dec 18 22:00:31 linux1 kernel: [drm] nouveau 0000:01:00.0: Setting dpms mode 3 on CRTC 0
Dec 18 23:43:21 linux1 kernel: [drm] nouveau 0000:01:00.0: Setting dpms mode 0 on CRTC 0
Dec 18 23:43:21 linux1 kernel: [drm] nouveau 0000:01:00.0: Setting dpms mode 0 on vga encoder (output 0)
Dec 18 23:47:59 linux1 kernel: npviewer.bin[5585]: segfault at 653 ip 00007fe48725d144 sp 00007fff8a6fbf08 error 4 in libflashplayer.so[7fe486c33000+8c7000]
Dec 18 23:47:59 linux1 abrt: abrt daemon is not running. If it crashed, /proc/sys/kernel/core_pattern contains a stale value, consider resetting it to 'core'
Dec 19 00:07:09 linux1 smbd[8713]: [2009/12/19 00:07:09,  0] lib/util_sock.c:539(read_fd_with_timeout)
Dec 19 00:07:09 linux1 smbd[8713]: [2009/12/19 00:07:09,  0] lib/util_sock.c:1491(get_peer_addr_internal)
Dec 19 00:07:09 linux1 smbd[8713]:   getpeername failed. Error was Transport endpoint is not connected
Dec 19 00:07:09 linux1 smbd[8713]:   read_fd_with_timeout: client 0.0.0.0 read error = No route to host.
Dec 19 00:20:12 linux1 kernel: [drm] nouveau 0000:01:00.0: Setting dpms mode 3 on vga encoder (output 0)
Dec 19 00:20:12 linux1 kernel: [drm] nouveau 0000:01:00.0: Setting dpms mode 3 on CRTC 0
Dec 19 04:22:02 linux1 pcscd: winscard.c:309:SCardConnect() Reader E-Gate 0 0 Not Found
Dec 19 04:22:02 linux1 pcscd: winscard.c:309:SCardConnect() Reader E-Gate 0 0 Not Found
Dec 19 04:22:02 linux1 pcscd: winscard.c:309:SCardConnect() Reader E-Gate 0 0 Not Found
Dec 19 04:22:02 linux1 pcscd: winscard.c:309:SCardConnect() Reader E-Gate 0 0 Not Found
Dec 19 04:57:15 linux1 kernel: qemu-kvm[29956]: segfault at 310f868590 ip 00000031e5281510 sp 00007fff3c0aba68 error 4 in libc-2.11.so[31e5200000+16f000]
Dec 19 04:57:16 linux1 abrt: abrt daemon is not running. If it crashed, /proc/sys/kernel/core_pattern contains a stale value, consider resetting it to 'core'
Dec 19 04:57:16 linux1 kernel: br0: port 2(vnet0) entering disabled state
Dec 19 04:57:16 linux1 kernel: device vnet0 left promiscuous mode
Dec 19 04:57:16 linux1 kernel: br0: port 2(vnet0) entering disabled state
Dec 19 04:57:16 linux1 libvirtd: 04:57:16.411: error : qemudDomainGetMemoryBalloon:3518 : operation failed: could not query memory balloon allocation
Dec 19 04:57:17 linux1 ntpd[1644]: Deleting interface #25 vnet0, fe80::f83a:b9ff:fe23:e35a#123, interface stats: received=0, sent=0, dropped=0, active_time=242515 secs
Dec 19 08:39:38 linux1 kernel: [drm] nouveau 0000:01:00.0: Setting dpms mode 0 on CRTC 0
Dec 19 08:39:38 linux1 kernel: [drm] nouveau 0000:01:00.0: Setting dpms mode 0 on vga encoder (output 0)
Comment 9 Stephen Murray 2009-12-29 19:12:43 EST
OK. After running with the nVidia driver on my Fedora host the Vista guest still crashes with the same error. It has to be something in the Vista guest, but the fact that a very similar setup at the office works perfectly 100% of the time leaves me baffled. I have eliminated the obvious differences between the two. I'm out of ideas at this point.
Comment 10 Avi Kivity 2009-12-30 12:58:44 EST
Can we get a core and a stacktrace of the crash?
Comment 11 Stephen Murray 2009-12-30 13:55:30 EST
I have a 2.1GB coredump in "/var/cache/abrt/ccpp........." but after bzip2'ing it is still 845MB in size. Could I dump that entire directory to an ftp site or something ? Does the coredump contain a stacktrace ? If no, how could I get one ?
Comment 12 Avi Kivity 2009-12-31 02:35:01 EST
abrt should be able to extract the backtrace, or try

  gdb /path/to/qemu /path/to/core
  bt

You will need to install the debuginfo packages.
Comment 13 Stephen Murray 2009-12-31 09:50:54 EST
Here's the backtrace. How can I get the coredump to you ?

[root@linux1 ccpp-1262116035-2517]# gdb /usr/bin/qemu-kvm /var/cache/abrt/ccpp-1262116035-2517/coredump 
GNU gdb (GDB) Fedora (7.0-13.fc12)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/bin/qemu-kvm...(no debugging symbols found)...done.
[New Thread 2534]
[New Thread 7155]
[New Thread 2533]
Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libm.so.6
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/librt.so.1
Reading symbols from /lib64/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libz.so.1
Reading symbols from /lib64/libasound.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libasound.so.2
Reading symbols from /usr/lib64/libpulse-simple.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libpulse-simple.so.0
Reading symbols from /usr/lib64/libgnutls.so.26...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libgnutls.so.26
Reading symbols from /usr/lib64/libsasl2.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libsasl2.so.2
Reading symbols from /usr/lib64/libpci.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libpci.so.3
Reading symbols from /lib64/libutil.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libutil.so.1
Reading symbols from /usr/lib64/libSDL-1.2.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libSDL-1.2.so.0
Reading symbols from /usr/lib64/libX11.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libX11.so.6
Reading symbols from /lib64/libncurses.so.5...(no debugging symbols found)...done.
Loaded symbols for /lib64/libncurses.so.5
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /usr/lib64/libpulse.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libpulse.so.0
Reading symbols from /lib64/libtinfo.so.5...(no debugging symbols found)...done.
Loaded symbols for /lib64/libtinfo.so.5
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /usr/lib64/libpulsecommon-0.9.21.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libpulsecommon-0.9.21.so
Reading symbols from /usr/lib64/libSM.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libSM.so.6
Reading symbols from /usr/lib64/libICE.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libICE.so.6
Reading symbols from /usr/lib64/libXtst.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libXtst.so.6
Reading symbols from /lib64/libwrap.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib64/libwrap.so.0
Reading symbols from /usr/lib64/libsndfile.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libsndfile.so.1
Reading symbols from /usr/lib64/libasyncns.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libasyncns.so.0
Reading symbols from /lib64/libdbus-1.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdbus-1.so.3
Reading symbols from /usr/lib64/libtasn1.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libtasn1.so.3
Reading symbols from /lib64/libgcrypt.so.11...(no debugging symbols found)...done.
Loaded symbols for /lib64/libgcrypt.so.11
Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libresolv.so.2
Reading symbols from /lib64/libcrypt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libcrypt.so.1
Reading symbols from /usr/lib64/libxcb.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libxcb.so.1
Reading symbols from /lib64/libuuid.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libuuid.so.1
Reading symbols from /usr/lib64/libXext.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libXext.so.6
Reading symbols from /usr/lib64/libXi.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libXi.so.6
Reading symbols from /lib64/libnsl.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libnsl.so.1
Reading symbols from /usr/lib64/libFLAC.so.8...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libFLAC.so.8
Reading symbols from /usr/lib64/libvorbisenc.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libvorbisenc.so.2
Reading symbols from /usr/lib64/libvorbis.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libvorbis.so.0
Reading symbols from /usr/lib64/libogg.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libogg.so.0
Reading symbols from /lib64/libcap-ng.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib64/libcap-ng.so.0
Reading symbols from /lib64/libgpg-error.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib64/libgpg-error.so.0
Reading symbols from /usr/lib64/libfreebl3.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libfreebl3.so
Reading symbols from /usr/lib64/libXau.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libXau.so.6
Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libnss_files.so.2
Reading symbols from /usr/lib64/sasl2/libcrammd5.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/sasl2/libcrammd5.so
Reading symbols from /usr/lib64/sasl2/libsasldb.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/sasl2/libsasldb.so
Reading symbols from /lib64/libdb-4.7.so...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdb-4.7.so
Reading symbols from /usr/lib64/sasl2/liblogin.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/sasl2/liblogin.so
Reading symbols from /usr/lib64/sasl2/libplain.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/sasl2/libplain.so
Reading symbols from /usr/lib64/sasl2/libanonymous.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/sasl2/libanonymous.so
Reading symbols from /usr/lib64/sasl2/libdigestmd5.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/sasl2/libdigestmd5.so
Reading symbols from /usr/lib64/libcrypto.so.10...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libcrypto.so.10
Core was generated by `/usr/bin/qemu-kvm -S -M pc-0.11 -cpu qemu32 -m 2048 -smp 2 -name Vista -uuid 77'.
Program terminated with signal 11, Segmentation fault.
#0  0x00000031e5281510 in memset () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install qemu-system-x86-0.11.0-12.fc12.x86_64
(gdb) bt
#0  0x00000031e5281510 in memset () from /lib64/libc.so.6
#1  0x00000000004bac96 in pthread_attr_setdetachstate ()
#2  0x000000000040a82a in pthread_attr_setdetachstate ()
#3  0x00000000004231aa in pthread_attr_setdetachstate ()
#4  0x000000000040f157 in pthread_attr_setdetachstate ()
#5  0x00000031e521eb1d in __libc_start_main () from /lib64/libc.so.6
#6  0x0000000000408479 in pthread_attr_setdetachstate ()
#7  0x00007fff6aa86948 in ?? ()
#8  0x000000000000001c in ?? ()
#9  0x0000000000000028 in ?? ()
#10 0x00007fff6aa87d84 in ?? ()
#11 0x00007fff6aa87d96 in ?? ()
#12 0x00007fff6aa87d99 in ?? ()
#13 0x00007fff6aa87d9c in ?? ()
#14 0x00007fff6aa87da4 in ?? ()
#15 0x00007fff6aa87da9 in ?? ()
#16 0x00007fff6aa87db0 in ?? ()
#17 0x00007fff6aa87db3 in ?? ()
#18 0x00007fff6aa87db8 in ?? ()
#19 0x00007fff6aa87dbd in ?? ()
#20 0x00007fff6aa87dbf in ?? ()
#21 0x00007fff6aa87dc5 in ?? ()
#22 0x00007fff6aa87dcb in ?? ()
---Type <return> to continue, or q <return> to quit---
#23 0x00007fff6aa87dd1 in ?? ()
#24 0x00007fff6aa87df6 in ?? ()
#25 0x00007fff6aa87dff in ?? ()
#26 0x00007fff6aa87e36 in ?? ()
#27 0x00007fff6aa87e41 in ?? ()
#28 0x00007fff6aa87e47 in ?? ()
#29 0x00007fff6aa87e49 in ?? ()
#30 0x00007fff6aa87e50 in ?? ()
#31 0x00007fff6aa87e90 in ?? ()
#32 0x00007fff6aa87e97 in ?? ()
#33 0x00007fff6aa87ef1 in ?? ()
#34 0x00007fff6aa87ef6 in ?? ()
#35 0x00007fff6aa87f26 in ?? ()
#36 0x00007fff6aa87f2b in ?? ()
#37 0x00007fff6aa87f47 in ?? ()
#38 0x00007fff6aa87f4f in ?? ()
#39 0x00007fff6aa87f53 in ?? ()
#40 0x00007fff6aa87f5d in ?? ()
#41 0x00007fff6aa87f62 in ?? ()
#42 0x00007fff6aa87f67 in ?? ()
#43 0x00007fff6aa87f72 in ?? ()
#44 0x00007fff6aa87f79 in ?? ()
#45 0x00007fff6aa87f7e in ?? ()
---Type <return> to continue, or q <return> to quit---
#46 0x00007fff6aa87f8a in ?? ()
#47 0x00007fff6aa87f8f in ?? ()
#48 0x00007fff6aa87f96 in ?? ()
#49 0x00007fff6aa87f9f in ?? ()
#50 0x0000000000000000 in ?? ()
(gdb)
Comment 14 Avi Kivity 2009-12-31 11:28:20 EST
That's unusable.  Please install the debuginfo packages as I asked.

'debuginfo-install qemu-kvm' might work.
Comment 15 Stephen Murray 2010-01-01 17:51:01 EST
OK. I installed the debuginfo for qemu-kvm as you detailed, here is the new output:

[root@linux1 ccpp-1262116035-2517]# gdb /usr/bin/qemu-kvm /var/cache/abrt/ccpp-1262116035-2517/coredump 
GNU gdb (GDB) Fedora (7.0-13.fc12)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/bin/qemu-kvm...Reading symbols from /usr/lib/debug/usr/bin/qemu-kvm.debug...done.
done.
[New Thread 2534]
[New Thread 7155]
[New Thread 2533]
Missing separate debuginfo for 
Try: yum --enablerepo='*-debuginfo' install /usr/lib/debug/.build-id/ae/59b54e5901ca199abc40f4f70ea782019612d2
Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libm.so.6
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/librt.so.1
Reading symbols from /lib64/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libz.so.1
Reading symbols from /lib64/libasound.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libasound.so.2
Reading symbols from /usr/lib64/libpulse-simple.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libpulse-simple.so.0
Reading symbols from /usr/lib64/libgnutls.so.26...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libgnutls.so.26
Reading symbols from /usr/lib64/libsasl2.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libsasl2.so.2
Reading symbols from /usr/lib64/libpci.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libpci.so.3
Reading symbols from /lib64/libutil.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libutil.so.1
Reading symbols from /usr/lib64/libSDL-1.2.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libSDL-1.2.so.0
Reading symbols from /usr/lib64/libX11.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libX11.so.6
Reading symbols from /lib64/libncurses.so.5...(no debugging symbols found)...done.
Loaded symbols for /lib64/libncurses.so.5
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /usr/lib64/libpulse.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libpulse.so.0
Reading symbols from /lib64/libtinfo.so.5...(no debugging symbols found)...done.
Loaded symbols for /lib64/libtinfo.so.5
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /usr/lib64/libpulsecommon-0.9.21.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libpulsecommon-0.9.21.so
Reading symbols from /usr/lib64/libSM.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libSM.so.6
Reading symbols from /usr/lib64/libICE.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libICE.so.6
Reading symbols from /usr/lib64/libXtst.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libXtst.so.6
Reading symbols from /lib64/libwrap.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib64/libwrap.so.0
Reading symbols from /usr/lib64/libsndfile.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libsndfile.so.1
Reading symbols from /usr/lib64/libasyncns.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libasyncns.so.0
Reading symbols from /lib64/libdbus-1.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdbus-1.so.3
Reading symbols from /usr/lib64/libtasn1.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libtasn1.so.3
Reading symbols from /lib64/libgcrypt.so.11...(no debugging symbols found)...done.
Loaded symbols for /lib64/libgcrypt.so.11
Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libresolv.so.2
Reading symbols from /lib64/libcrypt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libcrypt.so.1
Reading symbols from /usr/lib64/libxcb.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libxcb.so.1
Reading symbols from /lib64/libuuid.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libuuid.so.1
Reading symbols from /usr/lib64/libXext.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libXext.so.6
Reading symbols from /usr/lib64/libXi.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libXi.so.6
Reading symbols from /lib64/libnsl.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libnsl.so.1
Reading symbols from /usr/lib64/libFLAC.so.8...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libFLAC.so.8
Reading symbols from /usr/lib64/libvorbisenc.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libvorbisenc.so.2
Reading symbols from /usr/lib64/libvorbis.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libvorbis.so.0
Reading symbols from /usr/lib64/libogg.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libogg.so.0
Reading symbols from /lib64/libcap-ng.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib64/libcap-ng.so.0
Reading symbols from /lib64/libgpg-error.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib64/libgpg-error.so.0
Reading symbols from /usr/lib64/libfreebl3.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libfreebl3.so
Reading symbols from /usr/lib64/libXau.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libXau.so.6
Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libnss_files.so.2
Reading symbols from /usr/lib64/sasl2/libcrammd5.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/sasl2/libcrammd5.so
Reading symbols from /usr/lib64/sasl2/libsasldb.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/sasl2/libsasldb.so
Reading symbols from /lib64/libdb-4.7.so...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdb-4.7.so
Reading symbols from /usr/lib64/sasl2/liblogin.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/sasl2/liblogin.so
Reading symbols from /usr/lib64/sasl2/libplain.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/sasl2/libplain.so
Reading symbols from /usr/lib64/sasl2/libanonymous.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/sasl2/libanonymous.so
Reading symbols from /usr/lib64/sasl2/libdigestmd5.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/sasl2/libdigestmd5.so
Reading symbols from /usr/lib64/libcrypto.so.10...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libcrypto.so.10
Core was generated by `/usr/bin/qemu-kvm -S -M pc-0.11 -cpu qemu32 -m 2048 -smp 2 -name Vista -uuid 77'.
Program terminated with signal 11, Segmentation fault.
#0  0x00000031e5281510 in memset () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install SDL-1.2.13-10.fc12.x86_64 alsa-lib-1.0.21-3.fc12.x86_64 cyrus-sasl-lib-2.1.23-4.fc12.x86_64 cyrus-sasl-md5-2.1.23-4.fc12.x86_64 cyrus-sasl-plain-2.1.23-4.fc12.x86_64 db4-4.7.25-13.fc12.x86_64 dbus-libs-1.2.16-8.fc12.x86_64 flac-1.2.1-6.fc12.x86_64 glibc-2.11-2.x86_64 gnutls-2.8.5-1.fc12.x86_64 libICE-1.0.6-1.fc12.x86_64 libSM-1.1.0-7.fc12.x86_64 libX11-1.3-1.fc12.x86_64 libXau-1.0.5-1.fc12.x86_64 libXext-1.1-2.fc12.x86_64 libXi-1.3-1.fc12.x86_64 libXtst-1.0.99.2-3.fc12.x86_64 libasyncns-0.8-1.fc12.x86_64 libcap-ng-0.6.2-3.fc12.x86_64 libgcrypt-1.4.4-8.fc12.x86_64 libgpg-error-1.6-4.x86_64 libogg-1.1.4-2.fc12.x86_64 libsndfile-1.0.20-3.fc12.x86_64 libtasn1-2.3-1.fc12.x86_64 libuuid-2.16.2-5.fc12.x86_64 libvorbis-1.2.3-3.fc12.x86_64 libxcb-1.4-2.fc12.x86_64 ncurses-libs-5.7-3.20090207.fc12.x86_64 nss-softokn-freebl-3.12.4-10.fc12.x86_64 openssl-1.0.0-0.13.beta4.fc12.x86_64 pciutils-libs-3.1.4-6.fc12.x86_64 pulseaudio-libs-0.9.21-1.fc12.x86_64 tcp_wrappers-libs-7.6-56.fc12.x86_64 zlib-1.2.3-23.fc12.x86_64
(gdb) bt
#0  0x00000031e5281510 in memset () from /lib64/libc.so.6
#1  0x00000000004bac96 in audio_capture_mix_and_clear (samples=-1255203771, 
    rpos=<value optimized out>, hw=<value optimized out>) at audio/audio.c:1290
#2  audio_run_out (samples=-1255203771, rpos=<value optimized out>, 
    hw=<value optimized out>) at audio/audio.c:1354
#3  audio_timer (samples=-1255203771, rpos=<value optimized out>, 
    hw=<value optimized out>) at audio/audio.c:1482
#4  0x000000000040a82a in qemu_run_timers (current_time=<value optimized out>, 
    ptimer_head=0x8322b8) at /usr/src/debug/qemu-kvm-0.11.0/vl.c:1036
#5  main_loop_wait (current_time=<value optimized out>, ptimer_head=0x8322b8)
    at /usr/src/debug/qemu-kvm-0.11.0/vl.c:4220
#6  0x00000000004231aa in kvm_main_loop ()
    at /usr/src/debug/qemu-kvm-0.11.0/qemu-kvm.c:2079
#7  0x000000000040f157 in main_loop ()
    at /usr/src/debug/qemu-kvm-0.11.0/vl.c:4393
#8  main () at /usr/src/debug/qemu-kvm-0.11.0/vl.c:6263
(gdb) quit
Comment 16 Avi Kivity 2010-01-03 15:42:30 EST
Weird.  samples is negative.

What audio driver is qemu using?

Suggest trying to reproduce with audio disabled just to confirm audio is the only cause.
Comment 17 Stephen Murray 2010-01-03 17:18:46 EST
The sound device defined in "/etc/libvirt/qemu/Vista.xml" was 

"<sound model='es1370'/>"

I removed it from the config and restarted everything. There is no longer a sound device in the virtual machine, Vista even complains about there being "no sound device".

I will let it run this way and update the bug report when I have more.
Comment 18 Stephen Murray 2010-01-10 13:14:30 EST
I have been running with sound removed from the guest xml file, and have had NO system crashes during that time. I will continue this way for another week, and if there are still no crashes I will reinstate the sound to see if the crash returns. Based solely upon these results to date, it does seem to be pointing to the audio function.

Assuming it is audio, is there any additional information I can extract from the existing coredumps that will help to diagnose the problem ?
Comment 19 Stephen Murray 2010-01-17 12:22:28 EST
The Vista guest ran for 2 weeks without a single failure, following removal of the es1370 sound device. I have put that device back into the config and restarted the guest. Let's see how long it can stay up.
Comment 20 alan 2010-01-17 13:09:16 EST
Could also make sense for my crashes - the desktop would no doubt be trying to use sound, the consoles wouldn't. I'll try the same experiment
Comment 21 Stephen Murray 2010-01-18 12:23:45 EST
OK, it ran for less than 24 hours before crashing with the old error. This, for me, confirms that the sound device is involved in this crash. What are our next steps ?

I am reverting to the "no sound device" config to keep my guest stable until we can fix this problem.
Comment 22 Stephen Murray 2010-02-06 09:14:06 EST
Hi. My Vista guest has been running without problem for 3 another weeks, clearly it is the sound device at fault. How do we fix this ? Do you need me to gather more information ?
Comment 23 alan 2010-02-08 11:17:11 EST
Ditto - in the Linux guest  case removing the sound driver fixes the bug

Sound + kvm only works properly on FC11
Comment 24 Justin M. Forbes 2010-02-09 10:40:36 EST
Could you test with the qemu from the virt-preview repository? Details for enabling the repository are available at https://fedoraproject.org/wiki/Virtualization_Preview_Repository
Comment 25 Stephen Murray 2010-02-10 14:59:19 EST
I installed the virt-preview, bounced libvirtd, re-defined the audio device (ES1310) to Vista, and started Vista.

I ran "rpm -qa" before/after to compare versions. Here is what I had before, and now have:

[root@linux1 ~]# grep -e libvirt -e qemu rpm-virt-*
rpm-virt-orig:gpxe-roms-qemu-0.9.7-6.fc12.noarch
rpm-virt-orig:libvirt-0.7.1-15.fc12.x86_64
rpm-virt-orig:libvirt-client-0.7.1-15.fc12.x86_64
rpm-virt-orig:libvirt-python-0.7.1-15.fc12.x86_64
rpm-virt-orig:qemu-common-0.11.0-12.fc12.x86_64
rpm-virt-orig:qemu-debuginfo-0.11.0-12.fc12.x86_64
rpm-virt-orig:qemu-img-0.11.0-12.fc12.x86_64
rpm-virt-orig:qemu-kvm-0.11.0-12.fc12.x86_64
rpm-virt-orig:qemu-system-x86-0.11.0-12.fc12.x86_64

rpm-virt-rawhide:gpxe-roms-qemu-0.9.7-6.fc12.noarch
rpm-virt-rawhide:libvirt-0.7.1-15.fc12.x86_64
rpm-virt-rawhide:libvirt-client-0.7.1-15.fc12.x86_64
rpm-virt-rawhide:libvirt-python-0.7.1-15.fc12.x86_64
rpm-virt-rawhide:qemu-common-0.12.2-6.fc12.x86_64
rpm-virt-rawhide:qemu-debuginfo-0.11.0-12.fc12.x86_64
rpm-virt-rawhide:qemu-img-0.11.0-12.fc12.x86_64
rpm-virt-rawhide:qemu-kvm-0.12.2-6.fc12.x86_64
rpm-virt-rawhide:qemu-system-x86-0.12.2-6.fc12.x86_64
[root@linux1 ~]# 

I will update the case as appropriate.
Comment 26 Justin M. Forbes 2010-02-12 15:36:30 EST
Odd to see that you have qemu-img and qemu-debuginfo from F12, with other bits from virt-preview.
Comment 27 Stephen Murray 2010-02-12 19:29:28 EST
Good eyes. And libvirt too. I have them all updated now. Oddly, the Vista guest has not crashed since I made this incomplete change on Sunday, that's 5 days. Possibly pointing to the problem being in one of the initial rpm's I replaced ?
Comment 28 Stephen Murray 2010-02-22 12:52:10 EST
OK. I have been running for 10 days with the packages from the virt-preview repository, and have experienced no failures relating to the sound device. I have also installed a new guests running Windows 2003 Server and this too is not crashing. It would seem that this release has the fix to correct the crash.

Do you need to get any further information from this environment ? I need to revert back to the "standard" packages, as the preview ones have an unusual flaw of their own which I don't particularly like (specifically, I cannot boot Windows guests directly from disk, it gives an I/O error. I have to boot from CD, let the option to "boot from this CD" expire, at which point the Windows Boot Manager on the CD successfully boots from disk. It's just plain wrong that Windows does something that we can't !).
Comment 29 Fedora Update System 2010-04-26 08:16:40 EDT
qemu-0.12.3-4.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/qemu-0.12.3-4.fc12
Comment 30 Fedora Update System 2010-04-27 21:12:25 EDT
qemu-0.12.3-4.fc12 has been pushed to the Fedora 12 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update qemu'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/qemu-0.12.3-4.fc12
Comment 31 Steven Pritchard 2010-06-08 12:57:17 EDT
FWIW, I experienced the problem described in comment 28 with not being able to boot from the VM's hard drive in Fedora 13 and had to reinstall the host with Fedora 12 to fix it.  Please make sure that problem is fixed before pushing qemu 0.12.x to Fedora 12.
Comment 32 Jon Ciesla 2010-06-10 08:17:03 EDT
Use of kvm in f13 on my Turions still instantly reboots my laptop.
Comment 33 Bug Zapper 2010-11-04 00:21:10 EDT
This message is a reminder that Fedora 12 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 12.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '12'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 12's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 12 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 34 Bug Zapper 2010-12-03 21:13:48 EST
Fedora 12 changed to end-of-life (EOL) status on 2010-12-02. Fedora 12 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

Note You need to log in before you can comment on or make changes to this bug.