Bug 883654

Summary: qemu crashes when rebooting the guest: SpiceWorker-ERROR **: red_worker.c:4797:qxl_process_cursor: invalid cursor command 235
Product: Red Hat Enterprise Linux 6 Reporter: Xiaoqing Wei <xwei>
Component: qemu-kvmAssignee: Gerd Hoffmann <kraxel>
Status: CLOSED WONTFIX QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.4CC: acathrow, areis, bsarathy, dyasny, juzhang, kraxel, michen, mkenneth, qzhang, rhod, shuang, virt-maint
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-05-26 16:52:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
qemu qmp monitor none

Description Xiaoqing Wei 2012-12-05 04:46:53 UTC 
Description of problem:

qemu crashes when rebooting the guest: SpiceWorker-ERROR **: red_worker.c:4797:qxl_process_cursor: invalid cursor command 235
Version-Release number of selected component (if applicable):
qemu-kvm-rhev-0.12.1.2-2.340.el6.x86_64
seabios-0.6.1.2-25.el6.x86_64
sgabios-bin-0-0.3.20110621svn.el6.noarch
spice-server-0.12.0-7.el6.x86_64
vgabios-0.6b-3.7.el6.noarch
Guest: rhel6.3.64 w/ qxl driver installed(and runlevel 5)

How reproducible:
Only once (tried 200+ attempts, but didn't manage to reproduce)

Steps to Reproduce:
1. boot a guest w/qxl spice
/usr/local/staf/test/RHEV/kvm-new/autotest/client/tests/kvm/qemu -name 'vm1' -nodefaults -chardev socket,id=qmp_monitor_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20121204-141011-qcmV,server,nowait -mon chardev=qmp_monitor_id_qmpmonitor1,mode=control -chardev socket,id=serial_id_20121204-141011-qcmV,path=/tmp/serial-20121204-141011-qcmV,server,nowait -device isa-serial,chardev=serial_id_20121204-141011-qcmV -device ich9-usb-uhci1,id=usb1,bus=pci.0,addr=0x4 -drive file='/usr/local/staf/test/RHEV/kvm-new/autotest/client/tests/kvm/images/RHEL-Server-6.3-64-virtio.qcow2',if=none,id=drive-virtio-disk1,media=disk,cache=none,boot=off,snapshot=off,format=qcow2,aio=native -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1 -device virtio-net-pci,netdev=id9e6Dya,mac=9a:99:a3:46:7d:0a,id=ndev00id9e6Dya,bus=pci.0,addr=0x3 -netdev tap,id=id9e6Dya,vhost=on,fd=26 -m 2048 -smp 1,cores=0,threads=1,sockets=2 -cpu 'Opteron_G2' -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
\
 -spice port=3000,password=123456,addr=0,tls-port=3200,x509-dir=/tmp/spice_x509d,tls-channel=main,tls-channel=inputs,image-compression=auto_glz,jpeg-wan-compression=auto,zlib-glz-wan-compression=auto,streaming-video=all,agent-mouse=on,playback-compression=on,ipv4 \
 -vga qxl -global qxl-vga.vram_size=33554432 \
 \
 -rtc base=utc,clock=host,driftfix=slew -M rhel6.4.0 -boot order=cdn,once=c,menu=off    -no-kvm-pit-reinjection -enable-kvm 
2.
3.
  
Actual results:

qemu crashes
Expected results:

guest work well. no core dump
Additional info:


thread apply all bt
Core was generated by `/usr/local/staf/test/RHEV/kvm-new/autotest/client/tests/kvm/qemu -name vm1 -nod'.
Program terminated with signal 6, Aborted.
#0  0x00007f14ffcdc8a5 in raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
64	  return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
(gdb) 
Thread 4 (Thread 0x7f14f8a57700 (LWP 30954)):
#0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:136
#1  0x00007f1501cd5388 in _L_lock_854 () from /lib64/libpthread-2.12.so
#2  0x00007f1501cd5257 in __pthread_mutex_lock (mutex=0x7f15028879c0) at pthread_mutex_lock.c:61
#3  0x00007f150239d84a in kvm_main_loop_wait (env=0x7f1504dce600, timeout=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:1874
#4  0x00007f150239de9d in kvm_main_loop_cpu (_env=0x7f1504dce600) at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2007
#5  ap_main_loop (_env=0x7f1504dce600) at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2060
#6  0x00007f1501cd3851 in start_thread (arg=0x7f14f8a57700) at pthread_create.c:301
#7  0x00007f14ffd9290d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115

Thread 3 (Thread 0x7f15022ee980 (LWP 30943)):
#0  0x00007f1501cda54d in read () at ../sysdeps/unix/syscall-template.S:82
#1  0x00007f15004f4933 in read (fd=45, buf=0x7fff003cd61c "\025\177", size=4, block=<value optimized out>) at /usr/include/bits/unistd.h:45
#2  read_safe (fd=45, buf=0x7fff003cd61c "\025\177", size=4, block=<value optimized out>) at dispatcher.c:76
#3  0x00007f15004f4b66 in dispatcher_send_message (dispatcher=0x7f1504df7658, message_type=16, payload=0x7fff003cd650) at dispatcher.c:188
#4  0x00007f15004f517c in red_dispatcher_add_memslot (qxl_worker=<value optimized out>, mem_slot=<value optimized out>) at red_dispatcher.c:393
#5  qxl_worker_add_memslot (qxl_worker=<value optimized out>, mem_slot=<value optimized out>) at red_dispatcher.c:400
#6  0x00007f1502407705 in qemu_spice_create_host_memslot (ssd=0x7f15057f0ab0) at ui/spice-display.c:333
#7  0x00007f150250cae9 in qxl_hard_reset (d=0x7f15057f0840, loadvm=0) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/qxl.c:1100
#8  0x00007f1502376be2 in qemu_system_reset (report=true) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:3332
#9  0x00007f150239aed0 in qemu_kvm_system_reset (report=true) at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:1978
#10 0x00007f150239b0d3 in kvm_main_loop () at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2258
#11 0x00007f150237bbd8 in main_loop (argc=44, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4187
#12 main (argc=44, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:6525

Thread 2 (Thread 0x7f14fa5f5700 (LWP 1320)):
#0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:216
#1  0x00007f15023b8c47 in cond_timedwait (unused=<value optimized out>) at posix-aio-compat.c:102
#2  aio_thread (unused=<value optimized out>) at posix-aio-compat.c:329
#3  0x00007f1501cd3851 in start_thread (arg=0x7f14fa5f5700) at pthread_create.c:301
#4  0x00007f14ffd9290d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115

Thread 1 (Thread 0x7f14f0bc4700 (LWP 30962)):
#0  0x00007f14ffcdc8a5 in raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1  0x00007f14ffcde085 in abort () at abort.c:92
#2  0x00007f15005370d5 in spice_logv (log_domain=0x7f15005b3314 "SpiceWorker", log_level=SPICE_LOG_LEVEL_ERROR, strloc=0x7f15005b37e7 "red_worker.c:4797", function=0x7f15005b56f0 "qxl_process_cursor", format=0x7f15005b37cd "invalid cursor command %u", args=0x7f14f0bc39d0) at log.c:109
#3  0x00007f150053720a in spice_log (log_domain=<value optimized out>, log_level=<value optimized out>, strloc=<value optimized out>, function=<value optimized out>, format=<value optimized out>) at log.c:123
#4  0x00007f15004fbf82 in qxl_process_cursor (worker=0x7f14600008c0, cursor_cmd=0x7f146021dc00, group_id=<value optimized out>) at red_worker.c:4797
#5  0x00007f15004fe032 in red_process_cursor (worker=0x7f14600008c0, ring_is_empty=0x7f14f0bc3bdc, max_pipe_size=50) at red_worker.c:4851
#6  0x00007f1500515e2d in red_worker_main (arg=<value optimized out>) at red_worker.c:11850
#7  0x00007f1501cd3851 in start_thread (arg=0x7f14f0bc4700) at pthread_create.c:301
#8  0x00007f14ffd9290d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115
(gdb) quit
Comment 1 Xiaoqing Wei 2012-12-05 04:53:03 UTC 
The test scenario is same as 
Bug 865767 - qemu crashed when rhel6.3 64 bit guest reboots
but the bt info looks differ and:
accord to https://bugzilla.redhat.com/show_bug.cgi?id=865767#c24 , that bz has fixed.
Comment 7 Xiaoqing Wei 2012-12-07 02:34:22 UTC 
Created attachment 659182 [details]
qemu qmp monitor
Comment 8 Gerd Hoffmann 2012-12-10 08:36:59 UTC 
Not sure I can do anything about it without a reproducer.

Given that it seems not to reproduce implies this could also be a hardware issue like a flipped bit in RAM.  Likewise can't see a justification for the Regression tag.

Also note that this could be a guest driver bug.

Is the core still somewhere?  If so, can you print cursor_cmd (thread 1, stackframe #4) struct content please?