Bug 1520729

Summary: qemu aborted (core dumped) when reboot guest with spice
Product: Red Hat Enterprise Linux 7 Reporter: hachen <hachen>
Component: qemu-kvm-rhevAssignee: Gerd Hoffmann <kraxel>
Status: CLOSED DUPLICATE QA Contact: Guo, Zhiyi <zhguo>
Severity: high Docs Contact:
Priority: medium    
Version: 7.5CC: areis, chayang, hachen, jinzhao, juzhang, knoel, kraxel, marcandre.lureau, michen, stefanha, virt-maint, yfu, yiwei, zhguo
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-06-27 06:21:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1519876, 1558351    
Attachments:
Description Flags
Autotest debug.log none

Description hachen 2017-12-05 03:17:14 UTC
Description of problem:

qemu aborted (core dumped) when boot up with spice

Version-Release number of selected component (if applicable):

compose: RHEL-7.5-20171130.0
HOST:
qemu-kvm-rhev-2.10.0-10.el7.x86_64
kernel-3.10.0-799.el7.x86_64
seabios-bin-1.11.0-1.el7.noarch
sgabios-bin-0.20110622svn-4.el7.noarch
spice-server-0.14.0-2.el7.x86_64


How reproducible: 1/4



Steps to Reproduce:
1.Boot up guest
MALLOC_PERTURB_=1  /usr/libexec/qemu-kvm \
    -S  \
    -name 'avocado-vt-vm1'  \
    -sandbox off  \
    -machine pc  \
    -nodefaults  \
    -vga qxl  \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/avocado_aIlgWo/monitor-qmpmonitor1-20171204-052648-QC2qZXm1,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/avocado_aIlgWo/monitor-catch_monitor-20171204-052648-QC2qZXm1,server,nowait \
    -mon chardev=qmp_id_catch_monitor,mode=control \
    -device pvpanic,ioport=0x505,id=idXnHHzs  \
    -chardev socket,id=serial_id_serial0,path=/var/tmp/avocado_aIlgWo/serial-serial0-20171204-052648-QC2qZXm1,server,nowait \
    -device isa-serial,chardev=serial_id_serial0  \
    -chardev socket,id=seabioslog_id_20171204-052648-QC2qZXm1,path=/var/tmp/avocado_aIlgWo/seabios-20171204-052648-QC2qZXm1,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20171204-052648-QC2qZXm1,iobase=0x402 \
    -device ich9-usb-ehci1,id=usb1,addr=0x1d.7,multifunction=on,bus=pci.0 \
    -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=0x1d.0,firstport=0,bus=pci.0 \
    -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=0x1d.2,firstport=2,bus=pci.0 \
    -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=0x1d.4,firstport=4,bus=pci.0 \
    -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel75-64-virtio.qcow2 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=0x3 \
    -device virtio-net-pci,mac=9a:2e:2f:30:31:32,id=idSfRP1I,vectors=4,netdev=idlpeTzy,bus=pci.0,addr=0x4  \
    -netdev tap,id=idlpeTzy,vhost=on,vhostfd=20,fd=19 \
    -m 2048  \
    -smp 8,maxcpus=8,cores=4,threads=1,sockets=2  \
    -cpu 'Haswell-noTSX',+kvm_pv_unhalt \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -spice port=3000,password=123456,addr=0,image-compression=auto_glz,zlib-glz-wan-compression=auto,streaming-video=all,agent-mouse=on,playback-compression=on,ipv4  \
    -rtc base=utc,clock=host,driftfix=slew  \
    -boot order=cdn,once=c,menu=off,strict=off \
    -enable-kvm

2.qmp: {'execute': 'cont'}

3. reboot guest

Actual results:
[qemu output] /tmp/aexpect_CfJph0bv/aexpect-3yRhav.sh: line 1: 14153 Aborted 

Expected results:


Additional info:

05:28:28 INFO | [qemu output]
05:28:28 INFO | [qemu output] (process:14153): Spice-^[[1;33mWARNING^[[0m **: display-channel.c:2431:display_channel_validate_surface: canvas address is 0x560384403b08 for 0 (and is NULL)
05:28:28 INFO | [qemu output]
05:28:28 INFO | [qemu output]
05:28:28 INFO | [qemu output] (process:14153): Spice-^[[1;33mWARNING^[[0m **: display-channel.c:2432:display_channel_validate_surface: failed on 0
05:28:28 INFO | [qemu output]
05:28:28 INFO | [qemu output] (process:14153): Spice-^[[1;35mCRITICAL^[[0m **: display-channel.c:2035:display_channel_update: condition `display_channel_validate_surface(display, surface_id)' failed
05:28:29 INFO | [qemu output] Thread 12 (Thread 0x7f32e342e700 (LWP 14154)):
05:28:29 INFO | [qemu output] #0  0x00007f32ea589c89 in syscall () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] #1  0x000056038179c740 in qemu_event_wait (val=<optimized out>, f=<optimized out>) at /usr/src/debug/qemu-2.10.0/include/qemu/futex.h:26
05:28:29 INFO | [qemu output] #2  0x000056038179c740 in qemu_event_wait (ev=ev@entry=0x5603821a4228 <rcu_call_ready_event>) at util/qemu-thread-posix.c:442
05:28:29 INFO | [qemu output] #3  0x00005603817ac62e in call_rcu_thread (opaque=<optimized out>) at util/rcu.c:249
05:28:29 INFO | [qemu output] #4  0x00007f32ea865dd5 in start_thread () at /lib64/libpthread.so.0
05:28:29 INFO | [qemu output] #5  0x00007f32ea58f94d in clone () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] Thread 11 (Thread 0x7f32e1241700 (LWP 14158)):
05:28:29 INFO | [qemu output] #0  0x00007f32ea5863b7 in ioctl () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] #1  0x00005603814d7335 in kvm_vcpu_ioctl (cpu=cpu@entry=0x560383598000, type=type@entry=44672) at /usr/src/debug/qemu-2.10.0/accel/kvm/kvm-all.c:2180
05:28:29 INFO | [qemu output] #2  0x00005603814d7403 in kvm_cpu_exec (cpu=cpu@entry=0x560383598000) at /usr/src/debug/qemu-2.10.0/accel/kvm/kvm-all.c:2017
05:28:29 INFO | [qemu output] #3  0x00005603814b6722 in qemu_kvm_cpu_thread_fn (arg=0x560383598000) at /usr/src/debug/qemu-2.10.0/cpus.c:1128
05:28:29 INFO | [qemu output] #4  0x00007f32ea865dd5 in start_thread () at /lib64/libpthread.so.0
05:28:29 INFO | [qemu output] #5  0x00007f32ea58f94d in clone () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] Thread 10 (Thread 0x7f32e0a40700 (LWP 14159)):
05:28:29 INFO | [qemu output] #0  0x00007f32ea5863b7 in ioctl () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] #1  0x00005603814d7335 in kvm_vcpu_ioctl (cpu=cpu@entry=0x5603835fa000, type=type@entry=44672) at /usr/src/debug/qemu-2.10.0/accel/kvm/kvm-all.c:2180
05:28:29 INFO | [qemu output] #2  0x00005603814d7403 in kvm_cpu_exec (cpu=cpu@entry=0x5603835fa000) at /usr/src/debug/qemu-2.10.0/accel/kvm/kvm-all.c:2017
05:28:29 INFO | [qemu output] #3  0x00005603814b6722 in qemu_kvm_cpu_thread_fn (arg=0x5603835fa000) at /usr/src/debug/qemu-2.10.0/cpus.c:1128
05:28:29 INFO | [qemu output] #4  0x00007f32ea865dd5 in start_thread () at /lib64/libpthread.so.0
05:28:29 INFO | [qemu output] #5  0x00007f32ea58f94d in clone () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] Thread 9 (Thread 0x7f32e023f700 (LWP 14160)):
05:28:29 INFO | [qemu output] #0  0x00007f32ea5863b7 in ioctl () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] #1  0x00005603814d7335 in kvm_vcpu_ioctl (cpu=cpu@entry=0x56038361a000, type=type@entry=44672) at /usr/src/debug/qemu-2.10.0/accel/kvm/kvm-all.c:2180
05:28:29 INFO | [qemu output] #2  0x00005603814d7403 in kvm_cpu_exec (cpu=cpu@entry=0x56038361a000) at /usr/src/debug/qemu-2.10.0/accel/kvm/kvm-all.c:2017
05:28:29 INFO | [qemu output] #3  0x00005603814b6722 in qemu_kvm_cpu_thread_fn (arg=0x56038361a000) at /usr/src/debug/qemu-2.10.0/cpus.c:1128
05:28:29 INFO | [qemu output] #4  0x00007f32ea865dd5 in start_thread () at /lib64/libpthread.so.0
05:28:29 INFO | [qemu output] #5  0x00007f32ea58f94d in clone () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] Thread 8 (Thread 0x7f32dfa3e700 (LWP 14161)):
05:28:29 INFO | [qemu output] #0  0x00007f32ea5863b7 in ioctl () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] #1  0x00005603814d7335 in kvm_vcpu_ioctl (cpu=cpu@entry=0x560383636000, type=type@entry=44672) at /usr/src/debug/qemu-2.10.0/accel/kvm/kvm-all.c:2180
05:28:29 INFO | [qemu output] #2  0x00005603814d7403 in kvm_cpu_exec (cpu=cpu@entry=0x560383636000) at /usr/src/debug/qemu-2.10.0/accel/kvm/kvm-all.c:2017
05:28:29 INFO | [qemu output] #3  0x00005603814b6722 in qemu_kvm_cpu_thread_fn (arg=0x560383636000) at /usr/src/debug/qemu-2.10.0/cpus.c:1128
05:28:29 INFO | [qemu output] #4  0x00007f32ea865dd5 in start_thread () at /lib64/libpthread.so.0
05:28:29 INFO | [qemu output] #5  0x00007f32ea58f94d in clone () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] Thread 7 (Thread 0x7f32df23d700 (LWP 14162)):
05:28:29 INFO | [qemu output] #0  0x00007f32ea5863b7 in ioctl () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] #1  0x00005603814d7335 in kvm_vcpu_ioctl (cpu=cpu@entry=0x560383656000, type=type@entry=44672) at /usr/src/debug/qemu-2.10.0/accel/kvm/kvm-all.c:2180
05:28:29 INFO | [qemu output] #2  0x00005603814d7403 in kvm_cpu_exec (cpu=cpu@entry=0x560383656000) at /usr/src/debug/qemu-2.10.0/accel/kvm/kvm-all.c:2017
05:28:29 INFO | [qemu output] #3  0x00005603814b6722 in qemu_kvm_cpu_thread_fn (arg=0x560383656000) at /usr/src/debug/qemu-2.10.0/cpus.c:1128
05:28:29 INFO | [qemu output] #4  0x00007f32ea865dd5 in start_thread () at /lib64/libpthread.so.0
05:28:29 INFO | [qemu output] #5  0x00007f32ea58f94d in clone () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] Thread 6 (Thread 0x7f32dea3c700 (LWP 14163)):
05:28:29 INFO | [qemu output] #0  0x00007f32ea5863b7 in ioctl () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] #1  0x00005603814d7335 in kvm_vcpu_ioctl (cpu=cpu@entry=0x560383678000, type=type@entry=44672) at /usr/src/debug/qemu-2.10.0/accel/kvm/kvm-all.c:2180
05:28:29 INFO | [qemu output] #2  0x00005603814d7403 in kvm_cpu_exec (cpu=cpu@entry=0x560383678000) at /usr/src/debug/qemu-2.10.0/accel/kvm/kvm-all.c:2017
05:28:29 INFO | [qemu output] #3  0x00005603814b6722 in qemu_kvm_cpu_thread_fn (arg=0x560383678000) at /usr/src/debug/qemu-2.10.0/cpus.c:1128
05:28:29 INFO | [qemu output] #4  0x00007f32ea865dd5 in start_thread () at /lib64/libpthread.so.0
05:28:29 INFO | [qemu output] #5  0x00007f32ea58f94d in clone () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] Thread 5 (Thread 0x7f32de23b700 (LWP 14164)):
05:28:29 INFO | [qemu output] #0  0x00007f32ea5863b7 in ioctl () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] #1  0x00005603814d7335 in kvm_vcpu_ioctl (cpu=cpu@entry=0x560383692000, type=type@entry=44672) at /usr/src/debug/qemu-2.10.0/accel/kvm/kvm-all.c:2180
05:28:29 INFO | [qemu output] #2  0x00005603814d7403 in kvm_cpu_exec (cpu=cpu@entry=0x560383692000) at /usr/src/debug/qemu-2.10.0/accel/kvm/kvm-all.c:2017
05:28:29 INFO | [qemu output] #3  0x00005603814b6722 in qemu_kvm_cpu_thread_fn (arg=0x560383692000) at /usr/src/debug/qemu-2.10.0/cpus.c:1128
05:28:29 INFO | [qemu output] #4  0x00007f32ea865dd5 in start_thread () at /lib64/libpthread.so.0
05:28:29 INFO | [qemu output] #5  0x00007f32ea58f94d in clone () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] Thread 4 (Thread 0x7f32dda3a700 (LWP 14165)):
05:28:29 INFO | [qemu output] #0  0x00007f32ea5863b7 in ioctl () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] #1  0x00005603814d7335 in kvm_vcpu_ioctl (cpu=cpu@entry=0x5603836b6000, type=type@entry=44672) at /usr/src/debug/qemu-2.10.0/accel/kvm/kvm-all.c:2180
05:28:29 INFO | [qemu output] #2  0x00005603814d7403 in kvm_cpu_exec (cpu=cpu@entry=0x5603836b6000) at /usr/src/debug/qemu-2.10.0/accel/kvm/kvm-all.c:2017
05:28:29 INFO | [qemu output] #3  0x00005603814b6722 in qemu_kvm_cpu_thread_fn (arg=0x5603836b6000) at /usr/src/debug/qemu-2.10.0/cpus.c:1128
05:28:29 INFO | [qemu output] #4  0x00007f32ea865dd5 in start_thread () at /lib64/libpthread.so.0
05:28:29 INFO | [qemu output] #5  0x00007f32ea58f94d in clone () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] Thread 3 (Thread 0x7f32547ff700 (LWP 14166)):
05:28:29 INFO | [qemu output] #0  0x00007f32ea86c75d in read () at /lib64/libpthread.so.0
05:28:29 INFO | [qemu output] #1  0x00007f32ec644ae1 in spice_backtrace_gstack () at /lib64/libspice-server.so.1
05:28:29 INFO | [qemu output] #2  0x00007f32ec64c457 in spice_log () at /lib64/libspice-server.so.1
05:28:29 INFO | [qemu output] #3  0x00007f32ec601308 in display_channel_update () at /lib64/libspice-server.so.1
05:28:29 INFO | [qemu output] #4  0x00007f32ec62fb8b in handle_dev_update_async () at /lib64/libspice-server.so.1
05:28:29 INFO | [qemu output] #5  0x00007f32ec5fb29d in dispatcher_handle_recv_read () at /lib64/libspice-server.so.1
05:28:29 INFO | [qemu output] #6  0x00007f32ec601aab in watch_func () at /lib64/libspice-server.so.1
05:28:29 INFO | [qemu output] #7  0x00007f32ec1078f9 in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
05:28:29 INFO | [qemu output] #8  0x00007f32ec107c58 in g_main_context_iterate.isra.22 () at /lib64/libglib-2.0.so.0
05:28:29 INFO | [qemu output] #9  0x00007f32ec107f2a in g_main_loop_run () at /lib64/libglib-2.0.so.0
05:28:29 INFO | [qemu output] #10 0x00007f32ec62ff2a in red_worker_main () at /lib64/libspice-server.so.1
05:28:29 INFO | [qemu output] #11 0x00007f32ea865dd5 in start_thread () at /lib64/libpthread.so.0
05:28:29 INFO | [qemu output] #12 0x00007f32ea58f94d in clone () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] Thread 2 (Thread 0x7f32e1e46700 (LWP 15037)):
05:28:29 INFO | [qemu output] #0  0x00007f32ea86bc71 in do_futex_wait () at /lib64/libpthread.so.0
05:28:29 INFO | [qemu output] #1  0x00007f32ea86bd37 in __new_sem_wait_slow () at /lib64/libpthread.so.0
05:28:29 INFO | [qemu output] #2  0x00007f32ea86bdd5 in sem_timedwait () at /lib64/libpthread.so.0
05:28:29 INFO | [qemu output] #3  0x000056038179c4d7 in qemu_sem_timedwait (sem=sem@entry=0x5603831c12d8, ms=ms@entry=10000) at util/qemu-thread-posix.c:289
05:28:29 INFO | [qemu output] #4  0x0000560381797eec in worker_thread (opaque=0x5603831c1260) at util/thread-pool.c:92
05:28:29 INFO | [qemu output] #5  0x00007f32ea865dd5 in start_thread () at /lib64/libpthread.so.0
05:28:29 INFO | [qemu output] #6  0x00007f32ea58f94d in clone () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] Thread 1 (Thread 0x7f32f4c00d00 (LWP 14153)):
05:28:29 INFO | [qemu output] #0  0x00007f32ea584daf in ppoll () at /lib64/libc.so.6
05:28:29 INFO | [qemu output] #1  0x0000560381798859 in qemu_poll_ns (__ss=0x0, __timeout=0x7ffdb2ca9350, __nfds=<optimized out>, __fds=<optimized out>) at /usr/include/bits/poll2.h:77
05:28:29 INFO | [qemu output] #2  0x0000560381798859 in qemu_poll_ns (fds=<optimized out>, nfds=<optimized out>, timeout=timeout@entry=1371811938) at util/qemu-timer.c:334
05:28:29 INFO | [qemu output] #3  0x0000560381799658 in main_loop_wait (timeout=1371811938) at util/main-loop.c:255
05:28:29 INFO | [qemu output] #4  0x0000560381799658 in main_loop_wait (nonblocking=nonblocking@entry=0) at util/main-loop.c:515
05:28:29 INFO | [qemu output] #5  0x000056038147c46a in main () at vl.c:1917
05:28:29 INFO | [qemu output] #6  0x000056038147c46a in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4804
05:28:29 INFO | [qemu output] /tmp/aexpect_CfJph0bv/aexpect-3yRhav.sh: line 1: 14153 Aborted                 MALLOC_PERTURB_=1 /usr/libexec/qemu-kvm -S -name 'avocado-vt-vm1' -sandbox off -machine pc -nodefaults -vga qxl -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/avocado_aIlgWo/monitor-qmpmonitor1-20171204-052648-QC2qZXm1,server,nowait -mon chardev=qmp_id_qmpmonitor1,mode=control -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/avocado_aIlgWo/monitor-catch_monitor-20171204-052648-QC2qZXm1,server,nowait -mon chardev=qmp_id_catch_monitor,mode=control -device pvpanic,ioport=0x505,id=idXnHHzs -chardev socket,id=serial_id_serial0,path=/var/tmp/avocado_aIlgWo/serial-serial0-20171204-052648-QC2qZXm1,server,nowait -device isa-serial,chardev=serial_id_serial0 -chardev socket,id=seabioslog_id_20171204-052648-QC2qZXm1,path=/var/tmp/avocado_aIlgWo/seabios-20171204-052648-QC2qZXm1,server,nowait -device isa-debugcon,chardev=seabioslog_id_20171204-052648-QC2qZXm1,iobase=0x402 -device ich9-usb-ehci1,id=usb1,addr=0x1d.7,multifunction=on,bus=pci.0 -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=0x1d.0,firstport=0,bus=pci.0 -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=0x1d.2,firstport=2,bus=pci.0 -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=0x1d.4,firstport=4,bus=pci.0 -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel75-64-virtio.qcow2 -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=0x3 -device virtio-net-pci,mac=9a:2e:2f:30:31:32,id=idSfRP1I,vectors=4,netdev=idlpeTzy,bus=pci.0,addr=0x4 -netdev tap,id=idlpeTzy,vhost=on,vhostfd=20,fd=19 -m 2048 -smp 8,maxcpus=8,cores=4,threads=1,sockets=2 -cpu 'Haswell-noTSX',+kvm_pv_unhalt -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 -spice port=3000,password=123456,addr=0,image-compression=auto_glz,zlib-glz-wan-compression=auto,streaming-video=all,agent-mouse=on,playback-compression=on,ipv4 -rtc base=utc,clock=host,driftfix=slew -boot order=cdn,once=c,menu=off,strict=off -enable-kvm
05:28:29 INFO | [qemu output] (Process terminated with status 134)

Comment 2 hachen 2017-12-05 03:22:24 UTC
It was not found in previous compose test for REHV 7.4.
Added keyworld: Regression

Comment 5 Gerd Hoffmann 2018-01-12 10:40:01 UTC
Any chance to rerun this test with tracing enabled?
If so, please enable qxl* and qemu_spice* and display* tracepoints.

Comment 6 hachen 2018-01-15 05:37:57 UTC
tested 6 times, can not reproduce it.
compose: RHEL-7.5-20180111.1
qemu-kvm-rhev-2.10.0-16.el7.x86_64 (function)
qemu-kvm-1.5.3-152.el7.x86_64 (install)
kernel-3.10.0-823.el7.x86_64
seabios-bin-1.11.0-1.el7.noarch
sgabios-bin-0.20110622svn-4.el7.noarch
spice-server-0.14.0-2.el7.x86_64

Comment 7 Gerd Hoffmann 2018-01-26 09:09:28 UTC
(In reply to hachen from comment #6)
> tested 6 times, can not reproduce it.

Hmm, original report says it reproduced only 1/4.
So I think 6 runs might simply not be enough to hit it.
Can you try a few more runs?

Comment 8 hachen 2018-01-31 01:44:30 UTC
Tested 60 times, can not reproduce it on the following compose:

compose: RHEL-7.5-20180125.0
qemu-kvm-rhev-2.10.0-18.el7.x86_64
kernel-3.10.0-837.el7.x86_64
seabios-bin-1.11.0-1.el7.noarch
sgabios-bin-0.20110622svn-4.el7.noarch
spice-server-0.14.0-2.el7.x86_64

Comment 9 Gerd Hoffmann 2018-01-31 07:19:42 UTC
(In reply to hachen from comment #8)
> Tested 60 times, can not reproduce it on the following compose:
> 
> compose: RHEL-7.5-20180125.0
> qemu-kvm-rhev-2.10.0-18.el7.x86_64
> kernel-3.10.0-837.el7.x86_64
> seabios-bin-1.11.0-1.el7.noarch
> sgabios-bin-0.20110622svn-4.el7.noarch
> spice-server-0.14.0-2.el7.x86_64

Thanks.  Seems some patch has fixed it then and we can close the bug.

Ademar?  What would be the correct reason in this case?  Most likely it is a duplicate of some other bug, but no I have idea which one ....
So go for nextrelease?  Or worksforme?

Comment 12 Yanan Fu 2018-02-06 13:30:02 UTC
I hit this problem today. It happened when reboot the guest repeatedly.
Auto case name: boot_vm_in_hugepage. (In this case, we boot vm with 2M hugepages, then reboot it for 10 times after guest bootup.  But i think this issue has nothing to do with hugepage, it just happen when reboot vm)


Test version:
qemu: qemu-kvm-rhev-2.10.0-19.el7.x86_64
kernel: kernel-3.10.0-843.el7.x86_64
spice: spice-server-0.14.0-2.el7.x86_64

Very low probability for this problem.

Full test log, you can refer, thanks!
http://10.66.4.244/autotest_static_job_log/2292995/test-results/101-smp_8.4096m.repeat1.Host_RHEL.m7.u5.spice.qcow2.virtio_scsi.up.virtio_net.RHEL.7.5.x86_64.io-github-autotest-qemu.boot_vm_in_hugepage/

Comment 13 Ademar Reis 2018-02-07 03:23:29 UTC
(In reply to Yanan Fu from comment #12)
> I hit this problem today. It happened when reboot the guest repeatedly.
> Auto case name: boot_vm_in_hugepage. (In this case, we boot vm with 2M
> hugepages, then reboot it for 10 times after guest bootup.  But i think this
> issue has nothing to do with hugepage, it just happen when reboot vm)

Moving it to 7.6, as I don't really consider it a blocker at this point. Given how difficult it is to reproduce it, are we even sure this is a regression? Still, it's worth investigating given it's a crash.

Comment 14 hachen 2018-02-12 08:14:00 UTC
Found in:
compose: RHEL-7.5-20180211.0
qemu-kvm-rhev-2.10.0-20.el7.x86_64
kernel-3.10.0-845.el7.x86_64
seabios-bin-1.11.0-2.el7.noarch
sgabios-bin-0.20110622svn-4.el7.noarch
spice-server-0.14.0-2.el7.x86_64

log:
http://10.8.242.200/2303656/test-results/08-smp_16.8192m.repeat1.Host_RHEL.m7.u5.spice.qcow2.virtio_blk.up.virtio_net.RHEL.7.5.x86_64.io-github-autotest-qemu.reboot/

Comment 15 Gerd Hoffmann 2018-02-12 15:38:05 UTC
(In reply to hachen from comment #14)
> Found in:
> compose: RHEL-7.5-20180211.0
> qemu-kvm-rhev-2.10.0-20.el7.x86_64
> kernel-3.10.0-845.el7.x86_64
> seabios-bin-1.11.0-2.el7.noarch
> sgabios-bin-0.20110622svn-4.el7.noarch
> spice-server-0.14.0-2.el7.x86_64
> 
> log:
> http://10.8.242.200/2303656/test-results/08-smp_16.8192m.repeat1.Host_RHEL.
> m7.u5.spice.qcow2.virtio_blk.up.virtio_net.RHEL.7.5.x86_64.io-github-
> autotest-qemu.reboot/

That is without tracing too (comment #5) I assume?

Comment 16 hachen 2018-03-05 02:56:12 UTC
(In reply to Gerd Hoffmann from comment #15)
> (In reply to hachen from comment #14)
> > Found in:
> > compose: RHEL-7.5-20180211.0
> > qemu-kvm-rhev-2.10.0-20.el7.x86_64
> > kernel-3.10.0-845.el7.x86_64
> > seabios-bin-1.11.0-2.el7.noarch
> > sgabios-bin-0.20110622svn-4.el7.noarch
> > spice-server-0.14.0-2.el7.x86_64
> > 
> > log:
> > http://10.8.242.200/2303656/test-results/08-smp_16.8192m.repeat1.Host_RHEL.
> > m7.u5.spice.qcow2.virtio_blk.up.virtio_net.RHEL.7.5.x86_64.io-github-
> > autotest-qemu.reboot/
> 
> That is without tracing too (comment #5) I assume?

Yes, Could you please tell me detailed steps about enabling the tracings (comment #5)?
I am not sure if what I am doing is correct or not.

Comment 17 Gerd Hoffmann 2018-03-07 12:44:44 UTC
Stefan, can you help lease?

What I want is just a log of the tracepoints given in comment 5.  With the "log" tracer backend it would be a simple "qemu -trace qxl*" + capture stdout.

The rhel binaries are built with the dtrace backend though.

stap stap -e 'probe qemu.kvm.simpletrace.qxl* {}' -x $(pidof qemu-kvm) seems to get me a log.  But it is simpletrace format not plaintext.  And the simpletrace.py tool and the trace-events file needed to turn that into something readable are not packaged in qemu-kvm-rhev.rpm ...

Comment 18 Stefan Hajnoczi 2018-03-09 15:27:33 UTC
(In reply to Gerd Hoffmann from comment #17)
> stap stap -e 'probe qemu.kvm.simpletrace.qxl* {}' -x $(pidof qemu-kvm) seems
> to get me a log.  But it is simpletrace format not plaintext.  And the
> simpletrace.py tool and the trace-events file needed to turn that into
> something readable are not packaged in qemu-kvm-rhev.rpm ...

You're on the right track.

The files you mentioned are part of the qemu-kvm-rhev and qemu-kvm-common-rhev RPMs:
 * The trace-events-all file is at /usr/share/qemu-kvm/trace-events-all.
 * The simpletrace.py pretty-printer is at /usr/share/qemu-kvm/simpletrace.py.

Please collect the trace like this:

 # stap -e 'probe qemu.kvm.simpletrace.qxl* {}' -x $(pidof qemu-kvm) >/tmp/trace
 # /usr/share/qemu-kvm/simpletrace.py --no-header /usr/share/qemu-kvm/trace-events-all /tmp/trace

Comment 19 Gerd Hoffmann 2018-06-19 12:31:30 UTC
https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=16779009
please test

If it still happens please try to capture a trace (see comment 5, comment 18).

Comment 20 hachen 2018-06-20 02:38:32 UTC
Please check #comment19

Comment 21 Gerd Hoffmann 2018-06-26 07:04:16 UTC
ping.
https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=16867229
new scratch build (old one expired meanwhile).

Comment 23 Yiqian Wei 2018-06-26 09:26:41 UTC
Created attachment 1454611 [details]
Autotest debug.log