Red Hat Bugzilla – Bug 1445174
[RHEV7.4] [guest memory dump]dump-guest-memory QMP command with "detach" param makes qemu-kvm process aborted
Last modified: 2017-08-02 00:35:59 EDT
Description of problem: [guest memory dump]dump-guest-memory QMP command with "detach" param makes qemu-kvm process aborted. (qemu) qemu-kvm: /builddir/build/BUILD/qemu-2.9.0/memory.c:914: memory_region_transaction_commit: Assertion `qemu_mutex_iothread_locked()' failed. Version-Release number of selected component (if applicable): Test on: qemu-kvm-rhev-2.9.0-1.el7.x86_64 Host: kernel-debuginfo-3.10.0-656.el7.x86_64 kernel-3.10.0-656.el7.x86_64 kernel-debuginfo-common-x86_64-3.10.0-656.el7.x86_64 How reproducible: 3/3 Steps to Reproduce: 1.Boot up a guest eg, /usr/libexec/qemu-kvm \ -name 'avocado-vt-vm1' \ -sandbox off \ -machine pc \ -nodefaults \ -vga cirrus \ -device pvpanic,ioport=0x505,id=idHT1RPm \ -device ich9-usb-ehci1,id=usb1,addr=0x1d.7,multifunction=on,bus=pci.0 \ -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=0x1d.0,firstport=0,bus=pci.0 \ -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=0x1d.2,firstport=2,bus=pci.0 \ -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=0x1d.4,firstport=4,bus=pci.0 \ -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel74-64-virtio.qcow2 \ -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=0x3 \ -device virtio-net-pci,mac=9a:1b:1c:1d:1e:1f,id=id8xeo6O,vectors=4,netdev=idBP1nUD,bus=pci.0,addr=0x4 \ -netdev tap,id=idBP1nUD \ -m 4086 \ -smp 4,cores=2,threads=1,sockets=2 \ -cpu 'Westmere',+kvm_pv_unhalt \ -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ -vnc :0 \ -rtc base=utc,clock=host,driftfix=slew \ -boot order=cdn,once=c,menu=off,strict=off \ -enable-kvm \ -monitor stdio \ -qmp tcp:localhost:4444,server,nowait \ 2. Check dump-guest-memory command. #telnet localhost 4444 QMP: { "execute": "qmp_capabilities" } {"return": {}} {"execute": "dump-guest-memory", "arguments": { "detach": true, "paging": false, "protocol": "file:/home/dump.normal"}} Actual results: QMP: {"timestamp": {"seconds": 1493106109, "microseconds": 745723}, "event": "STOP"} {"return": {}} {"timestamp": {"seconds": 1493106114, "microseconds": 299349}, "event": "DUMP_COMPLETED", "data": {"result": {"total": 4301455360, "status": "completed", "completed": 4301455360}}} Connection closed by foreign host. HMP: (qemu) qemu-kvm: /builddir/build/BUILD/qemu-2.9.0/memory.c:914: memory_region_transaction_commit: Assertion `qemu_mutex_iothread_locked()' failed. guest_dump.sh: line 26: 7342 Aborted /usr/libexec/qemu-kvm -name 'avocado-vt-vm1' -sandbox off -machine pc -nodefaults -vga cirrus -device pvpanic,ioport=0x505,id=idHT1RPm -device ich9-usb-ehci1,id=usb1,addr=0x1d.7,multifunction=on,bus=pci.0 -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=0x1d.0,firstport=0,bus=pci.0 -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=0x1d.2,firstport=2,bus=pci.0 -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=0x1d.4,firstport=4,bus=pci.0 -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel74-64-virtio.qcow2 -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=0x3 -device virtio-net-pci,mac=9a:1b:1c:1d:1e:1f,id=id8xeo6O,vectors=4,netdev=idBP1nUD,bus=pci.0,addr=0x4 -netdev tap,id=idBP1nUD -m 4086 -smp 4,cores=2,threads=1,sockets=2 -cpu 'Westmere',+kvm_pv_unhalt -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 -vnc :0 -rtc base=utc,clock=host,driftfix=slew -boot order=cdn,once=c,menu=off,strict=off -enable-kvm -monitor stdio -qmp tcp:localhost:4444,server,nowait Expected results: {"timestamp": {"seconds": 1489137867, "microseconds": 606103}, "event": "STOP"} {"return": {}} {"timestamp": {"seconds": 1489137868, "microseconds": 716173}, "event": "DUMP_COMPLETED", "data": {"result": {"total": 2164457472, "status": "completed", "completed": 2164457472}}} {"timestamp": {"seconds": 1489137868, "microseconds": 716518}, "event": "RESUME"} Additional info:
Host: qemu-kvm-rhev-2.8.0-5.el7.x86_64 kernel-3.10.0-566.el7.x86_64 kernel-debuginfo-common-x86_64-3.10.0-566.el7.x86_64 kernel-debuginfo-3.10.0-566.el7.x86_64 I have also tried on rhel7.3 host with qemu 2.8,for step 2: QMP: { "execute": "qmp_capabilities" } {"return": {}} {"execute": "dump-guest-memory", "arguments": { "detach": true, "paging": false, "protocol": "file:/home/dump.normal"}} {"timestamp": {"seconds": 1493102889, "microseconds": 979138}, "event": "STOP"} {"return": {}} {"timestamp": {"seconds": 1493102893, "microseconds": 330485}, "event": "DUMP_COMPLETED", "data": {"result": {"total": 4301455360, "status": "completed", "completed": 4301455360}}} {"timestamp": {"seconds": 1493102893, "microseconds": 330953}, "event": "RESUME"} {"execute": "query-dump"} {"return": {"total": 4301455360, "status": "completed", "completed": 4301455360}} HMP: works fine
I'm having difficulties reproducing locally. Can you provide a stack backtrace?
hachen helped me to reproduce on a lab machine. Start qemu-kvm under gdb as follows: # gdb --args `sed '/Westmere/d' <dump.sh | tr -d '\\\\'` GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-98.el7 [...] Reading symbols from /usr/libexec/qemu-kvm...Reading symbols from /usr/lib/debug/usr/libexec/qemu-kvm.debug...done. done. (gdb) r Starting program: /usr/libexec/qemu-kvm -name \'avocado-vt-vm1\' -sandbox off -machine pc -nodefaults -vga cirrus -device pvpanic,ioport=0x505,id=idHT1RPm -device ich9-usb-ehci1,id=usb1,addr=0x1d.7,multifunction=on,bus=pci.0 -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=0x1d.0,firstport=0,bus=pci.0 -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=0x1d.2,firstport=2,bus=pci.0 -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=0x1d.4,firstport=4,bus=pci.0 -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel74-64-virtio.qcow2 -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=0x3 -device virtio-net-pci,mac=9a:1b:1c:1d:1e:1f,id=id8xeo6O,vectors=4,netdev=idBP1nUD,bus=pci.0,addr=0x4 -netdev tap,id=idBP1nUD -m 8192 -smp 4,cores=2,threads=1,sockets=2 -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 -vnc :0 -rtc base=utc,clock=host,driftfix=slew -boot order=cdn,once=c,menu=off,strict=off -enable-kvm -monitor stdio -qmp tcp:localhost:4444,server,nowait QMP conversation in a second terminal: # telnet localhost 4444 Trying ::1... Connected to localhost. Escape character is '^]'. {"QMP": {"version": {"qemu": {"micro": 0, "minor": 9, "major": 2}, "package": "(qemu-kvm-rhev-2.9.0-1.el7)"}, "capabilities": []}} { "execute": "qmp_capabilities" } {"return": {}} {"execute": "dump-guest-memory", "arguments": { "detach": true, "paging": false, "protocol": "file:/home/dump.normal"}} {"timestamp": {"seconds": 1493713144, "microseconds": 617728}, "event": "STOP"} {"return": {}} {"timestamp": {"seconds": 1493713162, "microseconds": 369668}, "event": "DUMP_COMPLETED", "data": {"result": {"total": 8606908416, "status": "completed", "completed": 8606908416}}} Takes a few seconds to the STOP event, then some more to the DUMP_COMPLETED event. It seems to crash right after. Backtrace: qemu-kvm: /builddir/build/BUILD/qemu-2.9.0/memory.c:914: memory_region_transaction_commit: Assertion `qemu_mutex_iothread_locked()' failed. Program received signal SIGABRT, Aborted. [Switching to Thread 0x7fffe3028700 (LWP 32154)] 0x00007fffed9461f7 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 56 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig); Missing separate debuginfos, use: debuginfo-install boost-system-1.53.0-27.el7.x86_64 boost-thread-1.53.0-27.el7.x86_64 bzip2-libs-1.0.6-13.el7.x86_64 celt051-0.5.1.3-8.el7.x86_64 dbus-libs-1.6.12-17.el7.x86_64 elfutils-libelf-0.168-5.el7.x86_64 elfutils-libs-0.168-5.el7.x86_64 flac-libs-1.3.0-5.el7_1.x86_64 gmp-6.0.0-15.el7.x86_64 gsm-1.0.13-11.el7.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.15.1-5.el7.x86_64 libICE-1.0.9-5.el7.x86_64 libSM-1.2.2-2.el7.x86_64 libX11-1.6.4-4.el7.x86_64 libXau-1.0.8-2.1.el7.x86_64 libXext-1.3.3-3.el7.x86_64 libXi-1.7.9-1.el7.x86_64 libXtst-1.2.3-1.el7.x86_64 libasyncns-0.8-7.el7.x86_64 libattr-2.4.46-12.el7.x86_64 libcap-2.22-9.el7.x86_64 libcom_err-1.42.9-10.el7.x86_64 libdb-5.3.21-20.el7.x86_64 libffi-3.0.13-18.el7.x86_64 libgcrypt-1.5.3-14.el7.x86_64 libgpg-error-1.12-3.el7.x86_64 libibverbs-13-4.el7.x86_64 libidn-1.28-4.el7.x86_64 libjpeg-turbo-1.2.90-5.el7.x86_64 libnl3-3.2.28-3.el7_3.x86_64 libogg-1.3.0-7.el7.x86_64 librdmacm-13-4.el7.x86_64 libselinux-2.5-11.el7.x86_64 libsndfile-1.0.25-10.el7.x86_64 libtasn1-4.10-1.el7.x86_64 libunwind-1.2-2.el7.x86_64 libvorbis-1.3.3-8.el7.x86_64 libxcb-1.12-1.el7.x86_64 nspr-4.13.1-1.0.el7.x86_64 nss-3.28.3-5.el7.x86_64 nss-softokn-freebl-3.28.3-4.el7.x86_64 nss-util-3.28.3-3.el7.x86_64 openldap-2.4.44-3.el7.x86_64 openssl-libs-1.0.2k-5.el7.x86_64 p11-kit-0.23.5-1.el7.x86_64 pcre-8.32-17.el7.x86_64 systemd-libs-219-32.el7.x86_64 tcp_wrappers-libs-7.6-77.el7.x86_64 xz-libs-5.2.2-1.el7.x86_64 (gdb) set height 0 (gdb) thread apply all backtrace full Thread 9 (Thread 0x7fffe3028700 (LWP 32154)): #0 0x00007fffed9461f7 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 resultvar = 0 pid = 32053 selftid = 32154 #1 0x00007fffed9478e8 in __GI_abort () at abort.c:90 save_stage = 2 act = {__sigaction_handler = {sa_handler = 0x7fffffffe0a0, sa_sigaction = 0x7fffffffe0a0}, sa_mask = {__val = {140737180659568, 93824997708984, 914, 93825044486464, 140737179290563, 4, 140737001976688, 1483905344, 12899545671512211968, 93825022016112, 0, 0, 0, 21474836480, 140737180659568, 140737180671592}}, sa_flags = -135835648, sa_restorer = 0x7fffeda91e68} sigs = {__val = {32, 0 <repeats 15 times>}} #2 0x00007fffed93f266 in __assert_fail_base (fmt=0x7fffeda91e68 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x555555a87316 "qemu_mutex_iothread_locked()", file=file@entry=0x555555a8d4b8 "/builddir/build/BUILD/qemu-2.9.0/memory.c", line=line@entry=914, function=function@entry=0x555555a8dc20 <__PRETTY_FUNCTION__.28716> "memory_region_transaction_commit") at assert.c:92 str = 0x555556ca99e0 "" total = 4096 #3 0x00007fffed93f312 in __GI___assert_fail (assertion=assertion@entry=0x555555a87316 "qemu_mutex_iothread_locked()", file=file@entry=0x555555a8d4b8 "/builddir/build/BUILD/qemu-2.9.0/memory.c", line=line@entry=914, function=function@entry=0x555555a8dc20 <__PRETTY_FUNCTION__.28716> "memory_region_transaction_commit") at assert.c:101 #4 0x00005555557af9ca in memory_region_transaction_commit () at /usr/src/debug/qemu-2.9.0/memory.c:914 as = <optimized out> #5 0x00005555557b1611 in memory_region_add_eventfd (mr=mr@entry=0x55555904ce20, addr=<optimized out>, size=size@entry=0, match_data=<optimized out>, data=<optimized out>, e=<optimized out>) at /usr/src/debug/qemu-2.9.0/memory.c:1989 mrfd = {addr = {start = 0x00000000000000000000000000000000, size = 0x00000000000000000000000000000000}, match_data = false, data = 0, e = 0x5555590de068} i = <optimized out> #6 0x00005555559569b4 in virtio_pci_ioeventfd_assign (d=0x55555904c000, notifier=0x5555590de068, n=0, assign=<optimized out>) at hw/virtio/virtio-pci.c:304 proxy = 0x55555904c000 vdev = <optimized out> vq = <optimized out> legacy = true modern = true fast_mmio = <optimized out> modern_pio = false modern_mr = 0x55555904ce20 modern_notify_mr = 0x55555904cf30 legacy_mr = 0x55555904c9f0 modern_addr = <optimized out> legacy_addr = 16 #7 0x000055555595a320 in virtio_bus_set_host_notifier (bus=<optimized out>, n=n@entry=0, assign=assign@entry=true) at hw/virtio/virtio-bus.c:283 vdev = 0x555559054510 k = 0x555556cecb40 __func__ = "virtio_bus_set_host_notifier" proxy = 0x55555904c000 vq = <optimized out> notifier = 0x5555590de068 r = 0 #8 0x00005555557c7155 in virtio_blk_data_plane_start (vdev=<optimized out>) at /usr/src/debug/qemu-2.9.0/hw/block/dataplane/virtio-blk.c:188 vblk = 0x555559054510 __func__ = "virtio_blk_data_plane_start" s = 0x55555907b380 qbus = 0x555559054498 k = <optimized out> i = 0 nvqs = 1 r = <optimized out> #9 0x0000555555959fda in virtio_bus_start_ioeventfd (bus=0x555559054498) at hw/virtio/virtio-bus.c:223 k = 0x555556cecb40 __func__ = "virtio_bus_start_ioeventfd" proxy = 0x55555904c000 vdev = 0x555559054510 vdc = 0x555556ccc000 r = <optimized out> #10 0x00005555557eb5e4 in virtio_vmstate_change (opaque=0x555559054510, running=<optimized out>, state=<optimized out>) at /usr/src/debug/qemu-2.9.0/hw/virtio/virtio.c:2230 vdev = 0x555559054510 qbus = 0x555559054498 __func__ = "virtio_vmstate_change" k = 0x555556cecb40 backend_run = <optimized out> #11 0x0000555555877a02 in vm_state_notify (running=running@entry=1, state=state@entry=RUN_STATE_RUNNING) at vl.c:1595 e = <optimized out> next = 0x5555587299e0 #12 0x000055555579a300 in vm_prepare_start () at /usr/src/debug/qemu-2.9.0/cpus.c:1821 requested = RUN_STATE__MAX res = 0 #13 0x000055555579a369 in vm_start () at /usr/src/debug/qemu-2.9.0/cpus.c:1831 #14 0x00005555557b9c75 in dump_cleanup (s=s@entry=0x555556062f60 <dump_state_global>) at /usr/src/debug/qemu-2.9.0/dump.c:80 #15 0x00005555557ba8f4 in dump_process (s=0x555556062f60 <dump_state_global>, errp=errp@entry=0x7fffe3027980) at /usr/src/debug/qemu-2.9.0/dump.c:1687 local_err = 0x0 result = 0x555558729520 __PRETTY_FUNCTION__ = "dump_process" #16 0x00005555557bc094 in dump_thread (data=<optimized out>) at /usr/src/debug/qemu-2.9.0/dump.c:1694 err = 0x0 s = <optimized out> #17 0x00007fffedcdbdc5 in start_thread (arg=0x7fffe3028700) at pthread_create.c:308 __res = <optimized out> pd = 0x7fffe3028700 now = <optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737001981696, 5111790796432296765, 1, 140737001982400, 140737001981696, 50, -5111730018288579779, -5111760132582192323}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <optimized out> pagesize_m1 = <optimized out> sp = <optimized out> freesize = <optimized out> #18 0x00007fffeda0934d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 Thread 8 (Thread 0x7ffdde9ff700 (LWP 32129)): #0 0x00007fffedcdf6d5 in pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 #1 0x0000555555a6de39 in qemu_cond_wait (cond=cond@entry=0x555556cb6b80, mutex=mutex@entry=0x555556cb6bb0) at util/qemu-thread-posix.c:133 err = <optimized out> __func__ = "qemu_cond_wait" #2 0x00005555559ac84b in vnc_worker_thread_loop (queue=queue@entry=0x555556cb6b80) at ui/vnc-jobs.c:205 job = <optimized out> entry = <optimized out> tmp = <optimized out> vs = {sioc = 0x0, ioc = 0x0, ioc_tag = 0, disconnecting = 0, dirty = {{0, 0, 0} <repeats 2048 times>}, lossy_rect = 0x0, vd = 0x0, need_update = 0, force_update = 0, has_dirty = 0, features = 0, absolute = 0, last_x = 0, last_y = 0, last_bmask = 0, client_width = 0, client_height = 0, share_mode = 0, vnc_encoding = 0, major = 0, minor = 0, auth = 0, subauth = 0, challenge = '\000' <repeats 15 times>, tls = 0x0, sasl = {conn = 0x0, wantSSF = false, runSSF = false, waitWriteSSF = 0, encoded = 0x0, encodedLength = 0, encodedOffset = 0, username = 0x0, mechlist = 0x0}, encode_ws = false, websocket = false, info = 0x0, output = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, input = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, write_pixels = 0x0, client_pf = {bits_per_pixel = 0 '\000', bytes_per_pixel = 0 '\000', depth = 0 '\000', rmask = 0, gmask = 0, bmask = 0, amask = 0, rshift = 0 '\000', gshift = 0 '\000', bshift = 0 '\000', ashift = 0 '\000', rmax = 0 '\000', gmax = 0 '\000', bmax = 0 '\000', amax = 0 '\000', rbits = 0 '\000', gbits = 0 '\000', bbits = 0 '\000', abits = 0 '\000'}, client_format = 0, client_be = false, audio_cap = 0x0, as = {freq = 0, nchannels = 0, fmt = AUD_FMT_U8, endianness = 0}, read_handler = 0x0, read_handler_expect = 0, modifiers_state = '\000' <repeats 255 times>, abort = false, output_mutex = {lock = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}}, bh = 0x0, jobs_buffer = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, tight = {type = 0, quality = 0 '\000', compression = 0 '\000', pixel24 = 0 '\000', tight = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, tmp = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, zlib = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, gradient = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, png = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, levels = {0, 0, 0, 0}, stream = {{next_in = 0x0, avail_in = 0, total_in = 0, next_out = 0x0, avail_out = 0, total_out = 0, msg = 0x0, state = 0x0, zalloc = 0x0, zfree = 0x0, opaque = 0x0, data_type = 0, adler = 0, reserved = 0}, {next_in = 0x0, avail_in = 0, total_in = 0, next_out = 0x0, avail_out = 0, total_out = 0, msg = 0x0, state = 0x0, zalloc = 0x0, zfree = 0x0, opaque = 0x0, data_type = 0, adler = 0, reserved = 0}, {next_in = 0x0, avail_in = 0, total_in = 0, next_out = 0x0, avail_out = 0, total_out = 0, msg = 0x0, state = 0x0, zalloc = 0x0, zfree = 0x0, opaque = 0x0, data_type = 0, adler = 0, reserved = 0}, {next_in = 0x0, avail_in = 0, total_in = 0, next_out = 0x0, avail_out = 0, total_out = 0, msg = 0x0, state = 0x0, zalloc = 0x0, zfree = 0x0, opaque = 0x0, data_type = 0, adler = 0, reserved = 0}}}, zlib = {zlib = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, tmp = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, stream = {next_in = 0x0, avail_in = 0, total_in = 0, next_out = 0x0, avail_out = 0, total_out = 0, msg = 0x0, state = 0x0, zalloc = 0x0, zfree = 0x0, opaque = 0x0, data_type = 0, adler = 0, reserved = 0}, level = 0}, hextile = {send_tile = 0x0}, zrle = {type = 0, fb = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, zrle = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, tmp = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, zlib = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, stream = {next_in = 0x0, avail_in = 0, total_in = 0, next_out = 0x0, avail_out = 0, total_out = 0, msg = 0x0, state = 0x0, zalloc = 0x0, zfree = 0x0, opaque = 0x0, data_type = 0, adler = 0, reserved = 0}, palette = {pool = {{idx = 0, color = 0, next = {le_next = 0x0, le_prev = 0x0}} <repeats 256 times>}, size = 0, max = 0, bpp = 0, table = {{lh_first = 0x0} <repeats 256 times>}}}, zywrle = {buf = {0 <repeats 4096 times>}}, mouse_mode_notifier = {notify = 0x0, node = {le_next = 0x0, le_prev = 0x0}}, next = {tqe_next = 0x0, tqe_prev = 0x0}} n_rectangles = <optimized out> saved_offset = <optimized out> #3 0x00005555559acd88 in vnc_worker_thread (arg=0x555556cb6b80) at ui/vnc-jobs.c:312 queue = 0x555556cb6b80 #4 0x00007fffedcdbdc5 in start_thread (arg=0x7ffdde9ff700) at pthread_create.c:308 __res = <optimized out> pd = 0x7ffdde9ff700 now = <optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140728338478848, 5111790796432296765, 1, 140728338479552, 140728338478848, 93825016753024, -5112984591657514179, -5111760132582192323}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <optimized out> pagesize_m1 = <optimized out> sp = <optimized out> freesize = <optimized out> #5 0x00007fffeda0934d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 Thread 7 (Thread 0x7fffe0c20700 (LWP 32074)): #0 0x00007fffedcdf6d5 in pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 #1 0x0000555555a6de39 in qemu_cond_wait (cond=<optimized out>, mutex=mutex@entry=0x555556060fe0 <qemu_global_mutex>) at util/qemu-thread-posix.c:133 err = <optimized out> __func__ = "qemu_cond_wait" #2 0x000055555579947b in qemu_kvm_cpu_thread_fn (cpu=<optimized out>) at /usr/src/debug/qemu-2.9.0/cpus.c:1085 cpu = 0x5555571e4000 r = <optimized out> #3 0x000055555579947b in qemu_kvm_cpu_thread_fn (arg=0x5555571e4000) at /usr/src/debug/qemu-2.9.0/cpus.c:1123 cpu = 0x5555571e4000 r = <optimized out> #4 0x00007fffedcdbdc5 in start_thread (arg=0x7fffe0c20700) at pthread_create.c:308 __res = <optimized out> pd = 0x7fffe0c20700 now = <optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140736964200192, 5111790796432296765, 1, 140736964200896, 140736964200192, 93825022181376, -5111731646081184963, -5111760132582192323}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <optimized out> pagesize_m1 = <optimized out> sp = <optimized out> freesize = <optimized out> #5 0x00007fffeda0934d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 Thread 6 (Thread 0x7fffe1421700 (LWP 32073)): #0 0x00007fffedcdf6d5 in pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 #1 0x0000555555a6de39 in qemu_cond_wait (cond=<optimized out>, mutex=mutex@entry=0x555556060fe0 <qemu_global_mutex>) at util/qemu-thread-posix.c:133 err = <optimized out> __func__ = "qemu_cond_wait" #2 0x000055555579947b in qemu_kvm_cpu_thread_fn (cpu=<optimized out>) at /usr/src/debug/qemu-2.9.0/cpus.c:1085 cpu = 0x5555571c0000 r = <optimized out> #3 0x000055555579947b in qemu_kvm_cpu_thread_fn (arg=0x5555571c0000) at /usr/src/debug/qemu-2.9.0/cpus.c:1123 cpu = 0x5555571c0000 r = <optimized out> #4 0x00007fffedcdbdc5 in start_thread (arg=0x7fffe1421700) at pthread_create.c:308 __res = <optimized out> pd = 0x7fffe1421700 now = <optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140736972592896, 5111790796432296765, 1, 140736972593600, 140736972592896, 93825022033920, -5111734967701517507, -5111760132582192323}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <optimized out> pagesize_m1 = <optimized out> sp = <optimized out> freesize = <optimized out> #5 0x00007fffeda0934d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 Thread 5 (Thread 0x7fffe1c22700 (LWP 32071)): #0 0x00007fffedcdf6d5 in pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 #1 0x0000555555a6de39 in qemu_cond_wait (cond=<optimized out>, mutex=mutex@entry=0x555556060fe0 <qemu_global_mutex>) at util/qemu-thread-posix.c:133 err = <optimized out> __func__ = "qemu_cond_wait" #2 0x000055555579947b in qemu_kvm_cpu_thread_fn (cpu=<optimized out>) at /usr/src/debug/qemu-2.9.0/cpus.c:1085 cpu = 0x5555571a4000 r = <optimized out> #3 0x000055555579947b in qemu_kvm_cpu_thread_fn (arg=0x5555571a4000) at /usr/src/debug/qemu-2.9.0/cpus.c:1123 cpu = 0x5555571a4000 r = <optimized out> #4 0x00007fffedcdbdc5 in start_thread (arg=0x7fffe1c22700) at pthread_create.c:308 __res = <optimized out> pd = 0x7fffe1c22700 now = <optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140736980985600, 5111790796432296765, 1, 140736980986304, 140736980985600, 93825021919232, -5111733867653018819, -5111760132582192323}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <optimized out> pagesize_m1 = <optimized out> sp = <optimized out> freesize = <optimized out> #5 0x00007fffeda0934d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 Thread 4 (Thread 0x7fffe2423700 (LWP 32069)): #0 0x00007fffedcdf6d5 in pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 #1 0x0000555555a6de39 in qemu_cond_wait (cond=<optimized out>, mutex=mutex@entry=0x555556060fe0 <qemu_global_mutex>) at util/qemu-thread-posix.c:133 err = <optimized out> __func__ = "qemu_cond_wait" #2 0x000055555579947b in qemu_kvm_cpu_thread_fn (cpu=<optimized out>) at /usr/src/debug/qemu-2.9.0/cpus.c:1085 cpu = 0x55555712a000 r = <optimized out> #3 0x000055555579947b in qemu_kvm_cpu_thread_fn (arg=0x55555712a000) at /usr/src/debug/qemu-2.9.0/cpus.c:1123 cpu = 0x55555712a000 r = <optimized out> #4 0x00007fffedcdbdc5 in start_thread (arg=0x7fffe2423700) at pthread_create.c:308 __res = <optimized out> pd = 0x7fffe2423700 now = <optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140736989378304, 5111790796432296765, 1, 140736989379008, 140736989378304, 93825021419520, -5111728371705492675, -5111760132582192323}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <optimized out> pagesize_m1 = <optimized out> sp = <optimized out> freesize = <optimized out> #5 0x00007fffeda0934d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 Thread 2 (Thread 0x7fffe3a3c700 (LWP 32057)): #0 0x00007fffeda037f9 in syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38 #1 0x0000555555a6e146 in qemu_event_wait (val=<optimized out>, f=<optimized out>) at /usr/src/debug/qemu-2.9.0/include/qemu/futex.h:26 value = <optimized out> #2 0x0000555555a6e146 in qemu_event_wait (ev=ev@entry=0x55555647e544 <rcu_call_ready_event>) at util/qemu-thread-posix.c:399 value = <optimized out> #3 0x0000555555a7d76e in call_rcu_thread (opaque=<optimized out>) at util/rcu.c:249 tries = 0 n = <optimized out> node = <optimized out> #4 0x00007fffedcdbdc5 in start_thread (arg=0x7fffe3a3c700) at pthread_create.c:308 __res = <optimized out> pd = 0x7fffe3a3c700 now = <optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737012549376, 5111790796432296765, 1, 140737012550080, 140737012549376, 0, -5111728641751561411, -5111760132582192323}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <optimized out> pagesize_m1 = <optimized out> sp = <optimized out> freesize = <optimized out> #5 0x00007fffeda0934d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 Thread 1 (Thread 0x7ffff7f7ccc0 (LWP 32053)): #0 0x00007fffef1cba10 in g_array_set_size (farray=0x555556c89d00, length=length@entry=0) at garray.c:548 array = 0x555556c89d00 __FUNCTION__ = "g_array_set_size" #1 0x0000555555a6add7 in main_loop_wait (nonblocking=nonblocking@entry=0) at util/main-loop.c:501 ret = -1 timeout = 4294967295 timeout_ns = <optimized out> #2 0x000055555575bcdc in main () at vl.c:1898 i = <optimized out> snapshot = <optimized out> linux_boot = <optimized out> initrd_filename = <optimized out> kernel_filename = <optimized out> kernel_cmdline = <optimized out> boot_order = <optimized out> boot_once = 0x555556c80888 "c" cyls = <optimized out> heads = <optimized out> secs = <optimized out> translation = <optimized out> opts = <optimized out> machine_opts = <optimized out> hda_opts = <optimized out> icount_opts = <optimized out> accel_opts = <optimized out> olist = <optimized out> optind = 45 optarg = 0x7fffffffe4c3 "tcp:localhost:4444,server,nowait" loadvm = <optimized out> machine_class = 0x0 cpu_model = <optimized out> vga_model = 0x7fffffffe0ea "cirrus" qtest_chrdev = <optimized out> qtest_log = <optimized out> pid_file = <optimized out> incoming = <optimized out> defconfig = <optimized out> userconfig = <optimized out> nographic = <optimized out> display_type = <optimized out> display_remote = <optimized out> log_mask = <optimized out> log_file = <optimized out> trace_file = <optimized out> maxram_size = <optimized out> ram_slots = <optimized out> vmstate_dump_file = <optimized out> main_loop_err = 0x0 err = 0x0 list_data_dirs = <optimized out> bdo_queue = {sqh_first = 0x0, sqh_last = 0x7fffffffda10} __func__ = "main" __FUNCTION__ = "main" #3 0x000055555575bcdc in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4720 i = <optimized out> snapshot = <optimized out> linux_boot = <optimized out> initrd_filename = <optimized out> kernel_filename = <optimized out> kernel_cmdline = <optimized out> boot_order = <optimized out> boot_once = 0x555556c80888 "c" cyls = <optimized out> heads = <optimized out> secs = <optimized out> translation = <optimized out> opts = <optimized out> machine_opts = <optimized out> hda_opts = <optimized out> icount_opts = <optimized out> accel_opts = <optimized out> olist = <optimized out> optind = 45 optarg = 0x7fffffffe4c3 "tcp:localhost:4444,server,nowait" loadvm = <optimized out> machine_class = 0x0 cpu_model = <optimized out> vga_model = 0x7fffffffe0ea "cirrus" qtest_chrdev = <optimized out> qtest_log = <optimized out> pid_file = <optimized out> incoming = <optimized out> defconfig = <optimized out> userconfig = <optimized out> nographic = <optimized out> display_type = <optimized out> display_remote = <optimized out> log_mask = <optimized out> log_file = <optimized out> trace_file = <optimized out> maxram_size = <optimized out> ram_slots = <optimized out> vmstate_dump_file = <optimized out> main_loop_err = 0x0 err = 0x0 list_data_dirs = <optimized out> bdo_queue = {sqh_first = 0x0, sqh_last = 0x7fffffffda10} __func__ = "main" __FUNCTION__ = "main"
Proposed a fix for upstream: https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg00429.html
Fix included in qemu-kvm-rhev-2.9.0-5.el7
I test on qemu-kvm-rhev-2.9.0-5.el7.x86_64 Host: kernel-debuginfo-3.10.0-656.el7.x86_64 kernel-3.10.0-656.el7.x86_64 kernel-debuginfo-common-x86_64-3.10.0-656.el7.x86_64 { "execute": "qmp_capabilities" } {"return": {}} {"timestamp": {"seconds": 1495609901, "microseconds": 378052}, "event": "VNC_CONNECTED", "data": {"server": {"auth": "none", "family": "ipv4", "service": "5900", "host": "0.0.0.0", "websocket": false}, "client": {"family": "ipv4", "service": "58234", "host": "10.66.61.77", "websocket": false}}} {"timestamp": {"seconds": 1495609901, "microseconds": 396866}, "event": "VNC_INITIALIZED", "data": {"server": {"auth": "none", "family": "ipv4", "service": "5900", "host": "0.0.0.0", "websocket": false}, "client": {"family": "ipv4", "service": "58234", "host": "10.66.61.77", "websocket": false}}} {"execute": "dump-guest-memory", "arguments": { "detach": true, "paging": false, "protocol": "file:/home/dump.normal"}} {"timestamp": {"seconds": 1495609923, "microseconds": 605537}, "event": "STOP"} {"return": {}} {"timestamp": {"seconds": 1495609942, "microseconds": 84745}, "event": "DUMP_COMPLETED", "data": {"result": {"total": 8606908416, "status": "completed", "completed": 8606908416}}} {"timestamp": {"seconds": 1495609942, "microseconds": 85550}, "event": "RESUME"} Given information above, bug verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:2392