1445174 – [RHEV7.4] [guest memory dump]dump-guest-memory QMP command with "detach" param makes qemu-kvm process aborted

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1445174 - [RHEV7.4] [guest memory dump]dump-guest-memory QMP command with "detach" param makes qemu-kvm process aborted

Summary: [RHEV7.4] [guest memory dump]dump-guest-memory QMP command with "detach" para...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	qemu-kvm-rhev
Sub Component:
Version:	7.4
Hardware:	x86_64
OS:	Linux
Priority:	unspecified
Severity:	unspecified
Target Milestone:	rc
Target Release:	---
Assignee:	Fam Zheng
QA Contact:	hachen
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2017-04-25 07:53 UTC by hachen
Modified:	2019-03-26 10:44 UTC (History)
CC List:	18 users (show)
Fixed In Version:	qemu-kvm-rhev-2.9.0-5.el7
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2017-08-02 04:35:59 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHSA-2017:2392	0	normal	SHIPPED_LIVE	Important: qemu-kvm-rhev security, bug fix, and enhancement update	2017-08-01 20:04:36 UTC

Description hachen 2017-04-25 07:53:10 UTC

Description of problem:
[guest memory dump]dump-guest-memory QMP command with "detach" param makes qemu-kvm process aborted.

(qemu) qemu-kvm: /builddir/build/BUILD/qemu-2.9.0/memory.c:914: memory_region_transaction_commit: Assertion `qemu_mutex_iothread_locked()' failed.


Version-Release number of selected component (if applicable):
Test on:
qemu-kvm-rhev-2.9.0-1.el7.x86_64

Host:
kernel-debuginfo-3.10.0-656.el7.x86_64
kernel-3.10.0-656.el7.x86_64
kernel-debuginfo-common-x86_64-3.10.0-656.el7.x86_64

How reproducible: 3/3


Steps to Reproduce:
1.Boot up a guest
eg,
/usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm1'  \
    -sandbox off  \
    -machine pc \
    -nodefaults  \
    -vga cirrus  \
    -device pvpanic,ioport=0x505,id=idHT1RPm  \
    -device ich9-usb-ehci1,id=usb1,addr=0x1d.7,multifunction=on,bus=pci.0 \
    -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=0x1d.0,firstport=0,bus=pci.0 \
    -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=0x1d.2,firstport=2,bus=pci.0 \
    -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=0x1d.4,firstport=4,bus=pci.0 \
    -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel74-64-virtio.qcow2 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=0x3 \
    -device virtio-net-pci,mac=9a:1b:1c:1d:1e:1f,id=id8xeo6O,vectors=4,netdev=idBP1nUD,bus=pci.0,addr=0x4  \
    -netdev tap,id=idBP1nUD \
    -m 4086 \
    -smp 4,cores=2,threads=1,sockets=2  \
    -cpu 'Westmere',+kvm_pv_unhalt \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -vnc :0  \
    -rtc base=utc,clock=host,driftfix=slew  \
    -boot order=cdn,once=c,menu=off,strict=off \
    -enable-kvm \
    -monitor stdio \
    -qmp tcp:localhost:4444,server,nowait \

2. Check dump-guest-memory command.
#telnet localhost 4444
QMP:
{ "execute": "qmp_capabilities" }
{"return": {}}
{"execute": "dump-guest-memory", "arguments": { "detach": true, "paging": false, "protocol": "file:/home/dump.normal"}}



Actual results:
QMP:
{"timestamp": {"seconds": 1493106109, "microseconds": 745723}, "event": "STOP"}
{"return": {}}
{"timestamp": {"seconds": 1493106114, "microseconds": 299349}, "event": "DUMP_COMPLETED", "data": {"result": {"total": 4301455360, "status": "completed", "completed": 4301455360}}}
Connection closed by foreign host.

HMP:
(qemu) qemu-kvm: /builddir/build/BUILD/qemu-2.9.0/memory.c:914: memory_region_transaction_commit: Assertion `qemu_mutex_iothread_locked()' failed.
guest_dump.sh: line 26:  7342 Aborted                 /usr/libexec/qemu-kvm -name 'avocado-vt-vm1' -sandbox off -machine pc -nodefaults -vga cirrus -device pvpanic,ioport=0x505,id=idHT1RPm -device ich9-usb-ehci1,id=usb1,addr=0x1d.7,multifunction=on,bus=pci.0 -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=0x1d.0,firstport=0,bus=pci.0 -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=0x1d.2,firstport=2,bus=pci.0 -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=0x1d.4,firstport=4,bus=pci.0 -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel74-64-virtio.qcow2 -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=0x3 -device virtio-net-pci,mac=9a:1b:1c:1d:1e:1f,id=id8xeo6O,vectors=4,netdev=idBP1nUD,bus=pci.0,addr=0x4 -netdev tap,id=idBP1nUD -m 4086 -smp 4,cores=2,threads=1,sockets=2 -cpu 'Westmere',+kvm_pv_unhalt -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 -vnc :0 -rtc base=utc,clock=host,driftfix=slew -boot order=cdn,once=c,menu=off,strict=off -enable-kvm -monitor stdio -qmp tcp:localhost:4444,server,nowait


Expected results:
{"timestamp": {"seconds": 1489137867, "microseconds": 606103}, "event": "STOP"}
{"return": {}}
{"timestamp": {"seconds": 1489137868, "microseconds": 716173}, "event": "DUMP_COMPLETED", "data": {"result": {"total": 2164457472, "status": "completed", "completed": 2164457472}}}
{"timestamp": {"seconds": 1489137868, "microseconds": 716518}, "event": "RESUME"}

Additional info:

Comment 2 hachen 2017-04-25 07:57:41 UTC

Host:
qemu-kvm-rhev-2.8.0-5.el7.x86_64

kernel-3.10.0-566.el7.x86_64
kernel-debuginfo-common-x86_64-3.10.0-566.el7.x86_64
kernel-debuginfo-3.10.0-566.el7.x86_64

I have also tried on rhel7.3 host with qemu 2.8,for step 2:
QMP:
{ "execute": "qmp_capabilities" }
{"return": {}}
{"execute": "dump-guest-memory", "arguments": { "detach": true, "paging": false, "protocol": "file:/home/dump.normal"}}
{"timestamp": {"seconds": 1493102889, "microseconds": 979138}, "event": "STOP"}
{"return": {}}
{"timestamp": {"seconds": 1493102893, "microseconds": 330485}, "event": "DUMP_COMPLETED", "data": {"result": {"total": 4301455360, "status": "completed", "completed": 4301455360}}}
{"timestamp": {"seconds": 1493102893, "microseconds": 330953}, "event": "RESUME"}
{"execute": "query-dump"}
{"return": {"total": 4301455360, "status": "completed", "completed": 4301455360}}

HMP:
works fine

Comment 5 Markus Armbruster 2017-04-26 12:27:32 UTC

I'm having difficulties reproducing locally.  Can you provide a stack backtrace?

Comment 8 Markus Armbruster 2017-05-02 08:41:23 UTC

hachen helped me to reproduce on a lab machine.

Start qemu-kvm under gdb as follows:

    # gdb --args `sed '/Westmere/d' <dump.sh | tr -d '\\\\'`
    GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-98.el7
    [...]
    Reading symbols from /usr/libexec/qemu-kvm...Reading symbols from /usr/lib/debug/usr/libexec/qemu-kvm.debug...done.
    done.
    (gdb) r
    Starting program: /usr/libexec/qemu-kvm -name \'avocado-vt-vm1\' -sandbox off -machine pc -nodefaults -vga cirrus -device pvpanic,ioport=0x505,id=idHT1RPm -device ich9-usb-ehci1,id=usb1,addr=0x1d.7,multifunction=on,bus=pci.0 -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=0x1d.0,firstport=0,bus=pci.0 -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=0x1d.2,firstport=2,bus=pci.0 -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=0x1d.4,firstport=4,bus=pci.0 -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel74-64-virtio.qcow2 -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=0x3 -device virtio-net-pci,mac=9a:1b:1c:1d:1e:1f,id=id8xeo6O,vectors=4,netdev=idBP1nUD,bus=pci.0,addr=0x4 -netdev tap,id=idBP1nUD -m 8192 -smp 4,cores=2,threads=1,sockets=2 -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 -vnc :0 -rtc base=utc,clock=host,driftfix=slew -boot order=cdn,once=c,menu=off,strict=off -enable-kvm -monitor stdio -qmp tcp:localhost:4444,server,nowait

QMP conversation in a second terminal:

    # telnet localhost 4444
    Trying ::1...
    Connected to localhost.
    Escape character is '^]'.
    {"QMP": {"version": {"qemu": {"micro": 0, "minor": 9, "major": 2}, "package": "(qemu-kvm-rhev-2.9.0-1.el7)"}, "capabilities": []}}
    { "execute": "qmp_capabilities" }
    {"return": {}}
    {"execute": "dump-guest-memory", "arguments": { "detach": true, "paging": false, "protocol": "file:/home/dump.normal"}}
    {"timestamp": {"seconds": 1493713144, "microseconds": 617728}, "event": "STOP"}
    {"return": {}}
    {"timestamp": {"seconds": 1493713162, "microseconds": 369668}, "event": "DUMP_COMPLETED", "data": {"result": {"total": 8606908416, "status": "completed", "completed": 8606908416}}}

Takes a few seconds to the STOP event, then some more to the DUMP_COMPLETED event.  It seems to crash right after.  Backtrace:

    qemu-kvm: /builddir/build/BUILD/qemu-2.9.0/memory.c:914: memory_region_transaction_commit: Assertion `qemu_mutex_iothread_locked()' failed.

    Program received signal SIGABRT, Aborted.
    [Switching to Thread 0x7fffe3028700 (LWP 32154)]
    0x00007fffed9461f7 in __GI_raise (sig=sig@entry=6)
	at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
    56	  return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
    Missing separate debuginfos, use: debuginfo-install boost-system-1.53.0-27.el7.x86_64 boost-thread-1.53.0-27.el7.x86_64 bzip2-libs-1.0.6-13.el7.x86_64 celt051-0.5.1.3-8.el7.x86_64 dbus-libs-1.6.12-17.el7.x86_64 elfutils-libelf-0.168-5.el7.x86_64 elfutils-libs-0.168-5.el7.x86_64 flac-libs-1.3.0-5.el7_1.x86_64 gmp-6.0.0-15.el7.x86_64 gsm-1.0.13-11.el7.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.15.1-5.el7.x86_64 libICE-1.0.9-5.el7.x86_64 libSM-1.2.2-2.el7.x86_64 libX11-1.6.4-4.el7.x86_64 libXau-1.0.8-2.1.el7.x86_64 libXext-1.3.3-3.el7.x86_64 libXi-1.7.9-1.el7.x86_64 libXtst-1.2.3-1.el7.x86_64 libasyncns-0.8-7.el7.x86_64 libattr-2.4.46-12.el7.x86_64 libcap-2.22-9.el7.x86_64 libcom_err-1.42.9-10.el7.x86_64 libdb-5.3.21-20.el7.x86_64 libffi-3.0.13-18.el7.x86_64 libgcrypt-1.5.3-14.el7.x86_64 libgpg-error-1.12-3.el7.x86_64 libibverbs-13-4.el7.x86_64 libidn-1.28-4.el7.x86_64 libjpeg-turbo-1.2.90-5.el7.x86_64 libnl3-3.2.28-3.el7_3.x86_64 libogg-1.3.0-7.el7.x86_64 librdmacm-13-4.el7.x86_64 libselinux-2.5-11.el7.x86_64 libsndfile-1.0.25-10.el7.x86_64 libtasn1-4.10-1.el7.x86_64 libunwind-1.2-2.el7.x86_64 libvorbis-1.3.3-8.el7.x86_64 libxcb-1.12-1.el7.x86_64 nspr-4.13.1-1.0.el7.x86_64 nss-3.28.3-5.el7.x86_64 nss-softokn-freebl-3.28.3-4.el7.x86_64 nss-util-3.28.3-3.el7.x86_64 openldap-2.4.44-3.el7.x86_64 openssl-libs-1.0.2k-5.el7.x86_64 p11-kit-0.23.5-1.el7.x86_64 pcre-8.32-17.el7.x86_64 systemd-libs-219-32.el7.x86_64 tcp_wrappers-libs-7.6-77.el7.x86_64 xz-libs-5.2.2-1.el7.x86_64
    (gdb) set height 0
    (gdb) thread apply all backtrace full

    Thread 9 (Thread 0x7fffe3028700 (LWP 32154)):
    #0  0x00007fffed9461f7 in __GI_raise (sig=sig@entry=6)
	at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
	    resultvar = 0
	    pid = 32053
	    selftid = 32154
    #1  0x00007fffed9478e8 in __GI_abort () at abort.c:90
	    save_stage = 2
	    act = 
	      {__sigaction_handler = {sa_handler = 0x7fffffffe0a0, sa_sigaction = 0x7fffffffe0a0}, sa_mask = {__val = {140737180659568, 93824997708984, 914, 93825044486464, 140737179290563, 4, 140737001976688, 1483905344, 12899545671512211968, 93825022016112, 0, 0, 0, 21474836480, 140737180659568, 140737180671592}}, sa_flags = -135835648, sa_restorer = 0x7fffeda91e68}
	    sigs = {__val = {32, 0 <repeats 15 times>}}
    #2  0x00007fffed93f266 in __assert_fail_base (fmt=0x7fffeda91e68 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x555555a87316 "qemu_mutex_iothread_locked()", file=file@entry=0x555555a8d4b8 "/builddir/build/BUILD/qemu-2.9.0/memory.c", line=line@entry=914, function=function@entry=0x555555a8dc20 <__PRETTY_FUNCTION__.28716> "memory_region_transaction_commit") at assert.c:92
	    str = 0x555556ca99e0 ""
	    total = 4096
    #3  0x00007fffed93f312 in __GI___assert_fail (assertion=assertion@entry=0x555555a87316 "qemu_mutex_iothread_locked()", file=file@entry=0x555555a8d4b8 "/builddir/build/BUILD/qemu-2.9.0/memory.c", line=line@entry=914, function=function@entry=0x555555a8dc20 <__PRETTY_FUNCTION__.28716> "memory_region_transaction_commit")
	at assert.c:101
    #4  0x00005555557af9ca in memory_region_transaction_commit ()
	at /usr/src/debug/qemu-2.9.0/memory.c:914
	    as = <optimized out>
    #5  0x00005555557b1611 in memory_region_add_eventfd (mr=mr@entry=0x55555904ce20, addr=<optimized out>, size=size@entry=0, match_data=<optimized out>, data=<optimized out>, e=<optimized out>) at /usr/src/debug/qemu-2.9.0/memory.c:1989
	    mrfd = 
	      {addr = {start = 0x00000000000000000000000000000000, size = 0x00000000000000000000000000000000}, match_data = false, data = 0, e = 0x5555590de068}
	    i = <optimized out>
    #6  0x00005555559569b4 in virtio_pci_ioeventfd_assign (d=0x55555904c000, notifier=0x5555590de068, n=0, assign=<optimized out>) at hw/virtio/virtio-pci.c:304
	    proxy = 0x55555904c000
	    vdev = <optimized out>
	    vq = <optimized out>
	    legacy = true
	    modern = true
	    fast_mmio = <optimized out>
	    modern_pio = false
	    modern_mr = 0x55555904ce20
	    modern_notify_mr = 0x55555904cf30
	    legacy_mr = 0x55555904c9f0
	    modern_addr = <optimized out>
	    legacy_addr = 16
    #7  0x000055555595a320 in virtio_bus_set_host_notifier (bus=<optimized out>, n=n@entry=0, assign=assign@entry=true) at hw/virtio/virtio-bus.c:283
	    vdev = 0x555559054510
	    k = 0x555556cecb40
	    __func__ = "virtio_bus_set_host_notifier"
	    proxy = 0x55555904c000
	    vq = <optimized out>
	    notifier = 0x5555590de068
	    r = 0
    #8  0x00005555557c7155 in virtio_blk_data_plane_start (vdev=<optimized out>)
	at /usr/src/debug/qemu-2.9.0/hw/block/dataplane/virtio-blk.c:188
	    vblk = 0x555559054510
	    __func__ = "virtio_blk_data_plane_start"
	    s = 0x55555907b380
	    qbus = 0x555559054498
	    k = <optimized out>
	    i = 0
	    nvqs = 1
	    r = <optimized out>
    #9  0x0000555555959fda in virtio_bus_start_ioeventfd (bus=0x555559054498)
	at hw/virtio/virtio-bus.c:223
	    k = 0x555556cecb40
	    __func__ = "virtio_bus_start_ioeventfd"
	    proxy = 0x55555904c000
	    vdev = 0x555559054510
	    vdc = 0x555556ccc000
	    r = <optimized out>
    #10 0x00005555557eb5e4 in virtio_vmstate_change (opaque=0x555559054510, running=<optimized out>, state=<optimized out>)
	at /usr/src/debug/qemu-2.9.0/hw/virtio/virtio.c:2230
	    vdev = 0x555559054510
	    qbus = 0x555559054498
	    __func__ = "virtio_vmstate_change"
	    k = 0x555556cecb40
	    backend_run = <optimized out>
    #11 0x0000555555877a02 in vm_state_notify (running=running@entry=1, state=state@entry=RUN_STATE_RUNNING) at vl.c:1595
	    e = <optimized out>
	    next = 0x5555587299e0
    #12 0x000055555579a300 in vm_prepare_start ()
	at /usr/src/debug/qemu-2.9.0/cpus.c:1821
	    requested = RUN_STATE__MAX
	    res = 0
    #13 0x000055555579a369 in vm_start () at /usr/src/debug/qemu-2.9.0/cpus.c:1831
    #14 0x00005555557b9c75 in dump_cleanup (s=s@entry=0x555556062f60 <dump_state_global>) at /usr/src/debug/qemu-2.9.0/dump.c:80
    #15 0x00005555557ba8f4 in dump_process (s=0x555556062f60 <dump_state_global>, errp=errp@entry=0x7fffe3027980) at /usr/src/debug/qemu-2.9.0/dump.c:1687
	    local_err = 0x0
	    result = 0x555558729520
	    __PRETTY_FUNCTION__ = "dump_process"
    #16 0x00005555557bc094 in dump_thread (data=<optimized out>)
	at /usr/src/debug/qemu-2.9.0/dump.c:1694
	    err = 0x0
	    s = <optimized out>
    #17 0x00007fffedcdbdc5 in start_thread (arg=0x7fffe3028700)
	at pthread_create.c:308
	    __res = <optimized out>
	    pd = 0x7fffe3028700
	    now = <optimized out>
	    unwind_buf = 
		  {cancel_jmp_buf = {{jmp_buf = {140737001981696, 5111790796432296765, 1, 140737001982400, 140737001981696, 50, -5111730018288579779, -5111760132582192323}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
	    not_first_call = <optimized out>
	    pagesize_m1 = <optimized out>
	    sp = <optimized out>
	    freesize = <optimized out>
    #18 0x00007fffeda0934d in clone ()
	at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

    Thread 8 (Thread 0x7ffdde9ff700 (LWP 32129)):
    #0  0x00007fffedcdf6d5 in pthread_cond_wait@@GLIBC_2.3.2 ()
	at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
    #1  0x0000555555a6de39 in qemu_cond_wait (cond=cond@entry=0x555556cb6b80, mutex=mutex@entry=0x555556cb6bb0) at util/qemu-thread-posix.c:133
	    err = <optimized out>
	    __func__ = "qemu_cond_wait"
    #2  0x00005555559ac84b in vnc_worker_thread_loop (queue=queue@entry=0x555556cb6b80) at ui/vnc-jobs.c:205
	    job = <optimized out>
	    entry = <optimized out>
	    tmp = <optimized out>
	    vs = 
		{sioc = 0x0, ioc = 0x0, ioc_tag = 0, disconnecting = 0, dirty = {{0, 0, 0} <repeats 2048 times>}, lossy_rect = 0x0, vd = 0x0, need_update = 0, force_update = 0, has_dirty = 0, features = 0, absolute = 0, last_x = 0, last_y = 0, last_bmask = 0, client_width = 0, client_height = 0, share_mode = 0, vnc_encoding = 0, major = 0, minor = 0, auth = 0, subauth = 0, challenge = '\000' <repeats 15 times>, tls = 0x0, sasl = {conn = 0x0, wantSSF = false, runSSF = false, waitWriteSSF = 0, encoded = 0x0, encodedLength = 0, encodedOffset = 0, username = 0x0, mechlist = 0x0}, encode_ws = false, websocket = false, info = 0x0, output = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, input = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, write_pixels = 0x0, client_pf = {bits_per_pixel = 0 '\000', bytes_per_pixel = 0 '\000', depth = 0 '\000', rmask = 0, gmask = 0, bmask = 0, amask = 0, rshift = 0 '\000', gshift = 0 '\000', bshift = 0 '\000', ashift = 0 '\000', rmax = 0 '\000', gmax = 0 '\000', bmax = 0 '\000', amax = 0 '\000', rbits = 0 '\000', gbits = 0 '\000', bbits = 0 '\000', abits = 0 '\000'}, client_format = 0, client_be = false, audio_cap = 0x0, as = {freq = 0, nchannels = 0, fmt = AUD_FMT_U8, endianness = 0}, read_handler = 0x0, read_handler_expect = 0, modifiers_state = '\000' <repeats 255 times>, abort = false, output_mutex = {lock = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}}, bh = 0x0, jobs_buffer = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, tight = {type = 0, quality = 0 '\000', compression = 0 '\000', pixel24 = 0 '\000', tight = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, tmp = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, zlib = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, gradient = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, png = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, levels = {0, 0, 0, 0}, stream = {{next_in = 0x0, avail_in = 0, total_in = 0, next_out = 0x0, avail_out = 0, total_out = 0, msg = 0x0, state = 0x0, zalloc = 0x0, zfree = 0x0, opaque = 0x0, data_type = 0, adler = 0, reserved = 0}, {next_in = 0x0, avail_in = 0, total_in = 0, next_out = 0x0, avail_out = 0, total_out = 0, msg = 0x0, state = 0x0, zalloc = 0x0, zfree = 0x0, opaque = 0x0, data_type = 0, adler = 0, reserved = 0}, {next_in = 0x0, avail_in = 0, total_in = 0, next_out = 0x0, avail_out = 0, total_out = 0, msg = 0x0, state = 0x0, zalloc = 0x0, zfree = 0x0, opaque = 0x0, data_type = 0, adler = 0, reserved = 0}, {next_in = 0x0, avail_in = 0, total_in = 0, next_out = 0x0, avail_out = 0, total_out = 0, msg = 0x0, state = 0x0, zalloc = 0x0, zfree = 0x0, opaque = 0x0, data_type = 0, adler = 0, reserved = 0}}}, zlib = {zlib = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, tmp = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, stream = {next_in = 0x0, avail_in = 0, total_in = 0, next_out = 0x0, avail_out = 0, total_out = 0, msg = 0x0, state = 0x0, zalloc = 0x0, zfree = 0x0, opaque = 0x0, data_type = 0, adler = 0, reserved = 0}, level = 0}, hextile = {send_tile = 0x0}, zrle = {type = 0, fb = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, zrle = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, tmp = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, zlib = {name = 0x0, capacity = 0, offset = 0, avg_size = 0, buffer = 0x0}, stream = {next_in = 0x0, avail_in = 0, total_in = 0, next_out = 0x0, avail_out = 0, total_out = 0, msg = 0x0, state = 0x0, zalloc = 0x0, zfree = 0x0, opaque = 0x0, data_type = 0, adler = 0, reserved = 0}, palette = {pool = {{idx = 0, color = 0, next = {le_next = 0x0, le_prev = 0x0}} <repeats 256 times>}, size = 0, max = 0, bpp = 0, table = {{lh_first = 0x0} <repeats 256 times>}}}, zywrle = {buf = {0 <repeats 4096 times>}}, mouse_mode_notifier = {notify = 0x0, node = {le_next = 0x0, le_prev = 0x0}}, next = {tqe_next = 0x0, tqe_prev = 0x0}}
	    n_rectangles = <optimized out>
	    saved_offset = <optimized out>
    #3  0x00005555559acd88 in vnc_worker_thread (arg=0x555556cb6b80)
	at ui/vnc-jobs.c:312
	    queue = 0x555556cb6b80
    #4  0x00007fffedcdbdc5 in start_thread (arg=0x7ffdde9ff700)
	at pthread_create.c:308
	    __res = <optimized out>
	    pd = 0x7ffdde9ff700
	    now = <optimized out>
	    unwind_buf = 
		  {cancel_jmp_buf = {{jmp_buf = {140728338478848, 5111790796432296765, 1, 140728338479552, 140728338478848, 93825016753024, -5112984591657514179, -5111760132582192323}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
	    not_first_call = <optimized out>
	    pagesize_m1 = <optimized out>
	    sp = <optimized out>
	    freesize = <optimized out>
    #5  0x00007fffeda0934d in clone ()
	at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

    Thread 7 (Thread 0x7fffe0c20700 (LWP 32074)):
    #0  0x00007fffedcdf6d5 in pthread_cond_wait@@GLIBC_2.3.2 ()
	at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
    #1  0x0000555555a6de39 in qemu_cond_wait (cond=<optimized out>, mutex=mutex@entry=0x555556060fe0 <qemu_global_mutex>) at util/qemu-thread-posix.c:133
	    err = <optimized out>
	    __func__ = "qemu_cond_wait"
    #2  0x000055555579947b in qemu_kvm_cpu_thread_fn (cpu=<optimized out>)
	at /usr/src/debug/qemu-2.9.0/cpus.c:1085
	    cpu = 0x5555571e4000
	    r = <optimized out>
    #3  0x000055555579947b in qemu_kvm_cpu_thread_fn (arg=0x5555571e4000)
	at /usr/src/debug/qemu-2.9.0/cpus.c:1123
	    cpu = 0x5555571e4000
	    r = <optimized out>
    #4  0x00007fffedcdbdc5 in start_thread (arg=0x7fffe0c20700)
	at pthread_create.c:308
	    __res = <optimized out>
	    pd = 0x7fffe0c20700
	    now = <optimized out>
	    unwind_buf = 
		  {cancel_jmp_buf = {{jmp_buf = {140736964200192, 5111790796432296765, 1, 140736964200896, 140736964200192, 93825022181376, -5111731646081184963, -5111760132582192323}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
	    not_first_call = <optimized out>
	    pagesize_m1 = <optimized out>
	    sp = <optimized out>
	    freesize = <optimized out>
    #5  0x00007fffeda0934d in clone ()
	at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

    Thread 6 (Thread 0x7fffe1421700 (LWP 32073)):
    #0  0x00007fffedcdf6d5 in pthread_cond_wait@@GLIBC_2.3.2 ()
	at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
    #1  0x0000555555a6de39 in qemu_cond_wait (cond=<optimized out>, mutex=mutex@entry=0x555556060fe0 <qemu_global_mutex>) at util/qemu-thread-posix.c:133
	    err = <optimized out>
	    __func__ = "qemu_cond_wait"
    #2  0x000055555579947b in qemu_kvm_cpu_thread_fn (cpu=<optimized out>)
	at /usr/src/debug/qemu-2.9.0/cpus.c:1085
	    cpu = 0x5555571c0000
	    r = <optimized out>
    #3  0x000055555579947b in qemu_kvm_cpu_thread_fn (arg=0x5555571c0000)
	at /usr/src/debug/qemu-2.9.0/cpus.c:1123
	    cpu = 0x5555571c0000
	    r = <optimized out>
    #4  0x00007fffedcdbdc5 in start_thread (arg=0x7fffe1421700)
	at pthread_create.c:308
	    __res = <optimized out>
	    pd = 0x7fffe1421700
	    now = <optimized out>
	    unwind_buf = 
		  {cancel_jmp_buf = {{jmp_buf = {140736972592896, 5111790796432296765, 1, 140736972593600, 140736972592896, 93825022033920, -5111734967701517507, -5111760132582192323}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
	    not_first_call = <optimized out>
	    pagesize_m1 = <optimized out>
	    sp = <optimized out>
	    freesize = <optimized out>
    #5  0x00007fffeda0934d in clone ()
	at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

    Thread 5 (Thread 0x7fffe1c22700 (LWP 32071)):
    #0  0x00007fffedcdf6d5 in pthread_cond_wait@@GLIBC_2.3.2 ()
	at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
    #1  0x0000555555a6de39 in qemu_cond_wait (cond=<optimized out>, mutex=mutex@entry=0x555556060fe0 <qemu_global_mutex>) at util/qemu-thread-posix.c:133
	    err = <optimized out>
	    __func__ = "qemu_cond_wait"
    #2  0x000055555579947b in qemu_kvm_cpu_thread_fn (cpu=<optimized out>)
	at /usr/src/debug/qemu-2.9.0/cpus.c:1085
	    cpu = 0x5555571a4000
	    r = <optimized out>
    #3  0x000055555579947b in qemu_kvm_cpu_thread_fn (arg=0x5555571a4000)
	at /usr/src/debug/qemu-2.9.0/cpus.c:1123
	    cpu = 0x5555571a4000
	    r = <optimized out>
    #4  0x00007fffedcdbdc5 in start_thread (arg=0x7fffe1c22700)
	at pthread_create.c:308
	    __res = <optimized out>
	    pd = 0x7fffe1c22700
	    now = <optimized out>
	    unwind_buf = 
		  {cancel_jmp_buf = {{jmp_buf = {140736980985600, 5111790796432296765, 1, 140736980986304, 140736980985600, 93825021919232, -5111733867653018819, -5111760132582192323}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
	    not_first_call = <optimized out>
	    pagesize_m1 = <optimized out>
	    sp = <optimized out>
	    freesize = <optimized out>
    #5  0x00007fffeda0934d in clone ()
	at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

    Thread 4 (Thread 0x7fffe2423700 (LWP 32069)):
    #0  0x00007fffedcdf6d5 in pthread_cond_wait@@GLIBC_2.3.2 ()
	at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
    #1  0x0000555555a6de39 in qemu_cond_wait (cond=<optimized out>, mutex=mutex@entry=0x555556060fe0 <qemu_global_mutex>) at util/qemu-thread-posix.c:133
	    err = <optimized out>
	    __func__ = "qemu_cond_wait"
    #2  0x000055555579947b in qemu_kvm_cpu_thread_fn (cpu=<optimized out>)
	at /usr/src/debug/qemu-2.9.0/cpus.c:1085
	    cpu = 0x55555712a000
	    r = <optimized out>
    #3  0x000055555579947b in qemu_kvm_cpu_thread_fn (arg=0x55555712a000)
	at /usr/src/debug/qemu-2.9.0/cpus.c:1123
	    cpu = 0x55555712a000
	    r = <optimized out>
    #4  0x00007fffedcdbdc5 in start_thread (arg=0x7fffe2423700)
	at pthread_create.c:308
	    __res = <optimized out>
	    pd = 0x7fffe2423700
	    now = <optimized out>
	    unwind_buf = 
		  {cancel_jmp_buf = {{jmp_buf = {140736989378304, 5111790796432296765, 1, 140736989379008, 140736989378304, 93825021419520, -5111728371705492675, -5111760132582192323}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
	    not_first_call = <optimized out>
	    pagesize_m1 = <optimized out>
	    sp = <optimized out>
	    freesize = <optimized out>
    #5  0x00007fffeda0934d in clone ()
	at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

    Thread 2 (Thread 0x7fffe3a3c700 (LWP 32057)):
    #0  0x00007fffeda037f9 in syscall ()
	at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
    #1  0x0000555555a6e146 in qemu_event_wait (val=<optimized out>, f=<optimized out>) at /usr/src/debug/qemu-2.9.0/include/qemu/futex.h:26
	    value = <optimized out>
    #2  0x0000555555a6e146 in qemu_event_wait (ev=ev@entry=0x55555647e544 <rcu_call_ready_event>) at util/qemu-thread-posix.c:399
	    value = <optimized out>
    #3  0x0000555555a7d76e in call_rcu_thread (opaque=<optimized out>)
	at util/rcu.c:249
	    tries = 0
	    n = <optimized out>
	    node = <optimized out>
    #4  0x00007fffedcdbdc5 in start_thread (arg=0x7fffe3a3c700)
	at pthread_create.c:308
	    __res = <optimized out>
	    pd = 0x7fffe3a3c700
	    now = <optimized out>
	    unwind_buf = 
		  {cancel_jmp_buf = {{jmp_buf = {140737012549376, 5111790796432296765, 1, 140737012550080, 140737012549376, 0, -5111728641751561411, -5111760132582192323}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
	    not_first_call = <optimized out>
	    pagesize_m1 = <optimized out>
	    sp = <optimized out>
	    freesize = <optimized out>
    #5  0x00007fffeda0934d in clone ()
	at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

    Thread 1 (Thread 0x7ffff7f7ccc0 (LWP 32053)):
    #0  0x00007fffef1cba10 in g_array_set_size (farray=0x555556c89d00, length=length@entry=0) at garray.c:548
	    array = 0x555556c89d00
	    __FUNCTION__ = "g_array_set_size"
    #1  0x0000555555a6add7 in main_loop_wait (nonblocking=nonblocking@entry=0)
	at util/main-loop.c:501
	    ret = -1
	    timeout = 4294967295
	    timeout_ns = <optimized out>
    #2  0x000055555575bcdc in main () at vl.c:1898
	    i = <optimized out>
	    snapshot = <optimized out>
	    linux_boot = <optimized out>
	    initrd_filename = <optimized out>
	    kernel_filename = <optimized out>
	    kernel_cmdline = <optimized out>
	    boot_order = <optimized out>
	    boot_once = 0x555556c80888 "c"
	    cyls = <optimized out>
	    heads = <optimized out>
	    secs = <optimized out>
	    translation = <optimized out>
	    opts = <optimized out>
	    machine_opts = <optimized out>
	    hda_opts = <optimized out>
	    icount_opts = <optimized out>
	    accel_opts = <optimized out>
	    olist = <optimized out>
	    optind = 45
	    optarg = 0x7fffffffe4c3 "tcp:localhost:4444,server,nowait"
	    loadvm = <optimized out>
	    machine_class = 0x0
	    cpu_model = <optimized out>
	    vga_model = 0x7fffffffe0ea "cirrus"
	    qtest_chrdev = <optimized out>
	    qtest_log = <optimized out>
	    pid_file = <optimized out>
	    incoming = <optimized out>
	    defconfig = <optimized out>
	    userconfig = <optimized out>
	    nographic = <optimized out>
	    display_type = <optimized out>
	    display_remote = <optimized out>
	    log_mask = <optimized out>
	    log_file = <optimized out>
	    trace_file = <optimized out>
	    maxram_size = <optimized out>
	    ram_slots = <optimized out>
	    vmstate_dump_file = <optimized out>
	    main_loop_err = 0x0
	    err = 0x0
	    list_data_dirs = <optimized out>
	    bdo_queue = {sqh_first = 0x0, sqh_last = 0x7fffffffda10}
	    __func__ = "main"
	    __FUNCTION__ = "main"
    #3  0x000055555575bcdc in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4720
	    i = <optimized out>
	    snapshot = <optimized out>
	    linux_boot = <optimized out>
	    initrd_filename = <optimized out>
	    kernel_filename = <optimized out>
	    kernel_cmdline = <optimized out>
	    boot_order = <optimized out>
	    boot_once = 0x555556c80888 "c"
	    cyls = <optimized out>
	    heads = <optimized out>
	    secs = <optimized out>
	    translation = <optimized out>
	    opts = <optimized out>
	    machine_opts = <optimized out>
	    hda_opts = <optimized out>
	    icount_opts = <optimized out>
	    accel_opts = <optimized out>
	    olist = <optimized out>
	    optind = 45
	    optarg = 0x7fffffffe4c3 "tcp:localhost:4444,server,nowait"
	    loadvm = <optimized out>
	    machine_class = 0x0
	    cpu_model = <optimized out>
	    vga_model = 0x7fffffffe0ea "cirrus"
	    qtest_chrdev = <optimized out>
	    qtest_log = <optimized out>
	    pid_file = <optimized out>
	    incoming = <optimized out>
	    defconfig = <optimized out>
	    userconfig = <optimized out>
	    nographic = <optimized out>
	    display_type = <optimized out>
	    display_remote = <optimized out>
	    log_mask = <optimized out>
	    log_file = <optimized out>
	    trace_file = <optimized out>
	    maxram_size = <optimized out>
	    ram_slots = <optimized out>
	    vmstate_dump_file = <optimized out>
	    main_loop_err = 0x0
	    err = 0x0
	    list_data_dirs = <optimized out>
	    bdo_queue = {sqh_first = 0x0, sqh_last = 0x7fffffffda10}
	    __func__ = "main"
	    __FUNCTION__ = "main"

Comment 9 Fam Zheng 2017-05-03 08:10:26 UTC

Proposed a fix for upstream:

https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg00429.html

Comment 11 Miroslav Rezanina 2017-05-16 13:03:26 UTC

Fix included in qemu-kvm-rhev-2.9.0-5.el7

Comment 13 hachen 2017-05-24 07:17:36 UTC

I test on 
qemu-kvm-rhev-2.9.0-5.el7.x86_64

Host:
kernel-debuginfo-3.10.0-656.el7.x86_64
kernel-3.10.0-656.el7.x86_64
kernel-debuginfo-common-x86_64-3.10.0-656.el7.x86_64

{ "execute": "qmp_capabilities" }
{"return": {}}
{"timestamp": {"seconds": 1495609901, "microseconds": 378052}, "event": "VNC_CONNECTED", "data": {"server": {"auth": "none", "family": "ipv4", "service": "5900", "host": "0.0.0.0", "websocket": false}, "client": {"family": "ipv4", "service": "58234", "host": "10.66.61.77", "websocket": false}}}
{"timestamp": {"seconds": 1495609901, "microseconds": 396866}, "event": "VNC_INITIALIZED", "data": {"server": {"auth": "none", "family": "ipv4", "service": "5900", "host": "0.0.0.0", "websocket": false}, "client": {"family": "ipv4", "service": "58234", "host": "10.66.61.77", "websocket": false}}}
{"execute": "dump-guest-memory", "arguments": { "detach": true, "paging": false, "protocol": "file:/home/dump.normal"}}
{"timestamp": {"seconds": 1495609923, "microseconds": 605537}, "event": "STOP"}
{"return": {}}
{"timestamp": {"seconds": 1495609942, "microseconds": 84745}, "event": "DUMP_COMPLETED", "data": {"result": {"total": 8606908416, "status": "completed", "completed": 8606908416}}}
{"timestamp": {"seconds": 1495609942, "microseconds": 85550}, "event": "RESUME"}

Given information above, bug verified.

Comment 15 errata-xmlrpc 2017-08-02 04:35:59 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Note You need to log in before you can comment on or make changes to this bug.