RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 988865 - Coredump when using QMP monitor directly after qemu execution
Summary: Coredump when using QMP monitor directly after qemu execution
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: qemu-kvm
Version: 6.5
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: rc
: ---
Assignee: Laszlo Ersek
QA Contact: Virtualization Bugs
URL:
Whiteboard:
: 1123904 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-07-26 15:15 UTC by Lukáš Doktor
Modified: 2015-01-27 22:07 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-11-14 16:11:34 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Lukáš Doktor 2013-07-26 15:15:17 UTC
How reproducible:
Always

Steps to Reproduce:
1. echo "you are dead" | qemu-kvm -qmp stdio

Actual results:
qemu coredump

Expected results:
survive this unsupported command

Additional info:
Fedora bug plus details here https://bugzilla.redhat.com/show_bug.cgi?id=947360

Comment 2 Sibiao Luo 2013-08-01 06:41:10 UTC
Reproduce this issue on qemu-kvm-rhev-0.12.1.2-2.382.el6.x86_64.

host info:
2.6.32-402.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.382.el6.x86_64

Steps:
1.start guest with -qmp unix:/tmp/a,server (without nowait).
2.# echo "you are dead" | /usr/libexec/qemu-kvm -qmp stdio
VNC server running on `::1:5900'
Segmentation fault (core dumped)

(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x00007ffbacf42702 in json_lexer_feed_char (lexer=0x7ffbae4bb570, ch=121 'y', flush=false)
    at /usr/src/debug/qemu-kvm-0.12.1.2/json-lexer.c:339
#2  0x00007ffbacf428b9 in json_lexer_feed (lexer=0x7ffbae4bb570, buffer=0x7fff448e85d0 "y", size=1)
    at /usr/src/debug/qemu-kvm-0.12.1.2/json-lexer.c:355
#3  0x00007ffbacee7fbb in monitor_control_read (opaque=<value optimized out>, buf=<value optimized out>, 
    size=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/monitor.c:5009
#4  0x00007ffbacf65e4c in qemu_chr_be_write (chan=<value optimized out>, cond=<value optimized out>, 
    opaque=0x7ffbae446c50) at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-char.c:192
#5  fd_chr_read (chan=<value optimized out>, cond=<value optimized out>, opaque=0x7ffbae446c50)
    at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-char.c:794
#6  0x00007ffbac56beb2 in g_main_context_dispatch () from /lib64/libglib-2.0.so.0
#7  0x00007ffbacee0bba in glib_select_poll (timeout=0) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:3992
#8  main_loop_wait (timeout=0) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4065
#9  0x00007ffbacf0356a in kvm_main_loop () at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2244
#10 0x00007ffbacee4558 in main_loop (argc=3, argv=<value optimized out>, envp=<value optimized out>)
    at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4259
#11 main (argc=3, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:6632
(gdb) 

Base on above, make qa_ack+ to it, please let me know if mistake, thanks.

Best Regards,
sluo

Comment 4 Laszlo Ersek 2013-08-05 09:54:18 UTC
This crash occurs because the monitor control read handler is invoked before the json parser used by the monitor is initialized. In other words, use-before-initialization.

"-qmp stdio" is equivalent to "-monitor stdio", just that it starts the monitor in control mode.

When qemu starts up, an stdio chardev is created for the monitor:

  qmp_chardev_add() / CHARDEV_BACKEND_KIND_STDIO
    qemu_chr_open_stdio()
      qemu_chr_open_fd()
        qemu_chr_generic_open()

qemu_chr_generic_open() schedules a CHR_EVENT_OPENED event as a bottom half (ie. asynchronoulsy).

Then the monitor is initialized and attached to the chardev:

  monitor_init()
    qemu_chr_add_handlers(... monitor_control_read, monitor_control_event, ...)

The below diagram describes what triggers the callbacks, and the dependency between the callbacks:

  bottom half scheduled                      input from stdin available
  qemu_chr_generic_open_bh()                             |
        |                                                |
        v                                                |
  CHR_EVENT_OPENED emitted                               |
  qemu_chr_be_event()                                    |
        |                                                |
        v                                                v
  monitor_control_event()                     monitor_control_read()
        |                                                |
        v                                                v
  json_message_parser_init()  <--depends---  json_message_parser_feed()

json_message_parser_feed() depends on json_message_parser_init() running first,
but they are both asynchronous handlers, and the events triggering them are not serialized and in this instance they happen to be delivered in the wrong order. IOW input from stdin is available earlier than CHR_EVENT_OPENED is issued (when the bottom half is scheduled).

This is fixed by upstream commit

  commit bd5c51ee6c4f1c79cae5ad2516d711a27b4ea8ec
  Author: Michael Roth <mdroth.ibm.com>
  Date:   Fri Jun 7 15:19:53 2013 -0500

      qemu-char: don't issue CHR_EVENT_OPEN in a BH

The upstream fix replaces the BH with a direct (synchronously) call. However that would break the original requirement for the BH ("issue the OPENED event only when chardev construction has completed"), so the upstream fix adds the direct call in a key spot where it is "late enough" for the original problem, and "early enough" so that it runs before the read handler.

This commit looks quite intrusive as it affects the chardev infrastructure and all chardev types. It also depends on some upstream commits that we don't have.

It's a further complication that the cited upstream commit didn't fix the problem completely. At least one followup was necessary for mux devices:

  commit 7b7ab18d0b9769b5f39e663fa55caed461b1202e
  Author: Michael Roth <mdroth.ibm.com>
  Date:   Tue Jul 30 13:04:22 2013 -0500

      chardev: fix CHR_EVENT_OPENED events for mux chardevs

This one depends on QOM stuff we don't have, and treats muxes as exceptional right in the CharDriverState struct.

I'd like to hear what others think about this BZ before embarking on the backport.

Comment 5 Laszlo Ersek 2013-08-08 11:40:17 UTC
The upstream fix, bd5c51ee, seems to be immature. Beside the followup mentioned in comment 4, 7b7ab18d, now another fixup has been posted:

http://lists.nongnu.org/archive/html/qemu-devel/2013-08/msg01015.html
[Qemu-devel] [PATCH] qemu-char: fix infinite recursion connecting to monitor pty

I think we should postpone this low-prio bug indefinitely, until the upstream fix matures.

Comment 6 Laszlo Ersek 2013-08-14 10:55:08 UTC
(In reply to Laszlo Ersek from comment #5)

> http://lists.nongnu.org/archive/html/qemu-devel/2013-08/msg01015.html
> [Qemu-devel] [PATCH] qemu-char: fix infinite recursion connecting to monitor
> pty

Committed in upstream as 3a3567d.

Comment 8 Ademar Reis 2015-01-27 22:07:34 UTC
*** Bug 1123904 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.