Bug 1060024
| Summary: | QMP socket breaks on unexpected close | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | CROC Cloud <cloud> | ||||
| Component: | qemu-kvm | Assignee: | Amit Shah <amit.shah> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> | ||||
| Severity: | urgent | Docs Contact: | |||||
| Priority: | urgent | ||||||
| Version: | 6.5 | CC: | amit.shah, areis, bsarathy, chayang, jkurik, juzhang, konishchev, mkenneth, mrezanin, qzhang, rbalakri, rhod, tdosek, ukar, virt-maint | ||||
| Target Milestone: | rc | Keywords: | Regression, ZStream | ||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | qemu-kvm-0.12.1.2-2.421.el6 | Doc Type: | Bug Fix | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | |||||||
| : | 1065225 (view as bug list) | Environment: | |||||
| Last Closed: | 2014-10-14 06:55:22 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1065225, 1069239, 1069582 | ||||||
| Attachments: |
|
||||||
|
Description
CROC Cloud
2014-01-31 09:35:53 UTC
Doesn't reproduce upstream.
Attaching gdb shows:
(gdb) t a a bt
Thread 2 (Thread 0x7f7c14f53700 (LWP 31911)):
#0 0x00007f7c1a7cfb37 in ioctl () from /lib64/libc.so.6
#1 0x00007f7c1d872f0a in kvm_run (env=0x7f7c1ec2a6b0)
at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:1015
#2 0x00007f7c1d8733c9 in kvm_cpu_exec (env=<value optimized out>)
at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:1744
#3 0x00007f7c1d8742ad in kvm_main_loop_cpu (_env=0x7f7c1ec2a6b0)
at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2005
#4 ap_main_loop (_env=0x7f7c1ec2a6b0)
at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2061
#5 0x00007f7c1d1869d1 in start_thread () from /lib64/libpthread.so.0
#6 0x00007f7c1a7d7b6d in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x7f7c1d792980 (LWP 31907)):
#0 0x00007f7c1a7d05e3 in select () from /lib64/libc.so.6
#1 0x00007f7c1d84e388 in main_loop_wait (timeout=1000)
at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4046
#2 0x00007f7c1d87144a in kvm_main_loop ()
at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2245
#3 0x00007f7c1d8512d9 in main_loop (argc=3, argv=<value optimized out>,
envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4266
#4 main (argc=3, argv=<value optimized out>, envp=<value optimized out>)
at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:6644
On RHEL6, the program shows:
Connecting to QMP...
OK
Connecting using socket...
OK
Connecting to QMP...
<and gets stuck>
On upstream, it shows:
Connecting to QMP...
OK
Connecting using socket...
OK
Connecting to QMP...
Traceback (most recent call last):
File "/var/tmp/break_qmp.py", line 13, in <module>
c.connect()
File "/home/amit/src/qemu/scripts/qmp/qmp.py", line 87, in connect
return self.__negotiate_capabilities()
File "/home/amit/src/qemu/scripts/qmp/qmp.py", line 59, in __negotiate_capabilities
raise QMPCapabilitiesError
qmp.QMPCapabilitiesError
and exits.
This is a high priority bug for us. Could you provide a patch which fixes the issue? CROC Cloud: thanks for taking the time to enter a bug report with us. We appreciate the feedback and look to use reports such as this to guide our efforts at improving our products. That being said, we're not able to guarantee the timeliness or suitability of a resolution for issues entered here because this is not a mechanism for requesting support. Since this issue appears to critical or time sensitive for you, please raise a ticket through your regular Red Hat support channels to make certain it receives the proper attention and prioritization for a timely resolution. For information on how to contact the Red Hat production support team, please visit: https://www.redhat.com/support/process/production/#howto We need to backport the following commit from upstream. I've confirmed it fixes the problem.
commit 056f49ff2cf645dc484956b00b65a3aa18a1a9a3
Author: Stratos Psomadakis <psomas>
Date: Mon Jan 27 12:30:15 2014 +0200
monitor: Cleanup mon->outbuf on write error
In case monitor_flush() fails to write the contents of mon->outbuf to
the output device, mon->outbuf is not cleaned up properly. Check the
return code of the qemu_chr_fe_write() function and cleanup the outbuf
if it fails.
References: http://lists.nongnu.org/archive/html/qemu-devel/2014-01/msg02890.html
Reproduced on qemu-kvm-rhev-0.12.1.2-2.420.el6.x86_64.
1. On host:
# /usr/libexec/qemu-kvm -qmp unix:/tmp/qmp,server,nowait
2. On host:
# python break_qmp.py qemu-kvm/(qemu-kvm source code path) /tmp/qmp
3. #top
Result:
After step 2:
# python break_qmp.py qemu-kvm/ /tmp/qmp
Connecting to QMP...
OK
Connecting using socket...
OK
Connecting to QMP...
(stuck here)
After step 3:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
7589 root 20 0 388m 19m 5160 R 100.2 0.3 0:23.42 qemu-kvm
1 root 20 0 19356 1544 1228 S 0.0 0.0 0:01.92 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.01 kthreadd
====================
On qemu-kvm-0.12.1.2-2.355.el6_4.5.x86_64, the issue does not exist.
After step 2, continuously showing up:
Connecting to QMP...
OK
Connecting using socket...
OK
Connecting to QMP...
OK
Connecting using socket...
OK
Connecting to QMP...
OK
Connecting using socket...
OK
......
After step 3:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
44000 root 20 0 376m 21m 4476 S 63.1 0.2 0:35.13 qemu-kvm
44050 root 20 0 148m 5484 2656 R 55.5 0.0 0:33.21 python
43756 root 20 0 98008 3964 2960 S 4.7 0.0 0:03.84 sshd
Uday, Please write GSSApproved in "Internal Whiteboard" Fix included in qemu-kvm-0.12.1.2-2.421.el6 *** Bug 1059612 has been marked as a duplicate of this bug. *** Hi Ronen, I have requested Tomas to add "GSSApproved" flag and he did this. So I am removing the needinfo flag from me. --Uday
Reproduced on qemu-kvm-0.12.1.2-2.415.el6.x86_64.
1. Start qemu-kvm on host.
[root@localhost ~]# /usr/libexec/qemu-kvm -qmp unix:/tmp/qmp,server,nowait
VNC server running on `::1:5900'
2. Run the script in the bug attachment.
[root@localhost home]# python break_qmp.py qemu-kvm/ /tmp/qmp
Connecting to QMP...
OK
Connecting using socket...
OK
Connecting to QMP...
^^^^^^^^ Stuck here
3. Host top info:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
29046 root 20 0 390m 23m 5180 R 100.2 0.3 0:57.87 /usr/libexec/qemu-kvm -qmp unix:/tmp/qmp,server,nowait
1 root 20 0 19364 1408 1144 S 0.0 0.0 0:00.89 /sbin/init
Verified pass on qemu-kvm-0.12.1.2-2.428.el6.x86_64:
# /usr/libexec/qemu-kvm -qmp unix:/tmp/qmp,server,nowait
VNC server running on `::1:5900'
# python break_qmp.py qemu-kvm/ /tmp/qmp
Connecting to QMP...
OK
Connecting using socket...
OK
Connecting to QMP...
OK
Connecting using socket...
OK
Connecting to QMP...
OK
Connecting using socket...
OK
......
(Continue printing the log)
Host top:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
29189 root 20 0 399m 24m 5572 R 69.5 0.3 1:34.39 qemu-kvm
29222 root 20 0 149m 5568 2728 S 44.5 0.1 0:01.75 python
Based on above, the bug is fixed.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2014-1490.html |