Bug 1186765
| Summary: | libvirtd crashes after chardev hotplug crashes qemu | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Ján Tomko <jtomko> | |
| Component: | libvirt | Assignee: | Ján Tomko <jtomko> | |
| Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> | |
| Severity: | high | Docs Contact: | ||
| Priority: | high | |||
| Version: | 7.1 | CC: | dyuan, jdenemar, jherrman, jtomko, lhuang, lmiksik, mzhan, ovasik, rbalakri, sherold, virt-bugs, zhwang, zpeng | |
| Target Milestone: | rc | Keywords: | Upstream, ZStream | |
| Target Release: | --- | |||
| Hardware: | x86_64 | |||
| OS: | Linux | |||
| Whiteboard: | ||||
| Fixed In Version: | libvirt-1.2.13-1.el7 | Doc Type: | Bug Fix | |
| Doc Text: |
When attaching a character device, libvirt did not correctly check the status of the domain after the libvirtd daemon exited the monitor operation. As a consequence, freed data from the domain definition could be accessed even when the domain had terminated unexpectedly, which caused libvirtd to crash as well. Now, libvirtd verifies that the domain is online before accessing its definition, which prevents libvirtd from crashing in the described situation.
|
Story Points: | --- | |
| Clone Of: | 1161024 | |||
| : | 1195155 (view as bug list) | Environment: | ||
| Last Closed: | 2015-11-19 06:08:54 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | 1161024 | |||
| Bug Blocks: | 1195155 | |||
|
Description
Ján Tomko
2015-01-28 14:02:44 UTC
I met the unclear error message during testing, maybe it was caused by below bug's patch. The correct message should be like this: error: operation failed: domain is no longer running Bug 1196934 - libvirt sometimes output useless error when qemu failed to start/migrate/restore qemu: do not overwrite the error in qemuDomainObjExitMonitor Steps for reproduced and verified: I can reproduce it in old libvirt [root@localhost ~]# rpm -q libvirt libvirt-1.2.8-16.el7.x86_64 [root@localhost ~]# virsh attach-device r71 console.xml error: Failed to attach device from console.xml error: Cannot recv data: Connection reset by peer error: Failed to reconnect to the hypervisor ... 1496 qemuDomainObjEnterMonitor(driver, vm); (gdb) s qemuDomainObjEnterMonitor (driver=driver@entry=0x7eff2c1ac640, obj=obj@entry=0x7eff2c234930) at qemu/qemu_domain.c:1605 1605 ignore_value(qemuDomainObjEnterMonitorInternal(driver, obj, (gdb) c Continuing. Program received signal SIGABRT, Aborted. 0x00007eff48e325d7 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 56 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig); Check it on latest fixed version, no crash happened. [root@localhost ~]# rpm -q libvirt qemu-kvm-rhev libvirt-1.2.15-2.el7.x86_64 qemu-kvm-rhev-2.3.0-1.el7.x86_64 [root@localhost ~]# ps aux | grep libvirtd| grep -v grep root 4003 0.0 0.2 1116280 22240 ? Ssl 11:04 0:00 /usr/sbin/libvirtd [root@localhost network-scripts]# gdb -p `pidof libvirtd` (gdb) info b Num Type Disp Enb Address What 1 breakpoint keep y 0x00007fb0cc1535e0 in qemuDomainAttachChrDevice at qemu/qemu_hotplug.c:1538 [root@localhost ~]# virsh start r71 Domain r71 started [root@localhost ~]# virsh attach-device r71 console.xml error: Failed to attach device from console.xml error: internal error: End of file from monitor <==== here is a new issue. During executing attach-device action, libvirtd will hit below breakpoint. Breakpoint 1, qemuDomainAttachChrDevice (driver=driver@entry=0x7fb0c412db20, vm=vm@entry=0x7fb0c421e2b0, chr=0x7fb0c4245290) at qemu/qemu_hotplug.c:1538 1538 { (gdb) n 1540 qemuDomainObjPrivatePtr priv = vm->privateData; (gdb) 1541 virDomainDefPtr vmdef = vm->def; (gdb) 1538 { (gdb) 1547 if (!virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_DEVICE)) { (gdb) 1542 char *devstr = NULL; (gdb) 1538 { (gdb) 1547 if (!virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_DEVICE)) { (gdb) 1543 char *charAlias = NULL; (gdb) 1547 if (!virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_DEVICE)) { (gdb) 1553 if (qemuAssignDeviceChrAlias(vmdef, chr, -1) < 0) (gdb) 1556 if (chr->deviceType == VIR_DOMAIN_CHR_DEVICE_TYPE_CONSOLE && (gdb) 1560 if (virDomainVirtioSerialAddrAutoAssign(NULL, (gdb) 1567 if (qemuBuildChrDeviceStr(&devstr, vm->def, chr, priv->qemuCaps) < 0) (gdb) 1570 if (virAsprintf(&charAlias, "char%s", chr->info.alias) < 0) (gdb) 1573 if (qemuDomainChrPreInsert(vmdef, chr) < 0) (gdb) 1576 qemuDomainObjEnterMonitor(driver, vm); (gdb) s qemuDomainObjEnterMonitor (driver=driver@entry=0x7fb0c412db20, obj=obj@entry=0x7fb0c421e2b0) at qemu/qemu_domain.c:1637 1637 ignore_value(qemuDomainObjEnterMonitorInternal(driver, obj, (gdb) n qemuDomainObjEnterMonitorInternal (driver=driver@entry=0x7fb0c412db20, obj=obj@entry=0x7fb0c421e2b0, asyncJob=asyncJob@entry=QEMU_ASYNC_JOB_NONE) at qemu/qemu_domain.c:1579 1579 { (gdb) c Continuing. [root@localhost ~]# ps aux | grep libvirtd| grep -v grep root 4003 0.0 0.2 1116280 22520 ? Ssl 11:04 0:00 /usr/sbin/libvirtd "End of file from monitor" is more specific than "domain is no longer running", because it says why libvirt no longer thinks the domain is running. Even though the less specific error message is more human-readable, using it instead of the more specific one would make debugging harder. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2015-2202.html |