Bug 852956
Summary: | 3.2 - prepareForShutdown is not called when connection to libvirt is broken with event: libvirtError: internal error client socket is closed | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Virtualization Manager | Reporter: | EricLee <bili> | ||||||||||
Component: | vdsm | Assignee: | Yaniv Bronhaim <ybronhei> | ||||||||||
Status: | CLOSED ERRATA | QA Contact: | movciari | ||||||||||
Severity: | medium | Docs Contact: | |||||||||||
Priority: | medium | ||||||||||||
Version: | 3.2.0 | CC: | acathrow, bazulay, chetan, cpelland, danken, dyasny, dyuan, ebenahar, hateya, iheim, italkohe, lpeer, michal.skrivanek, mzhan, pspacek, pzhukov, rwu, sgrinber, whuang, ykaul | ||||||||||
Target Milestone: | --- | Keywords: | ZStream | ||||||||||
Target Release: | 3.2.0 | ||||||||||||
Hardware: | x86_64 | ||||||||||||
OS: | Linux | ||||||||||||
Whiteboard: | virt | ||||||||||||
Fixed In Version: | vdsm-4.10.2-16.0.el6ev | Doc Type: | Bug Fix | ||||||||||
Doc Text: |
Domain codes and libvirt error codes were mixed by mistake, so restarting the libvirt daemon caused the libvirt client socket to close on Red Hat Enterprise Virtualization Manager. In addition, libvirt reported internal errors if libvirtd is restarted or stopped, for example after a crash. This update resolves the mixed codes and adds missing error codes. Restarting libvirtd now correctly restarts VDSM connections.
|
Story Points: | --- | ||||||||||
Clone Of: | |||||||||||||
: | 951576 (view as bug list) | Environment: | |||||||||||
Last Closed: | 2013-06-10 20:31:40 UTC | Type: | Bug | ||||||||||
Regression: | --- | Mount Type: | --- | ||||||||||
Documentation: | --- | CRM: | |||||||||||
Verified Versions: | Category: | --- | |||||||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||||||
Embargoed: | |||||||||||||
Bug Depends On: | |||||||||||||
Bug Blocks: | 915537, 922117, 928309, 951576 | ||||||||||||
Attachments: |
|
Created attachment 608048 [details]
/var/log/vdsm/vdsm.log
BTW, pure libvirt environment, restart libvirtd using service will not cause that error,and working well. It seems like vdsm is not properly catching closed connection to libvirtd. I'm moving this bug to vdsm for further investigation. seems like the code is obsolete, in current libvirt you can find a check for connection health: + (((last_error->code == VIR_ERR_SYSTEM_ERROR) && + (last_error->domain == VIR_FROM_REMOTE)) || + (last_error->code == VIR_ERR_RPC) || + (last_error->code == VIR_ERR_NO_CONNECT) || + (last_error->code == VIR_ERR_INVALID_CONN))) so adding those last two would be needed in libvirtconnection.py wrapMethod() Libvirt team says they have a callback for this in RHEL 6.4 so we may want to replace this in future/ upstream review: http://gerrit.ovirt.org/#/c/8283/ merged u/s master: 0bfe41291e12fe583641b1844c753d51e15fded7 Created attachment 736326 [details]
vdsm-lib-socket.log.tgz
full vdsm log
*** Bug 956197 has been marked as a duplicate of this bug. *** *** Bug 957335 has been marked as a duplicate of this bug. *** Created attachment 741901 [details] vdsm.log and libvirtd.log I was manage to reproduce this bug. It reproduced during verification of this https://bugzilla.redhat.com/show_bug.cgi?id=953645 we saw that prepareForShutdown is not called after vdsm looses connection to libvirt. I did "kill -6 (libvirtd_pid)" This is happening only when there are no running vms. otherwise, it does not reproduce. logs attached. moving back to ASSIGN. (In reply to comment #25) > > This is happening only when there are no running vms. otherwise, it does not > reproduce. This awkward behavior is not new. It's here since getCaps started polling libvirt for netinfo. It should be be fixed, but I'm not sure at all that the suggested http://gerrit.ovirt.org/14345/ should make it into rhev-3.2, as it is not very probable that libvirt would spontaneously die when there are no VMs running. Please open a new bug to track this behavior and let's solve it on rhev-3.3. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHSA-2013-0886.html |
Created attachment 608047 [details] /var/log/vdsm/libvirt.log Description of problem: "initctl restart libvirtd" will cause libvirt client socket close in rhevm Version-Release number of selected component (if applicable): # rpm -qa libvirt kernel qemu-kvm-rhev vdsm spice-server spice-server-0.10.1-10.el6.x86_64 libvirt-0.10.0-1.el6.x86_64 vdsm-4.9.6-31.0.el6_3.x86_64 qemu-kvm-rhev-0.12.1.2-2.307.el6.x86_64 kernel-2.6.32-298.el6.x86_64 How reproducible: 100% Steps to Reproduce: 1.Prepare a rhevm enviroment, with libvirt + vdsm packages. 2.#initctl restart libvirtd 3.Start a guest in rhevm,will get error: libvirtError: internal error client socket is closed and can not start guest normally. 4.#service vdsmd restart will take libvirtd to normal environment. 5.Start a guest in rhevm normally, then redo "initctl restart libvirtd", and check /var/log/vdsm/libvirt.log and /var/log/vdsm/vdsm.log also can get that error. Actual results: As steps. Expected results: Libvirtd runs normally. Additional info: