Bug 951576

Summary: prepareForShutdown is not called when connection to libvirt is broken with event: libvirtError: internal error client socket is closed
Product: Red Hat Enterprise Virtualization Manager Reporter: Idith Tal-Kohen <italkohe>
Component: vdsmAssignee: Yaniv Bronhaim <ybronhei>
Status: CLOSED ERRATA QA Contact: Elad <ebenahar>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.2.0CC: acathrow, bazulay, bili, chetan, cpelland, danken, dyasny, dyuan, hateya, iheim, lpeer, lyarwood, michal.skrivanek, mzhan, ofrenkel, pstehlik, pzhukov, rwu, sgrinber, whuang, yaliu, ykaul
Target Milestone: ---Keywords: ZStream
Target Release: 3.1.4   
Hardware: x86_64   
OS: Linux   
Whiteboard: virt
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Domain codes and libvirt error codes were mixed by mistake, so restarting the libvirt daemon caused the libvirt client socket to close on Red Hat Enterprise Virtualization Manager. In addition, libvirt reported internal errors if libvirtd is restarted or stopped, for example after a crash. This update resolves the mixed codes and adds missing error codes. Restarting libvirtd now correctly restarts VDSM connections.
Story Points: ---
Clone Of: 852956 Environment:
Last Closed: 2013-05-01 18:25:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 852956    
Bug Blocks: 915537, 928309    
Attachments:
Description Flags
logs none

Comment 2 Vinzenz Feenstra [evilissimo] 2013-04-17 17:05:33 UTC
The fix did not work we have to make a new one.

Comment 3 Barak 2013-04-18 08:07:38 UTC
*** Bug 951571 has been marked as a duplicate of this bug. ***

Comment 4 Vinzenz Feenstra [evilissimo] 2013-04-18 09:25:29 UTC
Merged u/s to master branch as: http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=commit;h=1817e6fe2a1ec873755d871296c2fb61fdf5d888

Comment 7 Elad 2013-04-22 13:59:36 UTC
Checked on RHEVM-3.1 - SI28:

rhevm-3.1.0-52.el6ev.noarch
vdsm-4.10.2-1.10.el6ev.x86_64
libvirt-0.10.2-18.el6_4.4.x86_64
sanlock-2.6-2.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.355.el6_4.2.x86_64


prepareForShutdown is called when connection to libvirt is broken after restart to libvirtd.


Thread-354::ERROR::2013-04-22 14:46:28,767::libvirtconnection::102::vds::(wrapper) connection to libvirt broken. taking vdsm down. ecode: 1 edom: 7
Thread-354::INFO::2013-04-22 14:46:29,691::vmChannels::187::vds::(stop) VM channels listener was stopped.
Thread-354::DEBUG::2013-04-22 14:46:29,692::task::568::TaskManager.Task::(_updateState) Task=`c1716d92-c0c0-40ec-9f6c-c999d80c8975`::moving from state init -> state preparing
Thread-354::INFO::2013-04-22 14:46:29,692::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None)

Comment 8 Michal Skrivanek 2013-04-23 08:50:13 UTC
*** Bug 955432 has been marked as a duplicate of this bug. ***

Comment 9 Elad 2013-04-30 11:44:02 UTC
I was manage to reproduce this bug. It is only reproduces with no running vms on the setup.

moving back to ASSIGN.

Comment 12 Elad 2013-04-30 12:47:23 UTC
Created attachment 741873 [details]
logs

vdsm.log and libvirtd.log attached.

Comment 13 Michal Skrivanek 2013-04-30 13:59:54 UTC
this particular one is happening because netinfo.py is not using libvirtconnection wrapper. In reality this should not be a big deal as if there is a more persistent issue/libvirt takes time to restart it will eventually trigger the shutdown elsewhere - most likely in statistics thread when there are existing VMs.

Comment 14 Michal Skrivanek 2013-04-30 14:11:04 UTC
Was there any impact? If not I'd close it anyway. I see the libvirt connection is directly used only in network-related functions. Dan, should we be using the wrapper instead or is this ok?

Comment 16 Elad 2013-04-30 21:58:33 UTC
Since the main scenario of this bug does not reproduce, I'm moving it back to VERIFIED.

I'll open a new BZ for the secondary scenario (killing libvird with SIGABRT without any running VMs)

Comment 17 Elad 2013-05-01 07:49:44 UTC
opened BZ# 958367 for the secondary scenario (killing libvird with SIGABRT without any running VMs)

Comment 19 errata-xmlrpc 2013-05-01 18:25:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-0774.html