Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1349829

Summary: ovirt-ha-agent should reuse json-rpc connections
Product: [oVirt] ovirt-hosted-engine-ha Reporter: Michal Skrivanek <michal.skrivanek>
Component: GeneralAssignee: Simone Tiraboschi <stirabos>
Status: CLOSED DUPLICATE QA Contact: Artyom <alukiano>
Severity: medium Docs Contact:
Priority: low    
Version: 2.0.0CC: alukiano, baptiste.agasse, bugs, c.handel, cshao, ddash, dfediuck, eedri, fdeutsch, frolland, gklein, gwatson, jbelka, mavital, mwest, oourfali, pkliczew, redhat, rgolan, sbonazzo, s.kieske, stirabos, warlord, ycui, ylavi
Target Milestone: ovirt-4.2.0Flags: rule-engine: ovirt-4.2?
michal.skrivanek: planning_ack?
michal.skrivanek: devel_ack?
michal.skrivanek: testing_ack?
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1343005
: 1417708 1417709 (view as bug list) Environment:
Last Closed: 2017-01-30 18:04:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Integration RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1343005, 1376843    
Bug Blocks: 1383108, 1417708, 1417709    

Description Michal Skrivanek 2016-06-24 11:13:03 UTC
+++ This bug was initially created as a clone of Bug #1343005 +++

The connection to vdsm is mimicking xml-rpc behavior of always creating a new client for each call. This should be changed to reuse the client instead of wasting resources

Comment 1 Red Hat Bugzilla Rules Engine 2016-07-03 08:05:42 UTC
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 2 Roy Golan 2016-07-13 09:01:26 UTC
The original problem of vdsm not closing FDs was solved by VDSM. There is no real impact here.

Comment 3 Yaniv Kaul 2016-09-07 10:35:02 UTC
Raising severity a bit. Log is a bit flooded right now with the messages about the connections.

Comment 4 Yaniv Kaul 2016-09-07 10:36:00 UTC
Raising severity a bit. Log is a bit flooded right now with the messages about the connections and the daemon is taking quite a bit of CPU:

3379 vdsm      20   0  551460  74804   6508 R  25.2  1.2   2:47.37 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-agent --no-daemon                                                                   
 4045 qemu      20   0 4517760 2.107g  10588 S   5.6 36.2   1:54.68 /usr/libexec/qemu-kvm -name HostedEngine -S -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off -cpu SandyBridge -m 3800 -realtime

Comment 5 Martin Sivák 2016-10-17 13:28:06 UTC
VDSM is not leaking anymore and this has negligible impact, lowering the priority

Comment 6 Piotr Kliczewski 2016-10-18 09:29:59 UTC
*** Bug 1383108 has been marked as a duplicate of this bug. ***

Comment 7 Martin Perina 2016-11-03 11:00:05 UTC
*** Bug 1383118 has been marked as a duplicate of this bug. ***

Comment 8 Derek Atkins 2016-11-11 14:54:11 UTC
According to Piotr, I'm hitting this bug when I see this in my logs:

# grep ERROR /var/log/vdsm/vdsm.log | tail
JsonRpc (StompReactor)::ERROR::2016-11-11 09:52:40,513::betterAsyncore::113::vds.dispatcher::(recv) SSL error during reading data: unexpected eof
JsonRpc (StompReactor)::ERROR::2016-11-11 09:52:44,125::betterAsyncore::113::vds.dispatcher::(recv) SSL error during reading data: unexpected eof
JsonRpc (StompReactor)::ERROR::2016-11-11 09:52:47,805::betterAsyncore::113::vds.dispatcher::(recv) SSL error during reading data: unexpected eof
JsonRpc (StompReactor)::ERROR::2016-11-11 09:52:51,385::betterAsyncore::113::vds.dispatcher::(recv) SSL error during reading data: unexpected eof
JsonRpc (StompReactor)::ERROR::2016-11-11 09:53:04,945::betterAsyncore::113::vds.dispatcher::(recv) SSL error during reading data: unexpected eof
JsonRpc (StompReactor)::ERROR::2016-11-11 09:53:11,932::betterAsyncore::113::vds.dispatcher::(recv) SSL error during reading data: unexpected eof
JsonRpc (StompReactor)::ERROR::2016-11-11 09:53:12,163::betterAsyncore::113::vds.dispatcher::(recv) SSL error during reading data: unexpected eof
JsonRpc (StompReactor)::ERROR::2016-11-11 09:53:16,252::betterAsyncore::113::vds.dispatcher::(recv) SSL error during reading data: unexpected eof
JsonRpc (StompReactor)::ERROR::2016-11-11 09:53:19,870::betterAsyncore::113::vds.dispatcher::(recv) SSL error during reading data: unexpected eof
JsonRpc (StompReactor)::ERROR::2016-11-11 09:53:23,460::betterAsyncore::113::vds.dispatcher::(recv) SSL error during reading data: unexpected eof

Comment 9 Yaniv Kaul 2016-12-11 09:57:10 UTC
Both patches are merged. What else is needed here?

Comment 10 Simone Tiraboschi 2016-12-12 16:48:12 UTC
That patches are jut about correctly closing the old connection to avoid memory leakages.

To properly reuse a single connection we miss all the keep alive and reconnect on failure logic; indeed it depends on 1376843 which got re-targeted to 4.2

Comment 12 Troy Dash 2017-01-30 18:04:46 UTC

*** This bug has been marked as a duplicate of bug 1417708 ***