Bug 1349829 - ovirt-ha-agent should reuse json-rpc connections
Summary: ovirt-ha-agent should reuse json-rpc connections
Keywords:
Status: CLOSED DUPLICATE of bug 1417708
Alias: None
Product: ovirt-hosted-engine-ha
Classification: oVirt
Component: General
Version: 2.0.0
Hardware: x86_64
OS: Linux
low
medium
Target Milestone: ovirt-4.2.0
: ---
Assignee: Simone Tiraboschi
QA Contact: Artyom
URL:
Whiteboard:
: 1383118 (view as bug list)
Depends On: 1343005 1376843
Blocks: 1383108 1417708 1417709
TreeView+ depends on / blocked
 
Reported: 2016-06-24 11:13 UTC by Michal Skrivanek
Modified: 2020-06-11 12:54 UTC (History)
25 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1343005
: 1417708 1417709 (view as bug list)
Environment:
Last Closed: 2017-01-30 18:04:46 UTC
oVirt Team: Integration
Embargoed:
rule-engine: ovirt-4.2?
michal.skrivanek: planning_ack?
michal.skrivanek: devel_ack?
michal.skrivanek: testing_ack?

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1381899 0 unspecified CLOSED The python jsonrpc client is parsing the API schema on each connect eating a lot of CPU cycles 2021-02-22 00:41:40 UTC
oVirt gerrit 57942 0 'None' MERGED jsonrpc: close client 2020-02-24 19:27:55 UTC
oVirt gerrit 59106 0 'None' MERGED jsonrpc: close client 2020-02-24 19:27:55 UTC

Internal Links: 1381899

Description Michal Skrivanek 2016-06-24 11:13:03 UTC 
+++ This bug was initially created as a clone of Bug #1343005 +++

The connection to vdsm is mimicking xml-rpc behavior of always creating a new client for each call. This should be changed to reuse the client instead of wasting resources
Comment 1 Red Hat Bugzilla Rules Engine 2016-07-03 08:05:42 UTC 
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.
Comment 2 Roy Golan 2016-07-13 09:01:26 UTC 
The original problem of vdsm not closing FDs was solved by VDSM. There is no real impact here.
Comment 3 Yaniv Kaul 2016-09-07 10:35:02 UTC 
Raising severity a bit. Log is a bit flooded right now with the messages about the connections.
Comment 4 Yaniv Kaul 2016-09-07 10:36:00 UTC 
Raising severity a bit. Log is a bit flooded right now with the messages about the connections and the daemon is taking quite a bit of CPU:

3379 vdsm      20   0  551460  74804   6508 R  25.2  1.2   2:47.37 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-agent --no-daemon                                                                   
 4045 qemu      20   0 4517760 2.107g  10588 S   5.6 36.2   1:54.68 /usr/libexec/qemu-kvm -name HostedEngine -S -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off -cpu SandyBridge -m 3800 -realtime
Comment 5 Martin Sivák 2016-10-17 13:28:06 UTC 
VDSM is not leaking anymore and this has negligible impact, lowering the priority
Comment 6 Piotr Kliczewski 2016-10-18 09:29:59 UTC 
*** Bug 1383108 has been marked as a duplicate of this bug. ***
Comment 7 Martin Perina 2016-11-03 11:00:05 UTC 
*** Bug 1383118 has been marked as a duplicate of this bug. ***
Comment 8 Derek Atkins 2016-11-11 14:54:11 UTC 
According to Piotr, I'm hitting this bug when I see this in my logs:

# grep ERROR /var/log/vdsm/vdsm.log | tail
JsonRpc (StompReactor)::ERROR::2016-11-11 09:52:40,513::betterAsyncore::113::vds.dispatcher::(recv) SSL error during reading data: unexpected eof
JsonRpc (StompReactor)::ERROR::2016-11-11 09:52:44,125::betterAsyncore::113::vds.dispatcher::(recv) SSL error during reading data: unexpected eof
JsonRpc (StompReactor)::ERROR::2016-11-11 09:52:47,805::betterAsyncore::113::vds.dispatcher::(recv) SSL error during reading data: unexpected eof
JsonRpc (StompReactor)::ERROR::2016-11-11 09:52:51,385::betterAsyncore::113::vds.dispatcher::(recv) SSL error during reading data: unexpected eof
JsonRpc (StompReactor)::ERROR::2016-11-11 09:53:04,945::betterAsyncore::113::vds.dispatcher::(recv) SSL error during reading data: unexpected eof
JsonRpc (StompReactor)::ERROR::2016-11-11 09:53:11,932::betterAsyncore::113::vds.dispatcher::(recv) SSL error during reading data: unexpected eof
JsonRpc (StompReactor)::ERROR::2016-11-11 09:53:12,163::betterAsyncore::113::vds.dispatcher::(recv) SSL error during reading data: unexpected eof
JsonRpc (StompReactor)::ERROR::2016-11-11 09:53:16,252::betterAsyncore::113::vds.dispatcher::(recv) SSL error during reading data: unexpected eof
JsonRpc (StompReactor)::ERROR::2016-11-11 09:53:19,870::betterAsyncore::113::vds.dispatcher::(recv) SSL error during reading data: unexpected eof
JsonRpc (StompReactor)::ERROR::2016-11-11 09:53:23,460::betterAsyncore::113::vds.dispatcher::(recv) SSL error during reading data: unexpected eof
Comment 9 Yaniv Kaul 2016-12-11 09:57:10 UTC 
Both patches are merged. What else is needed here?
Comment 10 Simone Tiraboschi 2016-12-12 16:48:12 UTC 
That patches are jut about correctly closing the old connection to avoid memory leakages.

To properly reuse a single connection we miss all the keep alive and reconnect on failure logic; indeed it depends on 1376843 which got re-targeted to 4.2
Comment 12 Troy Dash 2017-01-30 18:04:46 UTC 

*** This bug has been marked as a duplicate of bug 1417708 ***
Note You need to log in before you can comment on or make changes to this bug.