Bug 1463506 - High CPU load and slow response times on the smart proxy (smart_proxy_dynflow_core process) with rex at scale
High CPU load and slow response times on the smart proxy (smart_proxy_dynflow...
Status: CLOSED ERRATA
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Remote Execution (Show other bugs)
6.3.0
Unspecified Unspecified
high Severity high (vote)
: GA
: --
Assigned To: Ivan Necas
Jan Hutař
: Triaged
Depends On:
Blocks: 1487964
  Show dependency treegraph
 
Reported: 2017-06-21 03:17 EDT by Ivan Necas
Modified: 2018-02-21 12:06 EST (History)
8 users (show)

See Also:
Fixed In Version: tfm-rubygems-foreman-tasks-core-0.1.4
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1487964 (view as bug list)
Environment:
Last Closed: 2018-02-21 12:06:18 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Patch for more effective refreshing of remote execution commands (4.78 KB, patch)
2017-06-21 03:17 EDT, Ivan Necas
no flags Details | Diff


External Trackers
Tracker ID Priority Status Summary Last Updated
Foreman Issue Tracker 20182 None None None 2017-07-03 03:25 EDT

  None (edit)
Description Ivan Necas 2017-06-21 03:17:53 EDT
Created attachment 1289960 [details]
Patch for  more effective refreshing of remote execution commands

Description of problem:
Due to inefficiency of the refreshing of execution, smart_proxy_dynflow_core can
get under heavy CPU load, which can eventually lead to slow response times
and bloated queues.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. run remote execution job against 1000 hosts


Actual results:
smart_proxy_dynflow_core getting to 100% CPU soon, response times
on smart_proxy getting to tens of seconds

Expected results:
Moderate CPU load, fast reponse times on smart-proxy (below second)

Additional info:
Extracted from https://bugzilla.redhat.com/show_bug.cgi?id=1416542#c36 to have more clarity on different parts of the bug.
Comment 4 Adam Ruzicka 2017-07-03 03:25:37 EDT
Created redmine issue http://projects.theforeman.org/issues/20182 from this bug
Comment 5 Brad Buckingham 2017-07-04 18:24:26 EDT
Moving to ASSIGNED as there is a PR submitted upstream.
Comment 6 pm-sat@redhat.com 2017-07-10 12:12:32 EDT
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/20182 has been resolved.
Comment 13 Bryan Kearney 2018-02-21 12:06:18 EST
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA.

For information on the advisory, and where to find the updated files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:0336

Note You need to log in before you can comment on or make changes to this bug.