Bug 1283582
Summary: | dynflow_executor memory usage continues to grow, causing performance degredation | ||
---|---|---|---|
Product: | Red Hat Satellite | Reporter: | Stuart Auchterlonie <sauchter> |
Component: | Subscription Management | Assignee: | Ivan Necas <inecas> |
Status: | CLOSED ERRATA | QA Contact: | Chris Duryee <cduryee> |
Severity: | high | Docs Contact: | |
Priority: | unspecified | ||
Version: | 6.1.4 | CC: | bbuckingham, cduryee, cwelton, ehelms, erinn.looneytriggs, inecas, oshtaier, wpinheir |
Target Milestone: | Unspecified | Keywords: | Triaged |
Target Release: | Unused | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
URL: | http://projects.theforeman.org/issues/12650 | ||
Whiteboard: | |||
Fixed In Version: | rubygem-katello-3.0.0.38-1 | Doc Type: | Bug Fix |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2016-07-27 11:04:22 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | |||
Bug Blocks: | 1317008 |
Description
Stuart Auchterlonie
2015-11-19 11:00:47 UTC
Created redmine issue http://projects.theforeman.org/issues/12650 from this bug Upstream bug component is Tasks Plugin Upstream bug assigned to inecas Doesn't look like the memory leak has been plugged. These results are from 6.2 Beta (public) # grep ^VmData /var/log/dynflow_executor-memory-usage.log | uniq VmData: 2371532 kB VmData: 1082756 kB VmData: 901564 kB VmData: 1297548 kB VmData: 1494156 kB VmData: 1559692 kB VmData: 1826900 kB VmData: 1953612 kB Only downward movement relates to restart(s) of satellite. It's an improvement, as this system has been up for 2 weeks, so the last restart of satellite was around that time. I've isolated the the problem on the listening on candlepin events. There is an ::Actions::Candlepin::ListenOnCandlepinEvents action. I've watched memory consimption while using different kind of actions and the only time the memory was raising was when I was doing: I=0; while subscription-manager register --username admin --password changeme --org 'Summit2016' --environment Library --force; do I=$((I+1)); echo ================== $I; done Then if commented out the line ::Actions::Candlepin::ListenOnCandlepinEvents.ensure_running(world) in /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/lib/katello/engine.rb and restarted foreman-tasks After than, the subscription-manager register calls where not causing the leaks. I suspect qpid client library to cause this, I've found https://issues.apache.org/jira/browse/QPID-5872 that seems relevant and unresolved. It seems this issue is relevang https://issues.apache.org/jira/browse/QPID-3321, after adding `@session.sync` it seems the problems went away Proposed fix https://github.com/Katello/katello/pull/6065 Upstream bug assigned to inecas Upstream bug assigned to inecas Moving to POST since upstream bug http://projects.theforeman.org/issues/12650 has been closed tested with: tfm-rubygem-katello-3.0.0.57-1.el7sat.noarch how I tested: I used the instructions in https://github.com/Katello/katello/pull/6065#issue-156747193 No mem increase was observed marking as VERIFIED. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2016:1501 |