| Summary: | [Regression] dynflow_executor memory usage continues to grow, causing performance degradation | ||
|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Rick Dixon <rdixon> |
| Component: | Subscription Management | Assignee: | Ivan Necas <inecas> |
| Status: | CLOSED INSUFFICIENT_DATA | QA Contact: | Katello QA List <katello-qa-list> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 6.2.2 | CC: | anerurka, aperotti, bbuckingham, erinn.looneytriggs, inecas, jcallaha, kdixon, mmccune, mmello, pdwyer, rdixon, xdmoon |
| Target Milestone: | Unspecified | Keywords: | Performance, PrioBumpGSS, Triaged |
| Target Release: | Unused | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2016-11-30 14:10:17 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
|
Comment 8
Ivan Necas
2016-11-15 10:20:36 UTC
Thanks, it helps a bit, but I would like to ask for more details if possible: 1. restart foreman-tasks 2. start collecting periodically the `ps aux` output (let's say every minute, if possible) 3. perform the reproducing steps (I assume it's reproducible easily) 4. once the memory grew significantly, run `foreman-rake foreman_tasks:export_tasks tasks=all days=1` to share what kind of tasks were there 5. I would be also interested into qpid stats: qpid-stat --ssl-certificate=/etc/pki/katello/qpid_client_striped.crt -b amqps://localhost:5671 -e qpid-stat --ssl-certificate=/etc/pki/katello/qpid_client_striped.crt -b amqps://localhost:5671 -g qpid-stat --ssl-certificate=/etc/pki/katello/qpid_client_striped.crt -b amqps://localhost:5671 -c The investigation so far lead to two issues that caused unnecessary large tasks to be created at every repo sync: https://bugzilla.redhat.com/show_bug.cgi?id=1391704 https://bugzilla.redhat.com/show_bug.cgi?id=1398438 Both related to a repo syncing, when library is synced to a capsule Here is another bug, that came from analysis of further data provided https://bugzilla.redhat.com/show_bug.cgi?id=1399294 I'm closing this issue, as the original case when opening the issue was closed and there were additional bugs filed identifying specific usages that lead to the memory growth. The reason for closing is avoiding misunderstandings when dealing with several independent issues in one BZ. For future bugs regarding similar memory issues, please see https://access.redhat.com/site/solutions/2785811 to see possible causes of the issues, as well as info how to collect data for faster analysis and provide additional information, when any of the existing bug mentioned there doesn't correspond to the system behavior and a new BZ needs to be filed. |