Bug 1526473 - Large MiqServer process leads to large generic workers that get killed
Summary: Large MiqServer process leads to large generic workers that get killed
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Performance
Version: 5.8.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: GA
: 5.8.3
Assignee: Keenan Brock
QA Contact: Tasos Papaioannou
URL:
Whiteboard: c&u:worker:perf
Depends On: 1479356
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-12-15 15:12 UTC by Satoe Imaishi
Modified: 2022-07-09 09:12 UTC (History)
22 users (show)

Fixed In Version: 5.8.3.1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1479356
Environment:
Last Closed: 2018-02-28 13:11:30 UTC
Category: ---
Cloudforms Team: CFME Core
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2018:0374 0 normal SHIPPED_LIVE Important: Red Hat CloudForms security, bug fix, and enhancement update 2018-02-28 18:04:37 UTC

Comment 2 Joe Rafaniello 2017-12-15 15:54:31 UTC
Note, the changes to use unique set size (USS) instead of proportional set size (PSS) don't fix the server leaking.  We now will not penalize workers for inheriting a large amount of memory from a large miq server process when they're forked.  top will still show these workers as high memory usage (RSS) if they inherited memory from a large server.  You need to use tools such as smem, smem -P MIQ, to see the USS.  bin/rake evm:status will show the USS value now.

We will continue to track down and fix the server memory growth but at least now, we won't be prematurely killing workers.

Comment 3 CFME Bot 2018-01-03 16:34:50 UTC
New commit detected on ManageIQ/manageiq-gems-pending/fine:
https://github.com/ManageIQ/manageiq-gems-pending/commit/d51285ca19c96304c7e3b521ae16713d75cfcee1

commit d51285ca19c96304c7e3b521ae16713d75cfcee1
Author:     Joe Rafaniello <jrafanie>
AuthorDate: Mon Nov 13 16:26:26 2017 -0500
Commit:     Joe Rafaniello <jrafanie>
CommitDate: Fri Dec 15 11:20:13 2017 -0500

    Store unique set size (USS) in the PSS column
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1479356
    Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1526473
    
    Unique set size is a better way to detect workers that are growing
    unbounded since any memory/reference leaks would be shown in their
    uss.  If the server process is large when forking, new workers would
    inherit a big pss immediately.
    
    We should really rename the column/hash key to uss.

 lib/gems/pending/util/miq-process.rb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comment 4 Tasos Papaioannou 2018-01-24 17:58:47 UTC
Verified on 5.8.3.1.

Comment 7 errata-xmlrpc 2018-02-28 13:11:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:0374


Note You need to log in before you can comment on or make changes to this bug.