Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1452772 - [RFE] Passenger graceful killer cron job
Summary: [RFE] Passenger graceful killer cron job
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Infrastructure
Version: 6.2.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: 6.4.0
Assignee: Ivan Necas
QA Contact: Peter Ondrejka
URL:
Whiteboard:
: 1452665 1452771 1630958 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-05-19 15:15 UTC by Satellite Program
Modified: 2019-11-05 23:05 UTC (History)
9 users (show)

Fixed In Version: rubygem-foreman_maintain-0.2.9-2
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-10-16 15:27:32 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 19496 0 None None None 2017-05-19 15:15:35 UTC
Foreman Issue Tracker 24984 0 None None None 2018-09-19 14:46:37 UTC
Red Hat Bugzilla 1478364 0 unspecified CLOSED kill passenger threads that have not returned in 60 minutes 2021-02-22 00:41:40 UTC
Red Hat Product Errata RHSA-2018:2927 0 None None None 2018-10-16 15:28:07 UTC

Internal Links: 1478364

Description Satellite Program 2017-05-19 15:15:33 UTC 
Passenger open-source version does support process restarting on memory threshold. It looks like passenger have reserved SIGUSR1 for graceful restart. A cron job could do the job easily:

https://gist.github.com/kascote/366441

http://mensfeld.pl/2012/08/simple-rubyrails-passenger-memory-consumption-limit-monitoring/
Comment 1 Satellite Program 2017-05-19 15:15:38 UTC 
Created from redmine issue http://projects.theforeman.org/issues/19496
Comment 2 Satellite Program 2017-05-19 15:15:41 UTC 
Upstream bug assigned to lzap
Comment 3 Bryan Kearney 2017-05-19 15:16:47 UTC 
*** Bug 1452771 has been marked as a duplicate of this bug. ***
Comment 4 Bryan Kearney 2017-05-19 15:17:07 UTC 
*** Bug 1452665 has been marked as a duplicate of this bug. ***
Comment 5 Lukas Zapletal 2017-05-22 06:59:06 UTC 
Thanks, is this meant for 6.3 or 6.2? This is easily backportable, just a script and packaging change.
Comment 6 Satellite Program 2017-11-02 20:24:00 UTC 
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/19496 has been resolved.
Comment 7 Matt Pusateri 2018-09-18 21:00:04 UTC 
Dev, can we get some assistance on how to reproduce pushing passenger into the failed state.
Comment 8 Lukas Zapletal 2018-09-19 14:31:03 UTC 
Dumb approach, set the memory limit in this script to 400 MB, restart Satellite httpd and run

watch passenger-status

Click in the UI randomly, do some actions via CLI/API to increase memory consumption above 400 MB and then wait until the script gets executed from cron (or execute it). It should restart one process per execution.

The config is /etc/passenger-recycler.yaml and value you want to change is MAX_PRIV_RSS_MEMORY. Here is an example file:

https://github.com/swapab/foreman_maintain/blob/302581759bf5bc31de962f6cc5ccecb889c84748/config/passenger-recycler.yaml

It looks like for some reason this YAML file is not being distributed with foreman-maintain, likely a bug.
Comment 10 Lukas Zapletal 2018-09-19 15:19:08 UTC 
Correction of the YAML file link:

https://github.com/theforeman/foreman_maintain/blob/master/config/passenger-recycler.yaml

Filed BZ#1630958 for the missing YAML file.
Comment 11 Ivan Necas 2018-09-19 15:32:38 UTC 
*** Bug 1630958 has been marked as a duplicate of this bug. ***
Comment 12 Satellite Program 2018-09-19 16:05:52 UTC 
Upstream bug assigned to inecas
Comment 13 Satellite Program 2018-09-19 16:05:56 UTC 
Upstream bug assigned to inecas
Comment 15 Patrick Creech 2018-09-21 00:40:01 UTC 
adding sat-maintenance flag as the above pr deals with foreman-maintain
Comment 16 Peter Ondrejka 2018-09-24 13:04:44 UTC 
Verified on Satellite 6.4 snap 23, expected files are present: 

rpm -ql rubygem-foreman_maintain-0.2.10-1.el7sat.noarch | grep passenger
/etc/passenger-recycler.yaml
/usr/bin/passenger-recycler
/usr/share/gems/gems/foreman_maintain-0.2.10/bin/passenger-recycler
/usr/share/gems/gems/foreman_maintain-0.2.10/config/passenger-recycler.yaml
/usr/share/gems/gems/foreman_maintain-0.2.10/definitions/procedures/passenger_recycler.rb

With setup suggested in comment #8, running passenger-recycler:
...
Terminating 15319 (started 08:22:44) with private RSS size of 459 MB
Process successfully 15319 terminated
...

So the custom memory limit is respected. The cron task is not set automatically with rpm installation, however this is intended, as per discussion with inecas
Comment 18 errata-xmlrpc 2018-10-16 15:27:32 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2927
Note You need to log in before you can comment on or make changes to this bug.