Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1103224 - Timeout for passenger needs to be raised to avoid timeouts
Summary: Timeout for passenger needs to be raised to avoid timeouts
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Installation
Version: 6.0.4
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: Unspecified
Assignee: Lukas Zapletal
QA Contact: Sachin Ghai
URL: http://projects.theforeman.org/issues...
Whiteboard:
: 1103229 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-05-30 13:28 UTC by Bryan Kearney
Modified: 2023-09-07 18:38 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-07-02 14:08:32 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 6036 0 Normal Closed Timeout for passenger needs to be raised to avoid timeouts 2021-01-21 15:07:57 UTC

Description Bryan Kearney 2014-05-30 13:28:38 UTC 
The timeout for passeneger needs to be updated. Please update 

vi /usr/share/katello-installer/modules/apache/templates/mod/passenger.conf.erb

And add this line

 PassengerStartTimeout 360

Right above the </IfModule>
Comment 1 Justin Sherrill 2014-05-30 13:40:26 UTC 
[ 2014-05-30 08:34:57.9564 27564/7f1946200700 agents/HelperAgent/RequestHandler.h:1967 ]: [Client 25] Cannot checkout session. An error occurred while starting up the preloader: it did not write a startup response in time.
Error seen during timeout in /var/log/httpd/error_log:

[ 2014-05-30 08:34:57.9568 27564/7f1946200700 agents/HelperAgent
/RequestHandler.h:1967 ]: [Client 26] Cannot checkout session. An error occurred while starting up the preloader: it did not write a startup response in time.
[ 2014-05-30 09:01:57.8388 27564/7f19457be700 Pool2/Spawner.h:738 ]: [App 24003 stdout]
[ 2014-05-30 09:03:27.9270 27564/7f19457be700 Pool2/Implementation.cpp:849 ]: Could not spawn process for group /usr/share/foreman#default: An error occurred while starting up the preloader: it did not write a startup response in time.
     in 'void Passenger::ApplicationPool2::SmartSpawner::throwPreloaderSpawnException(const std::string&, Passenger::SpawnException::ErrorKind, Passenger::ApplicationPool2::Spawner::BackgroundIOCapturerPtr&, const Passenger::ApplicationPool2::Spawner::DebugDirPtr&)' (SmartSpawner.h:150)
     in 'std::string Passenger::ApplicationPool2::SmartSpawner::negotiatePreloaderStartup(Passenger::ApplicationPool2::SmartSpawner::StartupDetails&)' (SmartSpawner.h:558)
     in 'void Passenger::ApplicationPool2::SmartSpawner::startPreloader()' (SmartSpawner.h:206)
     in 'virtual Passenger::ApplicationPool2::ProcessPtr Passenger::ApplicationPool2::SmartSpawner::spawn(const Passenger::ApplicationPool2::Options&)' (SmartSpawner.h:744)
     in 'void Passenger::ApplicationPool2::Group::spawnThreadRealMain(const Passenger::ApplicationPool2::SpawnerPtr&, const Passenger::ApplicationPool2::Options&, unsigned int)' (Implementation.cpp:782)
Comment 2 Justin Sherrill 2014-05-30 13:41:13 UTC 
*** Bug 1103229 has been marked as a duplicate of this bug. ***
Comment 3 Lukas Zapletal 2014-05-30 14:53:31 UTC 
Justin, can you append production.log of the foreman instance? What is going on there? Also can you watch load/top during the initialization? What does it do? I am afraid 90 seconds should be enough even for Foreman/Katello...
Comment 4 Lukas Zapletal 2014-05-30 15:17:18 UTC 
It looks like this machine is really booting rails 2 minutes:

[root@quartet ~]# date <<< REQUEST STARTED
Fri May 30 10:59:01 EDT 2014

[root@quartet ~]# date <<< REQUEST HANDLED
Fri May 30 11:00:53 EDT 2014

Intel(R) Xeon(R) CPU            5120  @ 1.86GHz

The box was swapping a bit, not much:

[root@quartet ~]# free -m
             total       used       free     shared    buffers     cached
Mem:          9890       9660        229          0        418       3134
-/+ buffers/cache:       6107       3782
Swap:         6043        609       5434

When we killed one qemu instance from there, it helped a bit (approx 10%) but its still above 1.5 minute passenger default timeout.

Resolution: We need to make sure that we have set to 10 minutes:

PassengerStartTimeout 600
Comment 5 Lukas Zapletal 2014-05-30 15:18:10 UTC 
Also there is this upstream patch that preloads passenger processes ASAP when Apache2 httpd is restarted, so users/customers will not need to wait until the first request is processed:

https://github.com/theforeman/puppet-foreman/pull/194
Comment 6 Bryan Kearney 2014-06-03 14:05:43 UTC 
Created redmine issue http://projects.theforeman.org/issues/6034 from this bug
Comment 7 Bryan Kearney 2014-06-03 14:18:42 UTC 
lzap has another bug, taking it off.
Comment 8 Bryan Kearney 2014-06-03 14:24:36 UTC 
Connecting redmine issue http://projects.theforeman.org/issues/6036 from this bug
Comment 9 Lukas Zapletal 2014-06-11 15:04:06 UTC 
Upstream merged, this can be tricky to cherry-pick. Ping me if you need assistance.
Comment 10 Bryan Kearney 2014-06-11 15:05:08 UTC 
Moving to POST since upstream bug http://projects.theforeman.org/issues/6036 has been closed
Comment 11 Ivan Necas 2014-06-11 15:26:48 UTC 
The PR against katello-installer https://github.com/Katello/katello-installer/pull/72
Comment 12 Lukas Zapletal 2014-06-11 16:34:39 UTC 
Jason, the katello-installer has been merged.
Comment 14 Sachin Ghai 2014-06-12 08:17:15 UTC 
Installed sat6 beta snap9 and I didn't face the issue related to setting value "PassengerStartTimeout to 360. 

However, as per this bz comments, where I can see PassengerTimeout value set to 600?
Comment 15 Lukas Zapletal 2014-06-12 08:32:39 UTC 
/etc/httpd/conf.d/05-foreman.conf
Comment 16 Sachin Ghai 2014-06-12 08:41:20 UTC 
Thanks Lzap.

[root@dhcp201-103 yum.repos.d]# cat /etc/httpd/conf.d/05-foreman.conf | grep PassengerStartTimeout
PassengerStartTimeout 600
[root@dhcp201-103 yum.repos.d]# 

So I tried installation on two diff nodes, one with 8 cores and 12G ram and other on a VM with 2 cores and 4G ram.

And didn't see any issue. So moving this to verified.
Comment 17 Bryan Kearney 2014-07-02 14:08:32 UTC 
This was delivered with 6.0.3, which is the Satellite 6 Beta.
Comment 18 Bryan Kearney 2014-07-02 14:09:52 UTC 
This was delivered in 6.0.3, the Beta version of Satellite 6.0
Note You need to log in before you can comment on or make changes to this bug.