Bug 1188704 - Overloaded machine hits the timeout in Puppet+Foreman's node.rb script therefore puppet won't start running.
Summary: Overloaded machine hits the timeout in Puppet+Foreman's node.rb script theref...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: rhel-osp-installer
Version: 6.0 (Juno)
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: z1
: Installer
Assignee: Jiri Stransky
QA Contact: Leonid Natapov
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-02-03 14:51 UTC by Leonid Natapov
Modified: 2015-03-05 18:19 UTC (History)
7 users (show)

Fixed In Version: foreman-installer-1.6.0-0.3.RC1.el7ost
Doc Type: Bug Fix
Doc Text:
Some machines would hit the timeout limit when attempting to run Puppet. As a result, Puppet would not trigger. This fix increases the timeout and Puppet triggers successfully.
Clone Of:
Environment:
Last Closed: 2015-03-05 18:19:53 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:0641 0 normal SHIPPED_LIVE Red Hat Enterprise Linux OpenStack Platform Installer Bug Fix Advisory 2015-03-05 23:15:51 UTC

Description Leonid Natapov 2015-02-03 14:51:50 UTC
Overloaded machine hits the timeout in Puppet+Foreman's node.rb script therefore puppet won't start running.

Execution of '/etc/puppet/node.rb returnes 1 because of time out.
Therefore puppet won't start run automatically on the controller which will lead HA deployment to fail.

There's a default timeout of 10 seconds, which was quite probably hit on overloaded machine. Jiri was able to reproduce this bug by manually setting timeout to 1 second locally.


Workaround:

Start puppet manually.

Comment 3 Jiri Stransky 2015-02-03 14:59:46 UTC
A workaround which can prevent this from happening is to edit /etc/puppet/foreman.yaml and change the timeout from 10 to a higher number, e.g. 45 (the unit is seconds).

Fix will need to happen upstream in puppet-foreman to make the timeout value configurable from the installer, and then in rhel-osp-installer we'll set a higher value.

Comment 5 Jiri Stransky 2015-02-11 09:59:25 UTC
Merged upstream. Foreman has adopted an increased default timeout of 60 seconds, so the first patch is not needed (we'll go with the new defaults), only the second and third patches are necessary.

Comment 7 Mike Burns 2015-02-12 21:47:23 UTC
The actual issue is intermittent, but we can validate that the fix is in by checking /etc/puppet/foreman.yaml and seeing timeout set to 60.

Comment 9 Leonid Natapov 2015-02-18 11:01:57 UTC
tested with foreman-installer-1.6.0-0.3.RC1.el7ost.noarch.
doesn't reproduce.

Comment 11 errata-xmlrpc 2015-03-05 18:19:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0641.html


Note You need to log in before you can comment on or make changes to this bug.