Bug 1123463 - Expected puppet errors in kickstart environment cause deployment to fail
Summary: Expected puppet errors in kickstart environment cause deployment to fail
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: rhel-osp-installer
Version: unspecified
Hardware: Unspecified
OS: Unspecified
Target Milestone: ga
: Installer
Assignee: Petr Chalupa
QA Contact: Toni Freger
Depends On:
TreeView+ depends on / blocked
Reported: 2014-07-25 18:18 UTC by Lars Kellogg-Stedman
Modified: 2015-05-05 01:31 UTC (History)
11 users (show)

Fixed In Version: ruby193-rubygem-staypuft-0.1.22.el6ost
Doc Type: Bug Fix
Doc Text:
Previously, an initial puppet run during provisioning could result in puppet errors. The initial puppet run is only done to exchange certificates and errors in other puppet actions are not fatal. As a result, this would lead to erroneous reports of failed deployments. This has been fixed by ignoring errors in the first puppet run. Now, deployments no longer fail erroneously.
Clone Of:
Last Closed: 2014-08-21 18:06:36 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2014:1090 normal SHIPPED_LIVE Red Hat Enterprise Linux OpenStack Platform Enhancement Advisory 2014-08-22 15:28:08 UTC

Description Lars Kellogg-Stedman 2014-07-25 18:18:59 UTC
When installing the 'Neutron Networker' role on a system with multiple interfaces, it is likely that only the provisioning interface has an address assigned when the kickstart is running.

This means that Puppet manifests that rely on facts like "ipaddress_eth1" to determine the ip address of an interface will fail, leading to errors such as:

  Could not retrieve catalog from remote server: Error 400 on SERVER: Local ip for ovs agent must be set when tunneling is enabled at /etc/puppet/environments/production/modules/neutron/manifests/agents/ovs.pp:32 on node mac52540036b16c.localdomain

This error *only* manifests during kickstart, because once the system boots for real all the interfaces are brought up and the appropriate facts are available.

Unfortunately, the error that crops during kickstart causes the deployment to fail.  So there are two problems here:

- Should we bring up interfaces before running puppet?  Or should we simply not run puppet in the kickstart?

- Should errors cause the deployment to *stop*, or should the deployment remain active and waiting for a successful puppet run?  In which case, the puppet run after the system boots should correct the error and allow the deploy to continue.

Comment 1 Lars Kellogg-Stedman 2014-07-25 18:24:01 UTC

I have confirmed that during the kickstart install, only the provisioning interface has an address.

Comment 2 Lars Kellogg-Stedman 2014-07-29 00:44:01 UTC
pchalupa proposed the following patch:

diff --git a/app/lib/actions/staypuft/host/wait_until_host_ready.rb b/app/lib/actions/staypuft/host/wait_until_host_ready.rb
index 5fb3b86..8d755fb 100644
--- a/app/lib/actions/staypuft/host/wait_until_host_ready.rb
+++ b/app/lib/actions/staypuft/host/wait_until_host_ready.rb
 -59,7 +59,7 @@ module Actions
         def host_ready?(host_id)
           host = ::Host.find(host_id)
-          host.reports.order('reported_at DESC').any? do |report|
+          host.reports.order('reported_at DESC')[0..-2].any? do |report|
             check_for_failures(report, host.id)

This causes the workflow engine to ignore the first report from Puppet, so errors encountered during Kickstart are ignored.  I have tried this in my test environments and it appears to work as advertised.

Comment 3 Petr Chalupa 2014-07-29 05:54:03 UTC
fixed in https://github.com/theforeman/staypuft/pull/242

Comment 11 Toni Freger 2014-08-07 09:19:27 UTC
Works properly in ruby193-rubygem-staypuft-0.1.22.el6ost

Comment 12 errata-xmlrpc 2014-08-21 18:06:36 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.