Bug 1123463

Summary:	Expected puppet errors in kickstart environment cause deployment to fail
Product:	Red Hat OpenStack	Reporter:	Lars Kellogg-Stedman <lars>
Component:	rhel-osp-installer	Assignee:	Petr Chalupa <pchalupa>
Status:	CLOSED ERRATA	QA Contact:	Toni Freger <tfreger>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	unspecified	CC:	aberezin, ajeain, breeler, mburns, mhulan, morazi, pchalupa, rhos-maint, sclewis, tsanders, yeylon
Target Milestone:	ga	Keywords:	Rebase
Target Release:	Installer
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	ruby193-rubygem-staypuft-0.1.22.el6ost	Doc Type:	Bug Fix
Doc Text:	Previously, an initial puppet run during provisioning could result in puppet errors. The initial puppet run is only done to exchange certificates and errors in other puppet actions are not fatal. As a result, this would lead to erroneous reports of failed deployments. This has been fixed by ignoring errors in the first puppet run. Now, deployments no longer fail erroneously.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2014-08-21 18:06:36 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Lars Kellogg-Stedman 2014-07-25 18:18:59 UTC

When installing the 'Neutron Networker' role on a system with multiple interfaces, it is likely that only the provisioning interface has an address assigned when the kickstart is running.

This means that Puppet manifests that rely on facts like "ipaddress_eth1" to determine the ip address of an interface will fail, leading to errors such as:

  Could not retrieve catalog from remote server: Error 400 on SERVER: Local ip for ovs agent must be set when tunneling is enabled at /etc/puppet/environments/production/modules/neutron/manifests/agents/ovs.pp:32 on node mac52540036b16c.localdomain


This error *only* manifests during kickstart, because once the system boots for real all the interfaces are brought up and the appropriate facts are available.

Unfortunately, the error that crops during kickstart causes the deployment to fail.  So there are two problems here:

- Should we bring up interfaces before running puppet?  Or should we simply not run puppet in the kickstart?

- Should errors cause the deployment to *stop*, or should the deployment remain active and waiting for a successful puppet run?  In which case, the puppet run after the system boots should correct the error and allow the deploy to continue.

Comment 1 Lars Kellogg-Stedman 2014-07-25 18:24:01 UTC

s/likely/certain/

I have confirmed that during the kickstart install, only the provisioning interface has an address.

Comment 2 Lars Kellogg-Stedman 2014-07-29 00:44:01 UTC

pchalupa proposed the following patch:

diff --git a/app/lib/actions/staypuft/host/wait_until_host_ready.rb b/app/lib/actions/staypuft/host/wait_until_host_ready.rb
index 5fb3b86..8d755fb 100644
--- a/app/lib/actions/staypuft/host/wait_until_host_ready.rb
+++ b/app/lib/actions/staypuft/host/wait_until_host_ready.rb
 -59,7 +59,7 @@ module Actions
 
         def host_ready?(host_id)
           host = ::Host.find(host_id)
-          host.reports.order('reported_at DESC').any? do |report|
+          host.reports.order('reported_at DESC')[0..-2].any? do |report|
             check_for_failures(report, host.id)
             report_change?(report)
           end

This causes the workflow engine to ignore the first report from Puppet, so errors encountered during Kickstart are ignored.  I have tried this in my test environments and it appears to work as advertised.

Comment 3 Petr Chalupa 2014-07-29 05:54:03 UTC

fixed in https://github.com/theforeman/staypuft/pull/242

Comment 11 Toni Freger 2014-08-07 09:19:27 UTC

Works properly in ruby193-rubygem-staypuft-0.1.22.el6ost

Comment 12 errata-xmlrpc 2014-08-21 18:06:36 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-1090.html