Bug 1125136

Summary: Neutron Networker failed at 60% due to failure in starting iptables
Product: Red Hat OpenStack Reporter: Udi Kalifon <ukalifon>
Component: rhel-osp-installerAssignee: Mike Burns <mburns>
Status: CLOSED ERRATA QA Contact: Omri Hochman <ohochman>
Severity: unspecified Docs Contact:
Priority: high    
Version: Foreman (RHEL 6)CC: aortega, breeler, lbezdick, lnatapov, mburns, mmagr, morazi, rhos-maint, sclewis, yeylon
Target Milestone: ga   
Target Release: Installer   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: rhel-osp-installer-0.1.8-1.el6ost Doc Type: Bug Fix
Doc Text:
An ordering issue in the puppet classes was not waiting for firewalld to completely shut down before starting iptables. As a result, iptables would be started too soon and the firewalld process would kill it. The ordering has been fixed and now puppet waits for firewalld to stop completely before starting iptables.
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-08-21 18:07:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Udi Kalifon 2014-07-31 07:01:05 UTC
Description of problem:
Deploying on 1 controller, 1 networker and 2 computes. Puddle from July 29th. Deployment is stuck on the networker and you can see the following in /var/log/messages:

Jul 30 16:12:12 maca25400654fdd yum[11141]: Installed: iptables-services-1.4.21-13.el7.x86_64
Jul 30 16:12:13 maca25400654fdd puppet-agent[3387]: (/Stage[main]/Firewall::Linux::Redhat/Package[iptables-services]/ensure) created
Jul 30 16:12:14 maca25400654fdd systemd: Stopping firewalld - dynamic firewall daemon...
Jul 30 16:12:14 maca25400654fdd systemd: Starting IPv4 firewall with iptables...
Jul 30 16:12:14 maca25400654fdd iptables.init: iptables: Applying firewall rules: iptables-restore: line 14 failed
Jul 30 16:12:14 maca25400654fdd iptables.init: [FAILED]
Jul 30 16:12:14 maca25400654fdd systemd: iptables.service: main process exited, code=exited, status=1/FAILURE
Jul 30 16:12:14 maca25400654fdd systemd: Failed to start IPv4 firewall with iptables.
Jul 30 16:12:14 maca25400654fdd systemd: Unit iptables.service entered failed state.
Jul 30 16:12:14 maca25400654fdd puppet-agent[3387]: Could not start Service[iptables]: Execution of '/usr/bin/systemctl start iptables' returned 1: Job for iptables.service failed. See 'systemctl status iptables.service' and 'journalctl -xn' for details.
Jul 30 16:12:14 maca25400654fdd puppet-agent[3387]: Wrapped exception:
Jul 30 16:12:14 maca25400654fdd puppet-agent[3387]: Execution of '/usr/bin/systemctl start iptables' returned 1: Job for iptables.service failed. See 'systemctl status iptables.service' and 'journalctl -xn' for details.
Jul 30 16:12:14 maca25400654fdd puppet-agent[3387]: (/Stage[main]/Firewall::Linux::Redhat/Service[iptables]/ensure) change from stopped to running failed: Could not start Service[iptables]: Execution of '/usr/bin/systemctl start iptables' returned 1: Job for iptables.service failed. See 'systemctl status iptables.service' and 'journalctl -xn' for details.
Jul 30 16:12:14 maca25400654fdd puppet-agent[3387]: (/Stage[main]/Quickstack::Neutron::Firewall::Gre/Firewall[002 gre]/ensure) created



How reproducible:
randomly

Steps to Reproduce:
1. Make a Neutron deployment over 1 controller, 1 networker and 2 computes


Additional info:
Running "systemclt start iptables" manually works successfully. Trying "
"systemctl status" and journalctl doesn't show any more info (it shows that iptables is up and running).

Comment 2 Leonid Natapov 2014-07-31 11:27:16 UTC
Happened also with my deployment. Neutron VXLAN on the networker machine.

Comment 4 Martin Magr 2014-08-01 08:24:20 UTC
We had the same problem in Packstack. Firewalld which is shutting down is killing iptables which is starting. Synchronization of those two processes will be needed. Lukas will provide patch for you.

Comment 5 Lukas Bezdicka 2014-08-01 08:27:38 UTC
https://github.com/redhat-openstack/astapor/pull/334 should fix the issue

Comment 10 Omri Hochman 2014-08-11 12:43:27 UTC
Verified with : 
ruby193-rubygem-staypuft-0.2.2-1.el6ost.noarch
rhel-osp-installer-0.1.9-1.el6ost.noarch.

Comment 11 errata-xmlrpc 2014-08-21 18:07:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-1090.html