Bug 1370052

Summary: openstack undercloud install does not open iptables rules for rabbitmq
Product: Red Hat OpenStack Reporter: tkammer
Component: instack-undercloudAssignee: Emilien Macchi <emacchi>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Arik Chernetsky <achernet>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 9.0 (Mitaka)CC: aschultz, dsneddon, dtantsur, emacchi, jcoufal, jslagle, mburns, mcornea, nlevinki, rhel-osp-director-maint, tkammer
Target Milestone: Upstream M3Keywords: Triaged
Target Release: 11.0 (Ocata)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-07-14 15:37:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description tkammer 2016-08-25 07:44:18 UTC
Description of problem:
IPtables rules are not opened for rabbitmq-server on the undercloud installation.
For cases where the default chain is set to "drop", the undercloud installation does not open the necessary rules to the firewall in order to successfully deploy the undercloud.

Version-Release number of selected component (if applicable):
9

How reproducible:
always

Steps to Reproduce:
1. set default chain rule to "drop"
2. run: openstack undercloud install 
3. see failure

Actual results:
Last login: Thu Aug 25 02:53:26 2016 from dhcp-4-181.tlv.redhat.com
[root@tkammer-test-undercloud-1 ~]# systemctl status rabbitmq-server
● rabbitmq-server.service - RabbitMQ broker
   Loaded: loaded (/usr/lib/systemd/system/rabbitmq-server.service; disabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/rabbitmq-server.service.d
           └─limits.conf
   Active: failed (Result: exit-code) since Thu 2016-08-25 03:01:18 EDT; 27min ago
 Main PID: 27741 (code=exited, status=1/FAILURE)

Aug 25 03:01:18 tkammer-test-undercloud-1.localdomain rabbitmqctl[27848]: attempted to contact: ['rabbit@tkammer-test-undercloud-1']
Aug 25 03:01:18 tkammer-test-undercloud-1.localdomain rabbitmqctl[27848]: rabbit@tkammer-test-undercloud-1:
Aug 25 03:01:18 tkammer-test-undercloud-1.localdomain rabbitmqctl[27848]: * unable to connect to epmd (port 4369) on tkammer-test-undercloud-1: address (cannot connect to host/port)
Aug 25 03:01:18 tkammer-test-undercloud-1.localdomain rabbitmqctl[27848]: current node details:
Aug 25 03:01:18 tkammer-test-undercloud-1.localdomain rabbitmqctl[27848]: - node name: 'rabbitmq-cli-89@tkammer-test-undercloud-1'
Aug 25 03:01:18 tkammer-test-undercloud-1.localdomain rabbitmqctl[27848]: - home dir: /var/lib/rabbitmq
Aug 25 03:01:18 tkammer-test-undercloud-1.localdomain rabbitmqctl[27848]: - cookie hash: 5gpMosQ/UDu8TSDH+oFveQ==
Aug 25 03:01:18 tkammer-test-undercloud-1.localdomain systemd[1]: Failed to start RabbitMQ broker.
Aug 25 03:01:18 tkammer-test-undercloud-1.localdomain systemd[1]: Unit rabbitmq-server.service entered failed state.
Aug 25 03:01:18 tkammer-test-undercloud-1.localdomain systemd[1]: rabbitmq-server.service failed.

Expected results:
passing undercloud installation

Additional info:
When adding explicitly the needed rules:
iptables -I INPUT -p tcp --dport 4369 --syn -j ACCEPT && iptables -I INPUT -p tcp --dport 59984 --syn -j ACCEPT
The installation passes without a problem

Comment 5 Alex Schultz 2016-12-22 19:50:46 UTC
So I'm not sure if we need additional rules around this to open this up externally. I checked and it should be listening on 127.0.0.1:4369 which should be covered by the accept all to lo interface rule. From what I saw on my environment, the connections were occuring over the 127.0.0.1 address and not over the public network. Does 'tkammer-test-undercloud-1' properly resolve to 127.0.0.1?  I thought there was some requirements about setting the host name properly in /etc/hosts for the install.

Comment 7 Alex Schultz 2017-07-14 15:37:53 UTC
I'm closing this bug as I haven't been able to reproduce this and I think if rabbitmq couldn't start we'd be seeing this more often.  If someone hits this bug again, please feel free to reopen it and provide additional details (and logs).