Bug 1651032 - HAproxy fails to start during undercloud installation because rabbitmq is already binding on port 5672
Summary: HAproxy fails to start during undercloud installation because rabbitmq is alr...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: puppet-rabbitmq
Version: 10.0 (Newton)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 10.0 (Newton)
Assignee: Michele Baldessari
QA Contact: pkomarov
URL:
Whiteboard:
: 1651170 (view as bug list)
Depends On:
Blocks: 1651355 1651357
TreeView+ depends on / blocked
 
Reported: 2018-11-19 00:57 UTC by Marius Cornea
Modified: 2018-11-26 18:01 UTC (History)
12 users (show)

Fixed In Version: puppet-rabbitmq-5.6.0-5.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1651355 (view as bug list)
Environment:
Last Closed: 2018-11-26 18:00:40 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:3674 0 None None None 2018-11-26 18:01:27 UTC

Description Marius Cornea 2018-11-19 00:57:36 UTC
Description of problem:

[root@undercloud-0 stack]# systemctl status haproxy
● haproxy.service - HAProxy Load Balancer
   Loaded: loaded (/usr/lib/systemd/system/haproxy.service; enabled;
vendor preset: disabled)
   Active: failed (Result: exit-code) since Sat 2018-11-17 11:31:08
EST; 19min ago
 Main PID: 19203 (code=exited, status=1/FAILURE)

Nov 17 11:31:08 undercloud-0.redhat.local haproxy[19204]: Proxy
heat_api started.
Nov 17 11:31:08 undercloud-0.redhat.local haproxy[19204]: Proxy ironic started.
Nov 17 11:31:08 undercloud-0.redhat.local
haproxy-systemd-wrapper[19203]: [ALERT] 320/113108 (19204) : Starting
proxy rabbitmq: cannot bind socket [192.168.24.3:5672]
Nov 17 11:31:08 undercloud-0.redhat.local haproxy[19204]: Proxy
ironic-inspector started.
Nov 17 11:31:08 undercloud-0.redhat.local haproxy[19204]: Proxy
keystone_admin started.
Nov 17 11:31:08 undercloud-0.redhat.local haproxy[19204]: Proxy
keystone_public started.
Nov 17 11:31:08 undercloud-0.redhat.local
haproxy-systemd-wrapper[19203]: haproxy-systemd-wrapper: exit, haproxy
RC=1
Nov 17 11:31:08 undercloud-0.redhat.local systemd[1]: haproxy.service:
main process exited, code=exited, status=1/FAILURE
Nov 17 11:31:08 undercloud-0.redhat.local systemd[1]: Unit
haproxy.service entered failed state.
Nov 17 11:31:08 undercloud-0.redhat.local systemd[1]: haproxy.service failed.


[root@undercloud-0 stack]# grep -A6 rabbitmq /etc/haproxy/haproxy.cfg
listen rabbitmq
  bind 192.168.24.3:5672 transparent
  option tcpka
  timeout client 0
  timeout server 0
  server 192.168.24.1 192.168.24.1:5672 check fall 5 inter 2000 rise 2


[root@undercloud-0 stack]# ss -lpn  | grep :5672
tcp    LISTEN     0      128      :::5672                 :::*
          users:(("beam.smp",pid=15969,fd=52))

[root@undercloud-0 stack]# ps axu  | grep 15969
root       670  0.0  0.0 112708   976 pts/0    S+   11:53   0:00 grep
--color=auto 15969
rabbitmq 15969  1.4  0.8 4695972 212404 ?      Ssl  11:29   0:21
/usr/lib64/erlang/erts-7.3.1.4/bin/beam.smp -W w -A 128 -P 1048576 -K
true -- -root /usr/lib64/erlang -progname erl -- -home
/var/lib/rabbitmq -- -pa
/usr/lib/rabbitmq/lib/rabbitmq_server-3.6.3/ebin -noshell -noinput -s
rabbit boot -sname rabbit@undercloud-0 -boot start_sasl -config
/etc/rabbitmq/rabbitmq -kernel inet_default_connect_options
[{nodelay,true}] -sasl errlog_type error -sasl sasl_error_logger false
-rabbit error_logger
{file,"/var/log/rabbitmq/rabbit"} -rabbit
sasl_error_logger
{file,"/var/log/rabbitmq/rabbit"} -rabbit
enabled_plugins_file "/etc/rabbitmq/enabled_plugins" -rabbit
plugins_dir "/usr/lib/rabbitmq/lib/rabbitmq_server-3.6.3/plugins"
-rabbit plugins_expand_dir
"/var/lib/rabbitmq/mnesia/rabbit@undercloud-0-plugins-expand" -os_mon
start_cpu_sup false -os_mon start_disksup false -os_mon start_memsup
false -mnesia dir "/var/lib/rabbitmq/mnesia/rabbit@undercloud-0"
-kernel inet_dist_listen_min 25672 -kernel inet_dist_listen_max 25672


Version-Release number of selected component (if applicable):
2018-11-15.2

How reproducible:
100%

Steps to Reproduce:
1. Install OSP10 SSL enabled undercloud

Actual results:
Installation fails because haproxy is in failed state.

Expected results:
Installation runs clean.

Additional info:

Comment 2 Michele Baldessari 2018-11-19 14:21:04 UTC
*** Bug 1651170 has been marked as a duplicate of this bug. ***

Comment 10 pkomarov 2018-11-20 13:59:44 UTC
Verified, 

[stack@undercloud-0 ~]$ systemctl status haproxy
● haproxy.service - HAProxy Load Balancer
   Loaded: loaded (/usr/lib/systemd/system/haproxy.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2018-11-20 08:42:50 EST; 13min ago
 Main PID: 18976 (haproxy-systemd)
    Tasks: 3
   CGroup: /system.slice/haproxy.service
           ├─18976 /usr/sbin/haproxy-systemd-wrapper -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid
           ├─18977 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
           └─18978 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
[stack@undercloud-0 ~]$ cat core_puddle_version 
2018-11-19.2[stack@undercloud-0 ~]$ rhos-release -L
Installed repositories (rhel-7.6):
  10
  ceph-2
  ceph-osd-2
  rhel-7.6
[stack@undercloud-0 ~]$  grep -A6 rabbitmq /etc/haproxy/haproxy.cfg
listen rabbitmq
  bind 192.168.24.3:5672 transparent
  option tcpka
  timeout client 0
  timeout server 0
  server 192.168.24.1 192.168.24.1:5672 check fall 5 inter 2000 rise 2

[stack@undercloud-0 ~]$ ss -lpn  | grep :5672
tcp    LISTEN     0      128    192.168.24.3:5672                  *:*                  
tcp    LISTEN     0      128    192.168.24.1:5672                  *:*

Comment 12 errata-xmlrpc 2018-11-26 18:00:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3674


Note You need to log in before you can comment on or make changes to this bug.