Bug 1525369
| Summary: | Director installation, rabbitmq fails: epmd@0.0.0.0.socket failed to listen on sockets: Address already in use | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Eduard Barrera <ebarrera> | ||||||
| Component: | rabbitmq-server | Assignee: | Peter Lemenkov <plemenko> | ||||||
| Status: | CLOSED NOTABUG | QA Contact: | Udi Shkalim <ushkalim> | ||||||
| Severity: | unspecified | Docs Contact: | |||||||
| Priority: | unspecified | ||||||||
| Version: | 11.0 (Ocata) | CC: | apevec, chjones, ebarrera, jeckersb, lhh, pablo.iranzo, srevivo | ||||||
| Target Milestone: | --- | Keywords: | ZStream | ||||||
| Target Release: | --- | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2017-12-18 14:46:09 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
Created attachment 1367161 [details]
crashdump obtained starting rabbit manually
Is this deployment non-HA, single controller? Any standard HA installation should not use systemd at all for RabbitMQ. This appears to be an /etc/hosts misconfiguration. From the sosreport, I see: $ grep director etc/hosts 10.106.136.52 director director.sanitized-FQDN But there is no interface with that address configured: $ grep 10.106.136.52 ip_addr | wc -l 0 This is why RabbitMQ will not start. It needs to be able to correctly resolve that name. The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days |
Created attachment 1367159 [details] rabbit logs staring it manually /usr/lib/rabbitmq/bin/rabbitmq-server Description of problem: Director installation for OSP11 fails when trying to start rabbitmq with the following error: Nov 29 13:20:52 director groupadd[24538]: group added to /etc/group: name=rabbitmq, GID=494 Nov 29 13:20:52 director groupadd[24538]: group added to /etc/gshadow: name=rabbitmq Nov 29 13:20:52 director groupadd[24538]: new group: name=rabbitmq, GID=494 Nov 29 13:20:52 director useradd[24543]: new user: name=rabbitmq, UID=496, GID=494, home=/var/lib/rabbitmq, shell=/sbin/nologin Nov 29 13:20:54 director yum[24435]: Installed: rabbitmq-server-3.6.5-5.el7ost.noarch Nov 29 13:21:47 director rabbitmq-server[24969]: ERROR: epmd error for host director: timeout (timed out) <======================== Nov 29 13:21:47 director systemd[1]: rabbitmq-server.service: main process exited, code=exited, status=1/FAILURE Nov 29 13:21:49 director rabbitmqctl[25105]: Stopping and halting node rabbit@director ... Nov 29 13:21:56 director rabbitmqctl[25105]: Error: unable to connect to node rabbit@director: nodedown Nov 29 13:22:26 director rabbitmqctl[25105]: DIAGNOSTICS Nov 29 13:22:26 director rabbitmqctl[25105]: =========== Nov 29 13:22:26 director rabbitmqctl[25105]: attempted to contact: [rabbit@director] Nov 29 13:22:26 director rabbitmqctl[25105]: rabbit@director: Nov 29 13:22:26 director rabbitmqctl[25105]: * unable to connect to epmd (port 4369) on director: timeout (timed out) Nov 29 13:22:26 director rabbitmqctl[25105]: current node details: Nov 29 13:22:26 director rabbitmqctl[25105]: - node name: 'rabbitmq-cli-50@director' Nov 29 13:22:26 director rabbitmqctl[25105]: - home dir: /var/lib/rabbitmq Nov 29 13:22:26 director rabbitmqctl[25105]: - cookie hash: oV5IIuzubbTWlzyit/2bBg== Nov 29 13:22:26 director systemd[1]: Unit rabbitmq-server.service entered failed state. Nov 29 13:22:26 director systemd[1]: rabbitmq-server.service failed. Nov 29 13:21:16 director systemd[1]: epmd.0.0.socket failed to listen on sockets: Address already in use <======= Nov 29 13:21:16 director systemd[1]: Failed to listen on Erlang Port Mapper Daemon Activation Socket. Nov 29 13:21:16 director systemd[1]: Unit epmd.0.0.socket entered failed state. This socket comes from systemd init file: [Unit] Description=RabbitMQ broker After=network.target epmd.0.0.socket <====== Wants=network.target epmd.0.0.socket [Service] Type=notify User=rabbitmq .... One strange thing is that rabbit seems not enabled, the unit file is not on /etc/systemd/system/mutliuser.... but it is on a test system This socket is always present owned by systemd but It doesn't seems the problem since in a test system is also present too but rabbit is able to start. tcp 0 0 0.0.0.0:4369 0.0.0.0:* LISTEN 1/systemd A normal start via systemd doesn't throw any logs /var/logs/rabbitmq, but I managed to get something (logs and crash dump, attached )starting rabbit manually with /usr/lib/rabbitmq/bin/rabbitmq-server Version-Release number of selected component (if applicable): How reproducible: always Steps to Reproduce: 1. start rabbit via systemd #systemctl start rabbitmq-server 2. 3. Actual results: Nov 29 13:21:16 director systemd[1]: epmd.0.0.socket failed to listen on sockets: Address already in use <======= Nov 29 13:21:16 director systemd[1]: Failed to listen on Erlang Port Mapper Daemon Activation Socket. Nov 29 13:21:16 director systemd[1]: Unit epmd.0.0.socket entered failed state. Expected results: Rabbit started Additional info: