Bug 1360657 - rabbitmq-server fails to start
Summary: rabbitmq-server fails to start
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Fedora
Classification: Fedora
Component: rabbitmq-server
Version: 24
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Peter Lemenkov
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-07-27 09:10 UTC by Mathieu Bridon
Modified: 2016-08-31 16:17 UTC (History)
7 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2016-08-31 16:10:11 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Mathieu Bridon 2016-07-27 09:10:22 UTC
Description of problem:
I'm trying to start the service, to no avail.

Here's the error messages I get in the journal:

Jul 27 10:55:27 mathieus-laptop systemd[1]: Starting RabbitMQ broker...
Jul 27 10:55:27 mathieus-laptop audit[9052]: AVC avc:  denied  { read } for  pid=9052 comm="inet_gethost" name="unix" dev="proc" ino=4026532033 scontext=system_u:system_r:rabbitmq_t:s0 tcontext=system_u:object_r:proc_net_t:s0 tclass=file permissive=1
Jul 27 10:55:27 mathieus-laptop audit[8591]: AVC avc:  denied  { accept } for  pid=8591 comm="epmd" lport=4369 scontext=system_u:system_r:init_t:s0 tcontext=system_u:system_r:unconfined_service_t:s0 tclass=tcp_socket permissive=1
Jul 27 10:55:27 mathieus-laptop audit[8591]: AVC avc:  denied  { getattr } for  pid=8591 comm="epmd" laddr=127.0.0.1 lport=4369 faddr=127.0.0.1 fport=51345 scontext=system_u:system_r:init_t:s0 tcontext=system_u:system_r:unconfined_service_t:s0 tclass=tcp_socket permissive=1
Jul 27 10:55:31 mathieus-laptop rabbitmq-server[8939]: ERROR: epmd error for host mathieus-laptop: address (cannot connect to host/port)
Jul 27 10:55:31 mathieus-laptop systemd[1]: rabbitmq-server.service: Main process exited, code=exited, status=1/FAILURE
Jul 27 10:55:31 mathieus-laptop audit[9110]: AVC avc:  denied  { getattr } for  pid=9110 comm="rabbitmqctl" path="/var/lib/rabbitmq/mnesia/rabbit" dev="sda3" ino=2101255 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:rabbitmq_var_lib_t:s0 tclass=file permissive=1
Jul 27 10:55:31 mathieus-laptop audit[9123]: AVC avc:  denied  { getattr } for  pid=9123 comm="rabbitmqctl" path="/var/log/rabbitmq/rabbit" dev="sda3" ino=2101899 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:rabbitmq_var_log_t:s0 tclass=file permissive=1
Jul 27 10:55:31 mathieus-laptop rabbitmqctl[9057]: Stopping and halting node 'rabbit@mathieus-laptop' ...
Jul 27 10:55:34 mathieus-laptop rabbitmqctl[9057]: Error: unable to connect to node 'rabbit@mathieus-laptop': nodedown
Jul 27 10:55:37 mathieus-laptop rabbitmqctl[9057]: DIAGNOSTICS
Jul 27 10:55:37 mathieus-laptop rabbitmqctl[9057]: ===========
Jul 27 10:55:37 mathieus-laptop rabbitmqctl[9057]: attempted to contact: ['rabbit@mathieus-laptop']
Jul 27 10:55:37 mathieus-laptop rabbitmqctl[9057]: rabbit@mathieus-laptop:
Jul 27 10:55:37 mathieus-laptop rabbitmqctl[9057]:   * unable to connect to epmd (port 4369) on mathieus-laptop: address (cannot connect to host/port)
Jul 27 10:55:37 mathieus-laptop rabbitmqctl[9057]: current node details:
Jul 27 10:55:37 mathieus-laptop rabbitmqctl[9057]: - node name: 'rabbitmq-cli-31@mathieus-laptop'
Jul 27 10:55:37 mathieus-laptop rabbitmqctl[9057]: - home dir: /var/lib/rabbitmq
Jul 27 10:55:37 mathieus-laptop rabbitmqctl[9057]: - cookie hash: A0+gWuGkAgb8P/MD2n0e4A==
Jul 27 10:55:37 mathieus-laptop audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=rabbitmq-server comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Jul 27 10:55:37 mathieus-laptop systemd[1]: Failed to start RabbitMQ broker.
Jul 27 10:55:37 mathieus-laptop systemd[1]: rabbitmq-server.service: Unit entered failed state.
Jul 27 10:55:37 mathieus-laptop systemd[1]: rabbitmq-server.service: Failed with result 'exit-code'.

There's some SELinux messages, but I've set it to permissive and still rabbitmq fails to start.


Version-Release number of selected component (if applicable):
rabbitmq-server-3.6.2-3.fc24.noarch


How reproducible:
Always


Additional info:
Note that it was working just fine last week when I installed rabbitmq. I could start the service without issue at that time.

So it might be an update that broke it?

Comment 1 Peter Lemenkov 2016-08-31 15:52:00 UTC
[root@lemenkov ~]# systemctl status rabbitmq-server
● rabbitmq-server.service - RabbitMQ broker
   Loaded: loaded (/usr/lib/systemd/system/rabbitmq-server.service; disabled; vendor preset: disabled)
   Active: active (running) since Wed 2016-08-31 17:50:31 CEST; 10s ago
 Main PID: 8481 (beam.smp)
   Status: "Initialized"
    Tasks: 144 (limit: 512)
   CGroup: /system.slice/rabbitmq-server.service
           ├─8481 /usr/lib64/erlang/erts-7.3.1.2/bin/beam.smp -W w -A 128 -P 1048576 -K true -- -root /usr/lib64/erlang -progname erl -- -home /var/lib/rabbitmq -- -pa /usr/lib/rabbitmq/lib/rabbitmq_server-3.6.3
           ├─8563 /usr/lib64/erlang/erts-7.3.1.2/bin/epmd -daemon
           ├─8765 inet_gethost 4
           └─8766 inet_gethost 4

Aug 31 17:50:29 lemenkov.brq.redhat.com systemd[1]: Starting RabbitMQ broker...
Aug 31 17:50:30 lemenkov.brq.redhat.com rabbitmq-server[8481]:               RabbitMQ 3.6.3. Copyright (C) 2007-2016 Pivotal Software, Inc.
Aug 31 17:50:30 lemenkov.brq.redhat.com rabbitmq-server[8481]:   ##  ##      Licensed under the MPL.  See http://www.rabbitmq.com/
Aug 31 17:50:30 lemenkov.brq.redhat.com rabbitmq-server[8481]:   ##  ##
Aug 31 17:50:30 lemenkov.brq.redhat.com rabbitmq-server[8481]:   ##########  Logs: /var/log/rabbitmq/rabbit
Aug 31 17:50:30 lemenkov.brq.redhat.com rabbitmq-server[8481]:   ######  ##        /var/log/rabbitmq/rabbit
Aug 31 17:50:30 lemenkov.brq.redhat.com rabbitmq-server[8481]:   ##########
Aug 31 17:50:30 lemenkov.brq.redhat.com rabbitmq-server[8481]:               Starting broker...
Aug 31 17:50:31 lemenkov.brq.redhat.com systemd[1]: Started RabbitMQ broker.
Aug 31 17:50:31 lemenkov.brq.redhat.com rabbitmq-server[8481]:  completed with 0 plugins.
[root@lemenkov ~]#


I've tested SElinux and it's enabled:

[root@lemenkov ~]# getenforce
Enforcing
[root@lemenkov ~]# 


[root@lemenkov ~]# cat /etc/selinux/config 

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of these three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

Comment 2 Mathieu Bridon 2016-08-31 15:56:54 UTC
I guess an update might have fixed it?

Comment 3 Peter Lemenkov 2016-08-31 16:02:16 UTC
(In reply to Mathieu Bridon from comment #2)
> I guess an update might have fixed it?

I doubt that. I've just downgraded to rabbitmq-server-3.6.2-3.fc24.noarch and it still starts fine on my PC.

Perhaps someone messed with selinux permissions in your /var/lib/rabbitmq or /var/log/rabbitmq.

Try restoring selinux permissions by running

restorecon -R /var/lib/rabbitmq /var/log/rabbitmq

Comment 4 Peter Lemenkov 2016-08-31 16:10:11 UTC
Closing as WORKSFORME.

If you can reproduce it then feel free to reopen it.

Comment 5 Mathieu Bridon 2016-08-31 16:17:49 UTC
(In reply to Peter Lemenkov from comment #3)
> (In reply to Mathieu Bridon from comment #2)
> > I guess an update might have fixed it?
> 
> I doubt that. I've just downgraded to rabbitmq-server-3.6.2-3.fc24.noarch
> and it still starts fine on my PC.

I meant an update of something like the selinux policy.

In my original comment, when I said « it might be an update that broke it? », I didn't mean a rabbitmq update, since I hadn't had any after installing it, but I had received other updates, among which there might have been an selinux one.

> Perhaps someone messed with selinux permissions in your /var/lib/rabbitmq or
> /var/log/rabbitmq.
> 
> Try restoring selinux permissions by running
> 
> restorecon -R /var/lib/rabbitmq /var/log/rabbitmq

I'm not working with rabbitmq any more, so I don't have it setup any more.

Closing this is fine, since it works for you and I can't reproduce any more.

Sorry I didn't investigate further at the time I had the issue.


Note You need to log in before you can comment on or make changes to this bug.