Description of problem: When starting RabbitMQ (an AMQP broker) you have to set SELinux to Permissive, otherwise it cannot bind to a port. The AVC is: time->Wed Nov 20 12:48:53 2013 type=SYSCALL msg=audit(1384951733.587:103679): arch=c000003e syscall=49 success=yes exit=0 a0=10 a1=7f26c833cab0 a2=1c a3=a items=0 ppid=1 pid=28654 auid=4294967295 uid=991 gid=989 euid=991 suid=991 fsuid=991 egid=989 sgid=989 fsgid=989 ses=4294967295 tty=(none) comm="beam.smp" exe="/usr/lib64/erlang/erts-5.10.3/bin/beam.smp" subj=system_u:system_r:rabbitmq_beam_t:s0 key=(null) type=AVC msg=audit(1384951733.587:103679): avc: denied { name_bind } for pid=28654 comm="beam.smp" src=10000 scontext=system_u:system_r:rabbitmq_beam_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket audit2allow says: #============= rabbitmq_beam_t ============== #!!!! This avc can be allowed using the boolean 'nis_enabled' allow rabbitmq_beam_t unreserved_port_t:tcp_socket name_bind; I don't believe the comment about the boolean is relevant here, since this machine does not use/need NIS. Version-Release number of selected component (if applicable): selinux-policy-3.12.1-74.11.fc19.noarch rabbitmq-server-3.1.5-1.fc19.noarch erlang-erts-R16B-02.3.fc19.x86_64 How reproducible: 100% Steps to Reproduce: 1. Enable SELinux. 2. systemctl start rabbitmq-server 3. Actual results: Fails to start.
RabbitMQ listens on port 5672. This port is standardized for AMQP (unlike AMQP itself which is a total non-standard). It also uses a frankly confusing set of other ports for Erlang, apparently randomly assigned using some sort of Erlang portmapper. NFSv3 called, wants its stupid design back. No idea what NIS has to do with any of this.
OK so what happens is this: Erlang's "kernel" has some port settings: inet_dist_listen_min inet_dist_listen_max These can be set in the RabbitMQ configuration file. As far as I can tell there is no published default for these settings. When an Erlang process starts up it chooses a port from this range (I think starting with the lowest). It also registers its use of this port with a process called epmd -- the Erlang Port Mapper Daemon. You can also query epmd's current port mapping: # epmd -names epmd: up and running on port 4369 with data: name rabbit at port 56590 As noted there, epmd uses its own well-known port (4369).
I see from AVC msgs, it wants to use tcp/10000. So it can use random ports from the defined range, right?
It's very unclear to me what's happening, and not documented *at all*. However I spent some time last week looking at the source of the Erlang kernel: https://github.com/erlang/otp/blob/maint/lib/kernel/src/inet_tcp_dist.erl As you can see from the do_listen/1 function, if inet_dist_list_{min,max} are not set and there are no options that override them, then it chooses {0, 0}. That then causes the following loop to count up from 0 looking for a port that is not in use: do_listen(First,Last,Options) -> case inet_tcp:listen(First, Options) of {error, eaddrinuse} -> do_listen(First+1,Last,Options); Other -> Other end. The code gets very complex from then on, but if it's really binding with "port 0" then obviously the Linux kernel is going to choose a random port from the range in /proc/sys/net/ipv4/ip_local_port_range. (Port 32768, not port 10000 - no idea where that's coming from). TBH I really need to strace RabbitMQ to find out what's really happening. Unfortunately I just nuked that machine, but I'm going to reinstall it shortly.
Created attachment 828729 [details] recursive strace from starting up rabbit-mq server with SELinux Enforcing
Created attachment 828734 [details] recursive strace from starting up rabbit-mq server with SELinux Enforcing (updated, xz-compressed) The previous log was truncated in some way. This is a new log hopefully showing the complete start up sequence (also with SELinux Enforcing, so it is not successful). I have xz-compressed it because the original size is 27 MB.
Suspecting this is really RabbitMQ being broken, or at least "under-specified", I'm making this dependent on bug 998682.
Here's where the port 10000 thing comes in: https://github.com/rabbitmq/rabbitmq-server/blob/b47325e38b578b8e629baaeb2ab04040b273634f/src/rabbit_networking.erl#L42 Which is used later on while trying to probe for whether ipv6 is available and in single or dual stack: https://github.com/rabbitmq/rabbitmq-server/blob/b47325e38b578b8e629baaeb2ab04040b273634f/src/rabbit_networking.erl#L399 Ultimately ipv6_status hits the case of "port in use" (which is wrong -- instead it gets EACCESS due to selinux denial) here : https://github.com/rabbitmq/rabbitmq-server/blob/b47325e38b578b8e629baaeb2ab04040b273634f/src/rabbit_networking.erl#L477 and recursively calls itself with the next port. In my case on Fedora 20, it tries every port in succession until it gets to 32768 where it finally succeeds. This is where ephemeral_port_t begins and rabbitmq is allowed to name_bind there: $ sesearch -A -s rabbitmq_beam_t -p name_bind | grep rabbitmq.*ephemeral.*tcp allow rabbitmq_beam_t ephemeral_port_t : tcp_socket { name_bind name_connect } ; $ sepolicy network -t ephemeral_port_t | grep tcp ephemeral_port_t: tcp: 32768-61000 The end result is a ton of AVC denials and an extremely slow boot as it recursivlely calls ipv6_status to a depth of 22768 frames. But in the end, at least on my system, it does eventually start: $ telnet localhost amqp Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. testing AMQP Connection closed by foreign host.
Hi, I fixed this in selinux-policy-3.12.1-185.fc20 https://github.com/selinux-policy/selinux-policy/commit/0901e1c899feb997a224d87190209fc50b66e75a
Sorry this is F19 bug, this will be also backported.
This message is a notice that Fedora 19 is now at end of life. Fedora has stopped maintaining and issuing updates for Fedora 19. It is Fedora's policy to close all bug reports from releases that are no longer maintained. Approximately 4 (four) weeks from now this bug will be closed as EOL if it remains open with a Fedora 'version' of '19'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 19 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Fedora 19 changed to end-of-life (EOL) status on 2015-01-06. Fedora 19 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed.
I did experience this on F20.