SElinux is the culprit.  I found the solution on the bind-users' mailing list (a post by Chuck Anderson) here:

https://www.mail-archive.com/bind-users@lists.isc.org/msg18593.html

The solution is to specify:

 avoid-v4-udp-ports { 1935; 2605; 4321; 6514; range 8610 8614; };
 avoid-v6-udp-ports { 1935; 2605; 4321; 6514; range 8610 8614; };

...in /etc/named.conf.

The reason the SElinux AVCs weren't being logged was because these were "silent denials" (controlled by dontaudit rules).  In order to see the actual denials you must run:

# semodule -DB

Considering the SELinux is enabled by default, it would be great if the default/stock named.conf already had the above options; it would save a lot of time for anyone trying to troubleshoot this (there are no clues on the system due to the silent denials).

Thanks.

We can open up more ports or allow udp to listen on all ports. The goal is to prevent a hacked named from using other well known ports.

Dan, do you intend to adjust the policy to let BIND to use those ports?

Please consider updating the policy to allow BIND to use any port it wants -- there's not much to be gained by doing otherwise, especially when it's already allowed to originate traffic from a far more recognizable reserved port.

I have a dual-stacked resolver on Fedora 20 that's been banging into denials for these ports on both protocols for a while now:

1935 2605 4321 4444 5546 8554 8610 8611 8612 8613 8614

That list may be of use for anyone wishing to try the avoid-*-ports method for the time being.

-Rob

Fedora policy has 

corenet_udp_bind_all_unreserved_ports(named_t)

Which allows named to bind to any UDP port > 1024.

(In reply to Jorge Fábregas from comment #0)
> Description of problem: This a new system with only named running on it
> (just as a recursive nameserver) serving thousands of requests per minute. 
> We have lots of warnings like this:
> 
> May 31 10:45:05 lindns named[1720]: dispatch 0x7ff7e40eb150:
> open_socket(0.0.0.0#2605) -> permission denied: continuing

To be honest, I'm not able to reproduce this on 6.5 with SELinux in Enforcing mode and named configured with:

use-v4-udp-ports { 1935; 2605; 4321; 6514; range 8610 8614; };
use-v6-udp-ports { 1935; 2605; 4321; 6514; range 8610 8614; };

From wireshark I can see that named used e.g. UDP port 2605, 4321 and others successfully.


Also based on Dan's comment #6, the policy should not cause issues like this.

Tomas,

You're not going to be able to reproduce it if you specify those two lines in named.conf (since you're actually telling bind NOT to use those ports) & therefore you won't see the "dispatch/open socket/permission denied" messages in syslog (or the corresponding AVCs in audit.log if you disable the silent denials from SElinux).

In a nutshell, please either:

- fix the selinux policy for RHEL6 (as it is in the latest Fedora)

or

- add thoe two directives to the default named.conf

Thanks.

-- 
Jorge

(In reply to Jorge Fábregas from comment #8)
> Tomas,
> 
> You're not going to be able to reproduce it if you specify those two lines
> in named.conf (since you're actually telling bind NOT to use those ports) &
> therefore you won't see the "dispatch/open socket/permission denied"
> messages in syslog (or the corresponding AVCs in audit.log if you disable
> the silent denials from SElinux).

No, I configured BIND to use *ONLY* those ports. Please check the comment #7 once again.

We are not going to do any changes until reproduced and proved there is really some issue in the policy or BIND configuration.

My mistake Tomas.  I see your point.  Well if you don't see the AVCs (after turning on silent denials) I'll assume the selinux policy has been updated to include those ports for bind?  (I don't have access to the box anymore).

(In reply to Jorge Fábregas from comment #10)
> My mistake Tomas.  I see your point.  Well if you don't see the AVCs (after
> turning on silent denials) I'll assume the selinux policy has been updated
> to include those ports for bind?  (I don't have access to the box anymore).

Yes I did. There are no messages about named WRT socket binding.

The policy doesn't seem to changed. Maybe you had some custom policy installed?

I'm closing this Bug as NOTABUG as it obviously works on RHEL-6.5 with SELinux in Enforcing mode.

Tomas,

If you would, at least, give me a break & allow me to reply back before closing the bug...

I fired up new VM (RHEL 6.5) and the issue is TOTALLY there.  I never said that I modified my selinux policy. In fact, I don't know which RHEL 6.5 you're running.  I'm not the only in this bug recognizing the issue is still there.

I'm uploading screencast in 5 minutes.

-- 
Jorge

Created attachment 946486 [details]
Screencast of Issue

I hope everything is clear now with the screencast.  This is a plain-vanilla system with JUST the updates as of today.

OK, I can see that now.

I was testing it on RHEL-6.5 Server. However I ran named in the foreground, not using service command. It was running with a different SELinux label, I think that's why it worked.

unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 named 1319 0.0  1.3 160548 13348 pts/0 Sl+ 12:45   0:00 named -u named -fg

started by 'service':
unconfined_u:system_r:named_t:s0 named    1247  0.0  0.6 160548 12112 ?        Ssl  11:21   0:00 /usr/sbin/named -u named


Now I can see the AVCs...

I'm really reluctant to modifying the default named configuration, especially because it will not make any change for users that are running named with custom configuration on top of RHEL-6.

I'm moving this bug to selinux-policy to consider adjusting it (as it is already done in Fedora).

Created attachment 946832 [details]
named AVCs

Shouldn't this be related to ip_local_port_range (default 32768:61000)? In other words:

avoid-v4-udp-ports { range 0 32767; range 61001 65535; };

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-1375.html