Bug 739946 - NFS server fails to start
NFS server fails to start
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: selinux-policy-targeted (Show other bugs)
16
x86_64 Linux
unspecified Severity high
: ---
: ---
Assigned To: Miroslav Grepl
Ben Levenson
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2011-09-20 10:14 EDT by Göran Uddeborg
Modified: 2011-10-09 15:35 EDT (History)
5 users (show)

See Also:
Fixed In Version: selinux-policy-3.10.0-38.fc16
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2011-10-09 15:35:21 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
My local workaround module (278 bytes, application/octet-stream)
2011-09-21 03:36 EDT, Göran Uddeborg
no flags Details

  None (edit)
Description Göran Uddeborg 2011-09-20 10:14:12 EDT
Description of problem:
When trying to start nfs-server.service it enters a failed state.

I should warn that I'm using an F15 kernel, in case that matters.  (The nvidia drivers were not available for F16 kernels from rpmfusion, last I looked.)

Version-Release number of selected component (if applicable):
nfs-utils-1.2.4-8.fc16.x86_64
selinux-policy-targeted-3.10.0-28.fc16.noarch
kernel-2.6.40.3-0.fc15.x86_64

How reproducible:
Every time

Steps to Reproduce:
1. systemctl start nfs-server.service
  
Actual results:
Job failed. See system logs and 'systemctl status' for details.

Expected results:
Running nfsd servers

Additional info:
I get the following messages:

Sep 20 16:00:29 freddi rpc.nfsd[1942]: rpc.nfsd: unable to bind inet TCP socket: errno 13 (Permission denied)
Sep 20 16:00:29 freddi rpc.nfsd[1942]: rpc.nfsd: unable to bind inet6 TCP socket: errno 13 (Permission denied)
Sep 20 16:00:29 freddi rpc.nfsd[1942]: rpc.nfsd: unable to set any sockets for nfsd
Sep 20 16:00:29 freddi systemd[1]: nfs-server.service: control process exited, code=exited status=1
Sep 20 16:00:29 freddi systemd[1]: Unit nfs-server.service entered failed state.

This looks very much like bug 732968, but that was supposed to be fixed with this version of nfs-utils.

If I run /usr/bin/rpc.nfsd on the command line, the server does come up.  This difference normally would have me suspect SELinux denials.  But doing an "ausearch -m avc -ts recent" immediately after the failed attempt doesn't show anything.
Comment 1 Göran Uddeborg 2011-09-20 14:00:54 EDT
I forgot to turn off "dontaudit" rules.  When I do that I do get the AVC:s below.  So I guess either this is something that needs a true F16 kernel to work (like in the F16 discussion in bug 729451) or there is indeed something that needs to be fixed in the policy.

time->Tue Sep 20 19:53:29 2011
type=SYSCALL msg=audit(1316541209.629:430): arch=c000003e syscall=49 success=no exit=-13 a0=4 a1=7f46a1c872d0 a2=10 a3=7fff566a1470 items=0 ppid=1 pid=3289 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="rpc.nfsd" exe="/usr/sbin/rpc.nfsd" subj=system_u:system_r:nfsd_t:s0 key=(null)
type=AVC msg=audit(1316541209.629:430): avc:  denied  { name_bind } for  pid=3289 comm="rpc.nfsd" src=2049 scontext=system_u:system_r:nfsd_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket
----
time->Tue Sep 20 19:53:29 2011
type=SYSCALL msg=audit(1316541209.645:431): arch=c000003e syscall=49 success=no exit=-13 a0=4 a1=7f46a1c87370 a2=1c a3=7fff566a172c items=0 ppid=1 pid=3289 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="rpc.nfsd" exe="/usr/sbin/rpc.nfsd" subj=system_u:system_r:nfsd_t:s0 key=(null)
type=AVC msg=audit(1316541209.645:431): avc:  denied  { name_bind } for  pid=3289 comm="rpc.nfsd" src=2049 scontext=system_u:system_r:nfsd_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket
Comment 2 Daniel Walsh 2011-09-20 14:33:09 EDT
Ok we were not aware of this.  I just added this to F16 policy.

Fixed in selinux-policy-3.10.0-32.fc16
Comment 3 Göran Uddeborg 2011-09-21 03:36:13 EDT
Created attachment 524147 [details]
My local workaround module

A little additional info: I made myself a module to work around the problem until the official fix is available, and realised I need to allow udp_socket in addition to tcp_socket.  See the attachment for exactly what I did.

Maybe I should have given the port number a name instead of allowing any unreserved_port_t.  But I haven't learnt enough about SELinux to do that yet.  If it is possible to do such assignments in a module at all.
Comment 4 Daniel Walsh 2011-09-21 11:21:14 EDT
Ok will add corenet_udp_bind_nfs_port(nfsd_t)

Fixed in selinux-policy-3.10.0-33.fc16
Comment 5 Miroslav Grepl 2011-09-27 03:13:02 EDT
*** Bug 733127 has been marked as a duplicate of this bug. ***
Comment 6 Steve Dickson 2011-10-03 11:21:26 EDT
*** Bug 735547 has been marked as a duplicate of this bug. ***
Comment 7 Daniel Walsh 2011-10-03 11:52:08 EDT
Can someone confirm that with the latest policy nfs server is working in enforcing mode?
Comment 8 Peter Lemenkov 2011-10-03 12:01:20 EDT
Da(In reply to comment #7)
> Can someone confirm that with the latest policy nfs server is working in
> enforcing mode?

Wait a second - I'm upgrading my box right now.
Comment 9 Peter Lemenkov 2011-10-03 12:36:16 EDT
I just installed selinux-policy-3.10.0-35.fc16.noarch and selinux-policy-targeted-3.10.0-35.fc16.noarch and situation becomes much more spectacular. First of all is still doesn't started by systemd (I think it's an unrelated issue) so I still need to start it manually.

But when I start it manually it shows the following message:

Oct  3 20:31:49 nostromo kernel: [  150.418881] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
Oct  3 20:31:49 nostromo kernel: [  150.423786] NFSD: starting 90-second grace period
Oct  3 20:31:50 nostromo rpc.mountd[1338]: Could not bind socket: (13) Permission denied
Oct  3 20:31:50 nostromo rpc.mountd[1338]: Could not bind socket: (13) Permission denied
Oct  3 20:31:50 nostromo rpc.mountd[1338]: Could not bind socket: (13) Permission denied
Oct  3 20:31:50 nostromo rpc.mountd[1338]: Could not bind socket: (13) Permission denied
Oct  3 20:31:50 nostromo rpc.mountd[1338]: Could not bind socket: (13) Permission denied
Oct  3 20:31:50 nostromo rpc.mountd[1338]: Could not bind socket: (13) Permission denied
Oct  3 20:31:50 nostromo rpc.mountd[1338]: Could not bind socket: (13) Permission denied
Oct  3 20:31:50 nostromo rpc.mountd[1338]: Could not bind socket: (13) Permission denied
Oct  3 20:31:50 nostromo rpc.mountd[1338]: Could not bind socket: (13) Permission denied
Oct  3 20:31:50 nostromo rpc.mountd[1338]: Could not bind socket: (13) Permission denied
Oct  3 20:31:50 nostromo rpc.mountd[1338]: Could not bind socket: (13) Permission denied
Oct  3 20:31:50 nostromo rpc.mountd[1338]: Could not bind socket: (13) Permission denied
Oct  3 20:31:50 nostromo rpc.mountd[1338]: mountd: could not create listeners
Oct  3 20:31:50 nostromo systemd[1]: nfs-server.service: control process exited, code=exited status=1


If I switch to selinux=permissive then these messages are gone.
Comment 10 Daniel Walsh 2011-10-03 13:25:52 EDT
Do you see avc messages in /var/log/audit/audit.log?
Comment 11 Peter Lemenkov 2011-10-03 13:31:29 EDT
(In reply to comment #10)
> Do you see avc messages in /var/log/audit/audit.log?

Nope, it's empty (don't know why).
Comment 12 Daniel Walsh 2011-10-03 13:46:30 EDT
Peter what exactly do I need to do to start this service.

It is failing to start for me with SELinux enforcing and permissive.

/bin/systemctl  start nfs-server.service
Comment 13 Miroslav Grepl 2011-10-03 14:51:06 EDT
Strange I can start it either without an entry in the /etc/exports file

# systemctl start nfs-server.service
# systemctl status nfs-server.service
nfs-server.service - NFS Server
	  Loaded: loaded (/lib/systemd/system/nfs-server.service; disabled)
Active: active (running) since Mon, 03 Oct 2011 20:45:42 +0200; 5s ago


or with an entry in the /etc/exports file
Comment 14 Miroslav Grepl 2011-10-03 14:51:38 EDT
Peter, what is your configuration?
Comment 15 Daniel Walsh 2011-10-03 15:34:17 EDT
Ok we had to add port 20048 and 20049 as nfs_port_t and then it works.

semanage port -a -t nfs_port_t -p tcp 20048-20049
semanage port -a -t nfs_port_t -p udp 20048-20049

Should modify your port defs to allow the access.

selinux-policy-3.10.0-36.fc16

Should have this fix.
Comment 16 Miroslav Grepl 2011-10-03 15:41:50 EDT
Could you test it with

http://koji.fedoraproject.org/koji/buildinfo?buildID=266665
Comment 17 Jeff Layton 2011-10-03 17:49:41 EDT
(In reply to comment #15)
> Ok we had to add port 20048 and 20049 as nfs_port_t and then it works.
> 
> semanage port -a -t nfs_port_t -p tcp 20048-20049
> semanage port -a -t nfs_port_t -p udp 20048-20049
> 

I don't think we have any fixed port numbers in that range. mountd won't necessarily bind to those ports every time (unless you configure it to do so). It just asks for a port from libtirpc.
Comment 18 Göran Uddeborg 2011-10-04 04:47:36 EDT
Just in case the "think" was really a sign of uncertainty, and not just rhetorical, here is a little quote from the rpc.mountd manual

       -p  or  --port num
              Specifies the port number used for  RPC  listener  sockets.   If
              this  option  is  not  specified,  rpc.mountd  chooses  a random
              ephemeral port for each listener socket.

rpc.mountd on the machine I'm writing this has picked 15587, 15591, 15595, 15599, 15603, 15607, 15611, 15615, 15619, 15623, 15627, and 15631.

Would it make sense to ask for this to be added by default to nfs-utils?  Is there some "IANA" style function within Fedora where some port numbers could be registered?

Or would it be better to put rpc.mountd in a domain of its own, a domain which can name_bind to unreserved_port_t?
Comment 19 Fedora Update System 2011-10-04 07:16:14 EDT
selinux-policy-3.10.0-36.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/selinux-policy-3.10.0-36.fc16
Comment 20 Jeff Layton 2011-10-04 07:32:56 EDT
Yes, that "think" was rhetorical...

In order for this scheme to work, you'll need to insist that rpc.mountd use a fixed port number. You'll probably also need to do the same thing for rpc.statd.

Here's the catch though -- a lot of existing shops already set rpc.mountd  and statd to a static port number in order to make it simpler to firewall, and they don't necessarily pick the same ports. Switching that port number on them may be a problem for those folks (though they should be able to adapt). If we want to do this, then we really need some sort of way to transition those people smoothly.

Allowing name_bind to unreserved_port_t would be simpler...

Here's one possibility for containing mountd and statd:

Add 2 new selinux booleans to allow mountd and statd to bind to any port. We'd then ship /etc/sysconfig/nfs with the ports set to a fixed number. If someone wants to allow these daemons allow to bind somewhere else, they can set those booleans.
Comment 21 Göran Uddeborg 2011-10-04 08:17:11 EDT
rpc.statd actually runs in the rpcd_t domain, not in nfsd_t.  (At least for me when using 3.10.0-28.fc16; I haven't had an opportunity to upgrade yet.)

Doing

  sesearch --allow --source=rpcd_t --class=tcp_socket --perm=name_bind

gives me the same set of rules as the same query for nfsd_t does.  I don't understand why I haven't seen the same problems with rpc.statd as I saw with rpc.nfsd.
Comment 22 Fedora Update System 2011-10-04 16:48:50 EDT
Package selinux-policy-3.10.0-36.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing selinux-policy-3.10.0-36.fc16'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/selinux-policy-3.10.0-36.fc16
then log in and leave karma (feedback).
Comment 23 Peter Lemenkov 2011-10-05 01:22:38 EDT
Works for me now, thanks!
Comment 24 Miroslav Grepl 2011-10-05 01:34:30 EDT
Could you update karma. Thank you.
Comment 25 Fedora Update System 2011-10-09 15:35:21 EDT
selinux-policy-3.10.0-38.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.