Fedora Account System
Red Hat Associate
Red Hat Customer
Description of problem: On an up-to-date rawhide machine: Mar 16 07:30:01 hammer automount[22941]: >> nfs bindresvport: Permission denied Mar 16 07:30:01 hammer automount[22941]: mount(nfs): nfs: mount failure saga:/export/data1 on /data/sw1 Mar 16 07:30:01 hammer automount[22941]: failed to mount /data/sw1 A bit later this worked. Very quiet machine otherwise. This is the first time I've seen this, but figured it was good to start documenting early... Version-Release number of selected component (if applicable): autofs-4.1.4-16.2.2 How reproducible: Not yet.
Seen on another machine too: Mar 16 04:02:19 cynosure automount[22930]: >> nfs bindresvport: Permission denied Mar 16 04:02:19 cynosure automount[22930]: mount(nfs): nfs: mount failure earth:/export/local on /opt/local Mar 16 04:02:19 cynosure automount[22930]: failed to mount /opt/local But can access /opt/local fine now.
How many concurrent mount requests do you have running? What is the output of netstat -an when the problem occurs?
I would put the number of concurrent mount requests at or near one. These are very quiet machines. Both of these accesses were started from nightly cron jobs. Perhaps selinux issues, though I see no avc: messages.
Still NEEDINFO. Need the netstat -an output when this occurs. Also, what's the version of libc and kernel? It is not likely that this is an autofs issue.
Created attachment 126230 [details] netstat -an output This is from "now", but should not be appreciably different from when the error occurred. glibc-2.4-4 kernel-2.6.15-1.2054_FC5 selinux-policy-targeted-2.2.23-15 I'll be rebooting with audit=1 to check on selinux for sure.
netstat output from "now" doesn't help. The idea, here, is to see what is going on at the time the problem occurs. For example, if you have ghosting enabled in your maps, and you have a cronjob running that stats every directory on your system, then you'll get a ton of mount activity. We need to see that to diagnose what the actual problem is. It would also be beneficial if you could provide your automount maps. Thanks.
Created attachment 126414 [details] netstat output This is output from the following cron job: /data/sw1/rawhide.update; netstat -an /bin/sh: /data/sw1/rawhide.update: /bin/sh: bad interpreter: No such file or directory Problem doesn't happen everyday, so not 100% consistent. I am seeing some selinux stuff along with some "already mounted" errors: Mar 21 10:57:54 hammer automount[9846]: mount(nfs): warning: /data/sw1 is already mounted Mar 21 10:57:54 hammer automount[9847]: failed to mount /data/sw1/mergeupd.pl Mar 21 10:57:54 hammer automount[9848]: mount(nfs): warning: /data/sw1 is already mounted Mar 21 10:57:54 hammer automount[9849]: failed to mount /data/sw1/mergeupd.pl Mar 21 10:57:54 hammer automount[9850]: mount(nfs): warning: /data/sw1 is already mounted Mar 21 10:57:54 hammer automount[9851]: failed to mount /data/sw1/mergeupd.pl Mar 21 10:57:54 hammer kernel: audit(1142963874.679:1928): login pid=9836 uid=0 old auid=0 new auid=0 Mar 21 10:58:03 hammer kernel: audit(1142963883.039:1934): login pid=9862 uid=0 old auid=4294967295 new auid=0 Mar 21 10:58:03 hammer automount[9872]: mount(nfs): warning: /data/sw1 is already mounted Mar 21 10:58:03 hammer automount[9873]: failed to mount /data/sw1/mergeupd.pl Mar 21 10:58:03 hammer automount[9875]: mount(nfs): warning: /data/sw1 is already mounted Mar 21 10:58:03 hammer automount[9876]: failed to mount /data/sw1/mergeupd.pl Mar 21 10:58:03 hammer automount[9877]: mount(nfs): warning: /data/sw1 is already mounted Mar 21 10:58:03 hammer automount[9878]: failed to mount /data/sw1/mergeupd.pl Mar 21 10:58:03 hammer kernel: audit(1142963883.319:1940): login pid=9862 uid=0 old auid=0 new auid=0 Mar 21 10:58:34 hammer automount[9916]: mount(nfs): warning: /home/orion is already mounted Mar 21 10:58:50 hammer automount[9927]: mount(nfs): warning: /data/sw1 is already mounted Mar 21 10:58:50 hammer automount[9928]: mount(nfs): warning: /data/sw1 is already mounted Mar 21 10:59:10 hammer automount[9957]: mount(nfs): warning: /data/sw1 is already mounted Mar 21 10:59:10 hammer automount[9958]: mount(nfs): warning: /data/sw1 is already mounted Mar 21 10:59:14 hammer automount[9960]: mount(nfs): warning: /data/sw1 is already mounted Mar 21 10:59:14 hammer automount[9961]: mount(nfs): warning: /data/sw1 is already mounted Mar 21 10:59:14 hammer automount[9962]: mount(nfs): warning: /data/sw1 is already mounted Mar 21 10:59:15 hammer automount[9965]: mount(nfs): warning: /data/sw1 is already mounted Mar 21 10:59:15 hammer automount[9966]: mount(nfs): warning: /data/sw1 is already mounted Mar 21 10:59:15 hammer automount[9968]: mount(nfs): warning: /data/sw1 is already mounted Mar 21 10:59:15 hammer automount[9969]: failed to mount /data/sw1/* Mar 21 10:59:15 hammer automount[9970]: mount(nfs): warning: /data/sw1 is already mounted Mar 21 10:59:15 hammer automount[9971]: failed to mount /data/sw1/* Mar 21 10:59:22 hammer kernel: SELinux: initialized (dev 0:1c, type nfs), uses genfs_contexts Mar 21 10:59:22 hammer kernel: audit(1142963962.715:1945): avc: denied { write } for pid=9978 comm="mount" name="mtab" dev=dm-0 ino=7053 scontext=system_u:system_r:mount_t:s0 tcontext=root:object_r:etc_t:s0 tclass=file And I'm seeing problems with things not showing up in mount/df but being mounted.
Okay, think this is an selinux issue, but perhaps you can give some guidance. With all selinux messages turned on (even noaudit), I see the following errors on a failed mount that I don't on a successful one: audit(1144362324.768:2579): avc: denied { name_bind } for pid=30139 comm="mount" src=636 scontext=system_u:system_r:mount_t:s0 tcontext=system_u:object_r:ldap_port_t:s0 tclass=tcp_socket Looks like the selinux policy may not be allowing the full range of possible ports? Thinking it's a specific port (ldap_port_t) rather than a general rpc port?
Another datapoint: Apr 7 08:00:01 hammer kernel: audit(1144418401.464:3266): avc: denied { name_bind } for pid=18947 comm="mount" src=891 scontext=system_u:system_r:mount_t:s0 tcontext=system_u:object_r:inetd_child_port_t:s0 tclass=tcp_socket Apr 7 08:00:01 hammer kernel: audit(1144418401.472:3267): avc: denied { name_bind } for pid=18947 comm="mount" src=892 scontext=system_u:system_r:mount_t:s0 tcontext=system_u:object_r:inetd_child_port_t:s0 tclass=tcp_socket
Are you using NIS? Is allow_ypbind boolean turned on?
(In reply to comment #10) > Are you using NIS? Yes. > Is allow_ypbind boolean turned on? Yes. Not sure what this would have to do with this as it is mount that is being blocked, and the ports involved are various ones like 636 (ldap), 891/892 (inetd_child_port_t). I imagine that the problem would occur without NIS and using local automount maps.
Because NIS causes every application that uses it to attempt bind to a random low number port. In enforcing mode, a avc message will be generated if it attempts to bind to a known port. These should be dontaudited. Are you running in permissive mode? Does the app work in enforcing mode? IE Are you just reporting the AVC messages but every thing seems to work?
Ah, thanks. The mount fails in enforcing mode. However, I am running with the enableaudit policy to see the denials that are causing problems. They are at least the following: audit(1144362324.768:2579): avc: denied { name_bind } for pid=30139 comm="mount" src=636 scontext=system_u:system_r:mount_t:s0 tcontext=system_u:object_r:ldap_port_t:s0 tclass=tcp_socket Apr 7 08:00:01 hammer kernel: audit(1144418401.464:3266): avc: denied { name_bind } for pid=18947 comm="mount" src=891 scontext=system_u:system_r:mount_t:s0 tcontext=system_u:object_r:inetd_child_port_t:s0 tclass=tcp_socket Apr 7 08:00:01 hammer kernel: audit(1144418401.472:3267): avc: denied { name_bind } for pid=18947 comm="mount" src=892 scontext=system_u:system_r:mount_t:s0 tcontext=system_u:object_r:inetd_child_port_t:s0 tclass=tcp_socket So, either mount needs to be able to bind to these ports, or it needs to retry so it can get a port it can use. Otherwise automount will occasionally fail to mount a nfs directory.
This is strange it should be retrying automatically. Please restorecon /etc/mtab. You might want to run restorecond service, and update to latest policy. NIS SHould fail on the portmap of "reserved SELinux ports" but should continue until it is successful. Dan
Labels are fine. Policy is the latest. Looking through the nfs mount code in mount (util-linux), mount will not retry an nfs mount when doing a foreground (no bg option) mount and it gets a bindresvport error. This is how automount runs mount. Automount itself does not then retry the nfs mount either. Perhaps one or the other needs to change? Please also see bug 155940 for a similar issue with ypbind startup. ypbind does not retry if it cannot connect to a port at startup. I've submitted a patch to do so if that is needed.
After looking at bindrsvcport we see the problem. Basically mount is not allowed to name_bind on defined ports and bindrsvcport will only retry on the address is busy error not on EPERM. So mount will fail if you are unlucky enough to hit one of these ports. bindrsvcport tries to bind to ports 600-1023, So I am trying to write policy to allow mount to name_bind to this range of ports.
fixed in selinux-policy-2.2.38-1.FC5.
Appears fixed. I'll reopen if I see it again.