Bug 126129 (IT_59244)

Summary: rpc.mountd: getfh failed: Operation not permitted
Product: Red Hat Enterprise Linux 3 Reporter: Thomas J. Baker <tjb>
Component: nfs-utilsAssignee: Steve Dickson <steved>
Status: CLOSED DUPLICATE QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: 3.0CC: bfox, guy_megouri_cohen, muno, philippe.giacinti, tao, trondeg, tstewart
Target Milestone: ---   
Target Release: ---   
Hardware: i686   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2005-07-27 14:50:57 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Thomas J. Baker 2004-06-16 14:18:41 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6)
Gecko/20040510 Galeon/1.3.15

Description of problem:
After working fine for about two weeks, the nfs server started failing
to export home directories to certain clients while others worked
fine. For the clients that were failing, the server would log errors
as follows:

Jun 16 09:59:10 blackstar rpc.mountd: authenticated mount request from
zeke.sr.unh.edu:613 for /home/rcc/teh (/home/rcc)
Jun 16 09:59:10 blackstar rpc.mountd: getfh failed: Operation not
permitted

The only solution was to restart nfs and the problem went away.


Version-Release number of selected component (if applicable):
nfs-utils-1.0.6-21EL

How reproducible:
Didn't try


Additional info:

This is the first time this has ever happened. The server has been up
for two weeks and is fully updated.

Comment 1 Thomas J. Baker 2004-06-17 20:41:06 UTC
This happened again today. Restarting NFS fixed it. If you have any
ideas on what to look for when it happens again, let me know.

Comment 2 Trond Eivind Glomsrød 2004-06-18 07:32:20 UTC
FWIW, it happens occasinally here too.

Comment 3 Suzanne Hillman 2004-07-15 19:45:45 UTC
If you have a support contract, you might be best off going through
support the next time this happens:

https://www.redhat.com/apps/support/

Comment 4 Luc Lalonde 2004-07-29 01:39:04 UTC
We're getting the same messages here with RHAS 3.0

This seems to be unresolved for quite some time:

https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=92081

We're working around the problem by adding a CRON job that re-exports
every four hours:

exportfs -r

Comment 6 Ray Muno 2005-02-02 16:57:16 UTC
We also have this problem. Exportfs -r does not fix it, only
/etc/init.d/nfs restart

This is a showstopper for us. We have can kludge to fix this but we
would rather have it work.

Here is our synopsis:

We have a few boxes setup to evaluate NFS server replacement. Two of
them are running Redhat Enterprise 3 AS. One of them is a little behind
in terms of updates and is running an XFS (SGI journaled filesystem)
enabled kernel. The other is fully up to date in Redhat's view.

Everything works EXCEPT....

Problem scenario

Filesystems are exported to netgroups. Netgroups come from NIS.

Client mounts filesystem and then unmounts. This is typical behavior for
us since we use automounted filesystems.

While unmounted, exportfs is run on the server.

Client can no longer mount the file system.

Only way to restart service for that client is to do
"/etc/init.d/nfs restart".

If the client is explicitly listed in /etc/exports, not @netgroup, it
does not fail.

Log message on the server is:

Feb  2 08:24:28 SERVER rpc.mountd: authenticated mount request from
CLIENT.me.umn.edu:931 for /scratch/test (/scratch/test)
Feb  2 08:24:28 SERVER rpc.mountd: getfh failed: Operation not permitted

If exportfs is run while the client has it mounted, there is no problem.

Exportfs behavior if filesystem is still mounted

[root@SERVER etc]# exportfs -va
exporting @NFS-TEST:/scratch/test
reexporting CLIENT.me.umn.edu:/scratch/test to kernel

exportfs behavior if filesystem has been mounted and umnmounted.

[root@SERVER etc]# exportfs -va
exporting @NFS-TEST:/scratch/test
unexporting CLIENT.me.umn.edu:/scratch/test from kernel

Eliminating NIS netgroups and using local files does not fix the
problem.  The only fix seems to be an explicit export to each client.

This problem looks like it has been around for a long time, in some
form, even back in to RH 7.x.  The "fixes" people report, that we have
found, offer us no remedy.

It looks like something is broken in rpc.mountd.

This problem is 100% repeatable. It is definitely a server issue. It
affects every client OS we run, Linux, IRIX, Solaris and FreeBSD.


Comment 7 jehan procaccia 2005-02-10 16:21:04 UTC
I am experiencing the same problem, and it becomes really anoying . 
If someone has a solution ... please let us know .
If this is a real bug (not tunning or configuration), should we
complain to redhat or we have to wait for a new kernel release ?
mine is:
2.4.21-27.ELsmp on Red Hat Enterprise Linux ES release 3 (Taroon Update 4)

Comment 8 Ray Muno 2005-02-10 17:09:08 UTC
We have definitley narrowed pur problem down to netgroup expansion in
/etc/exports.  

To remedy the situation, we have implemented a Perl script that reads a copy of
the original exports file, expands all netgroups to IP numbers for individual
hosts, and generates a new exports file with each host explicitly listed for
each exported directory.  This has taken care of our problem.

It looks like this problem is gone in Fedora Core 3. Perhaps there is hope for
RHEL 4. We have a box getting setup with the Beta version to test.

Comment 10 Tim Stewart 2005-06-21 22:03:54 UTC
Any news on whether or not this is fixed with RHEL4?  I'm experiencing the same behaviour as above on 
RHEL3.

Comment 11 Ray Muno 2005-07-05 13:29:01 UTC
We did not see the problem on RHEL4.

Comment 12 Steve Dickson 2005-07-06 16:52:22 UTC
Does exporting with 'no_subtree_check' help?

Comment 13 gmc 2005-07-12 06:36:08 UTC
(In reply to comment #12)
> Does exporting with 'no_subtree_check' help?

No

Comment 14 Neil Horman 2005-07-12 11:16:54 UTC
Just out of curiosity, what uid is rpc.mountd running as? This is the error that
you would receive in the event that mountd was running as a non-root user. 
Also,  can you confirm that there are no other programs running which may be
registering a mountd rpc program that are running as non-root?

Comment 15 gmc 2005-07-12 11:36:02 UTC
rpc.mountd running as uid 0
this is the output of rpcinfo (only the relevant):
    100005    1   udp    981  mountd
    100005    1   tcp    984  mountd
    100005    2   udp    981  mountd
    100005    2   tcp    984  mountd
    100005    3   udp    981  mountd
    100005    3   tcp    984  mountd


Comment 16 Steve Dickson 2005-07-27 13:13:54 UTC
hmm... I wonder if this is the same problem as in
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=164186

Comment 17 Neil Horman 2005-07-27 13:56:30 UTC
It does sound awfully close.


Comment 18 Steve Dickson 2005-07-27 14:50:57 UTC
OK, I'm going to close this bug as a duplicate of bz164186.
Please reopen if this is not the case.

*** This bug has been marked as a duplicate of 164186 ***