Is anything else from the same server already mounted on that client when the
problem happens? If so, try adding "nosharecache" to the mount options.

Yes, other shares from the same server are mounted. I tried mounting with "-o
nosharecache" and it worked!

But we are trying to maintain a homogeneous environment here, so it would be
good if- after some update in the future- we were able to mount without this
additional option (that we do not need somewhere else) again.

This problem is caused by different options being used in the mounts.
What mount options are being used for the mount points that are having problems?

Yes, can you please post all the mount options that are being used
for each mount? 

[root@ls3163 mnt]# mount
/dev/mapper/VolGroup00-LogVol00 on / type ext3 (rw,noatime)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/sda1 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
uxadmin:/vol/vol0/q_home1 on /net/sapmnt.home1 type nfs
(rw,nolock,hard,intr,addr=10.17.120.40)
uxadmin:/vol/vol0/q_home2 on /net/sapmnt.home2 type nfs
(rw,nolock,hard,intr,addr=10.17.120.40)

[root@ls3163 mnt]# mount uxadmin:/vol/vol0/q_global_all_sun global_all/
mount.nfs: /mnt/global_all is already mounted or busy

[root@ls3163 mnt]# mount -o hard,intr uxadmin:/vol/vol0/q_global_all_sun global_all/
mount.nfs: /mnt/global_all is already mounted or busy

[root@ls3163 mnt]# mount -o nolock uxadmin:/vol/vol0/q_global_all_sun global_all/
mount.nfs: /mnt/global_all is already mounted or busy

[root@ls3163 mnt]# mount -o nolock,hard,intr uxadmin:/vol/vol0/q_global_all_sun
global_all/
[works!]

[root@ls3163 mnt]# mount -o nosharecache uxadmin:/vol/vol0/q_global_all_sun
global_all/
[also works]

These are the starting flags:
    (nolock,hard,intr)

mount -o hard,intr uxadmin:/vol/vol0/q_global_all_sun
fails because it the same server an the flags
are different (nolock,hard,intr) != (hard,intr)

mount -o nolock uxadmin:/vol/vol0/q_global_all_sun
fails for the same reason server == server but
(nolock,hard,intr) != (nolock)

mount -o nolock,hard,intr uxadmin:/vol/vol0/q_global_all_sun
success because (nolock,hard,intr) == (nolock,hard,intr)

mount -o nosharecache uxadmin:/vol/vol0/q_global_all_sun
success because the kernel uses a completely different 
data structure (i.e. super_block) to represent the mount
elimination any crossover.

That this check does, is ensure you are getting what you
ask for. Note the second mount of /vol/vol0/q_global_all_sun
still had nolock on it as an option. 

So, although this is a pain, I do thing its the right
thing to do.

Okay, you are right, but our problem is actually another one:

All our mounts from uxadmin have the same mount options (hard,intr,nolock) but
some mountpoints are mounted rw (like home1 or home2) and some should really be
mounted ro (like q_global_all_sun).

So this requires us to use nosharecache for either the ro or the rw mounts,
right? Considering that we got some hundreds of these mounts in /etc/mount.map
(with around typically 1-30 mounted at the same time), this is a quite ugly
thing to do by hand, especially because we did not run into this problem on any
other Linux machine.

Wouldn't it be better to provide some sort of autodetection or keep several
splitted caches for different groups of mounts that have the same mount options?

> So this requires us to use nosharecache for either the ro or the rw mounts,
> right?
Correct.
 
> Considering that we got some hundreds of these mounts in /etc/mount.map
> (with around typically 1-30 mounted at the same time), this is a quite ugly
> thing to do by hand, especially because we did not run into this problem on any
> other Linux machine.
I see your point, but what you thought was happen with ro/rw were not happening
with RHEL5. Filesystem you thought were rw were actually ro and visa versa. 
so in a sense you filesystem will be mounted in a more accurate  way.

> Wouldn't it be better to provide some sort of autodetection or keep several
> splitted caches for different groups of mounts that have the same mount 
> options?
Not sure... I'm open for suggestions, but auto-detecting how a mount
should could be quite complicated fairly quickly. 
  

> I see your point, but what you thought was happen with ro/rw were not happening
> with RHEL5. Filesystem you thought were rw were actually ro and visa versa. 
> so in a sense you filesystem will be mounted in a more accurate  way.

Okay, I justed tested it again and you are completely right: After the first
mount created the cache for the filesystem on the server, each successive mount
uses the same mount options without producing any kind of warning, notice or
error during the mount (tested on SLES9).

I think the ideal solution would be a detection algorithm in the kernel that
checks if two mounts on the same server are independent and then creates
separate caches but I understand that this is a radical change compared to the
way it works now and very hard to implement.

But may I propose that the error message is changed?

Instead of "mount.nfs: foobar is already mounted or busy" it should read
"mount.nfs: A NFS export fooserver is already mounted with different mount
options. Please make your checks and change the mount options or add
nosharecache." or the like.

The check for this could be done completely in userland by parsing /proc/mounts
if the kernel does not provide detailed information about the NFS mount error.

If this works, it could be possible to add a warning message for older system
(like RHEL5), something like: "NFS export foobar could not be mounted
ro,hard,intr. Mounted rw,hard,intr instead."

> I think the ideal solution would be a detection algorithm in the kernel that
> checks if two mounts on the same server are independent and then creates
> separate caches but I understand that this is a radical change compared to the
> way it works now and very hard to implement.
Right, with regard to hard implement which in turn makes it hard to
get it right for everybody... 

> But may I propose that the error message is changed?
Yeah, let me see what I can do about this... 

Hello,

I'm reviewing this bug as part of the kernel bug triage project, an attempt to
isolate current bugs in the Fedora kernel.

http://fedoraproject.org/wiki/KernelBugTriage

I am CC'ing myself to this bug and will try and assist you in resolving it if I can.

There hasn't been much activity on this bug for a while. Could you tell me if
you are still having problems with the latest kernel?

If the problem no longer exists then please close this bug or I'll do so in a
few days if there is no additional information lodged.

I see there is no easy way to fix this problem in kernel. 
Still, the problem is still there, and I'd like to know, if you are willing to 
change the error message at least (see comment #10 from Steve Dickson).

Well, it appears EBUSY, which is being  passed up from the kernel and
causing the error message will be staying, at least for the near future.

Created attachment 291468 [details]
Update NFS failed mount error message

This option changes the error message to be more informative/helpful:

			break;
		case EBUSY:
-			fprintf(stderr, "%s: %s is already mounted or busy\n",
progname, node);
-			break;
+			fprintf(stderr,
+				"%s: A NFS exported server is already "
+				"with different mount options.\n"
+				"  Check and change mount options "
+				"or add nosharecache.\n",
+				progname, mount_point);
+			break;
		case ENOENT:

The attached patch needs reviewing as I am not a c coder but thought I might be
able to handle this at least. Anyway, its there if anyone is interested. The
full patch is an update against nfs-utils of:

nfs-utils-1.1.0-mount-v4-errors.patch

Cheers
Chris

Hmm, actually:

			break;
		case EBUSY:
-			fprintf(stderr, "%s: %s is already mounted or busy\n",
progname, node);
-			break;
+			fprintf(stderr,
+				"%s: %s is already mounted "
+				"with different options.\n"
+				"  Check and change mount options "
+				"or add nosharecache.\n",
+				progname, mount_point);
+			break;
		case ENOENT:

Created attachment 291469 [details]
Update NFS failed mount error message

The only problem is EBUSY may or may not mean the filesystem
is mounted if different option. More times than not, EBUSY is
returned because the filesystem is already mounted (with the 
same options)

This message is a reminder that Fedora 7 is nearing the end of life. Approximately 30 (thirty) days from now Fedora will stop maintaining and issuing updates for Fedora 7. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '7'.

Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 7's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 7 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora please change the 'version' of this bug. If you are unable to change the version, please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. If possible, it is recommended that you try the newest available Fedora distribution to see if your bug still exists.

Please read the Release Notes for the newest Fedora distribution to make sure it will meet your needs:
http://docs.fedoraproject.org/release-notes/

The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping

please re-assign to Fedora 9

Fedora 7 changed to end-of-life (EOL) status on June 13, 2008. 
Fedora 7 is no longer maintained, which means that it will not 
receive any further security or bug fix updates. As a result we 
are closing this bug. 

If you can reproduce this bug against a currently maintained version 
of Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

I've been bitten by this issue today.  Going through strace did not help too
much, lucky googling gave me link to this bug report.  Uff.

Is this documented somewhere at least?  Or is this so well known issue?   It
was like facing some magic :)

  mount-a options:  rw,context=system_u:object_r:httpd_sys_content_t:s0
  mount-b options:  rw

Well, I tried to switch mount-b options to 'rw,sync' and it worked
(sometimes?) :), having context= there also worked, having other options did
not work.

I see that there is hard to guess real reason for EBUSY, but it does not seem
to be impossible based on already existing mount points.
Well, I'll never do the same mistake again, but others may do and we should
tell them, somehow.  In the worst case, making the warning more verbose
mentioning that it "could be this, see rhbz#253530" could save some painful
hours of debugging ..

Pavel

This message is a reminder that Fedora 20 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 20. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '20'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 20 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

On F21:
mount.nfs: mount(2): Device or resource busy
mount.nfs: /mnt/XXX is busy or already mounted

I encountered this today with RHEL 7.1, very annoying.

# exportfs -avr
exporting 192.168.38.1:/var/lib/dhcpd
exporting 192.168.38.1:/etc/dhcp

The moment I try to mount the second directory it fails with the same error. Had to add "nosharecache" to be able to mount both.

Please disregard, this was because of SELinux: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Security-Enhanced_Linux/sect-Security-Enhanced_Linux-Mounting_File_Systems-Multiple_NFS_Mounts.html

This message is a reminder that Fedora 21 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 21. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '21'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 21 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Switched against kernel by accident, I guess.  Reopening.

(In reply to Pavel Raiskup from comment #28)
> Switched against kernel by accident, I guess.  Reopening.

why are you reopening this??

That's not kernel bug, as I interpret that nfs-utils should be fixed to
provide better warning (and closing it by kernel guys was just expected
mistake).

I'm not seeing this problem

server:
fedora# exportfs -arv
exporting f25:/etc/dhcp
exporting f26:/etc/dhcp
exporting f25:/var/tmp
exporting f26:/var/tmp

clients:


f25# mount fedora:/var/tmp /mnt/foo
f25# mount fedora:/etc/dhcp /mnt/bar
f26# mount fedora:/var/tmp /mnt/foo
f26# mount fedora:/etc/dhcp /mnt/bar

What am I doing wrong?

See the comment #22.  You need to mount two exports from the same server, both
with different options.  And it was hard to debug what's going on.  I reopened
because there's totally unhelpful error reporting from nfs-utils for the EBUSY
errno.