Red Hat Bugzilla – Bug 61794
cannot umount after using nfs mount to loop mount
Last modified: 2007-04-18 12:41:07 EDT
I am not sure what component to report this against.
The problem described below occured on a UPGRADE system (7.2 -> beta3). I could
NOT get it to happen EVERY TIME on a similar FRESH install system (similar
components installed) -- but it has happened.
Description of Problem:
Having mounted a directory tree as a nfs export from a 7.2 system, I mounted an
iso image in the nfs mounted tree as a loopback mount. I then did a rpm -qip
against a file on the loopback mount. I then umounted the loopback and tried to
umount the nfs mount but got "umount: /mnt/images: device busy"
Version-Release number of selected component (if applicable):
7.2 upgraded to beta3
every time I tried
Steps to Reproduce:
1. upgrade install beta3
2. nfs mount (ro) a directory exported on a 7.2 system with iso images on it
3. loop mount (ro) an iso image
4. do rpm -qip on some package on the loop mounted tree
5. umount the loop mount
6. umount the nfs mount
umount ... device busy
nfs mount umounted.
On the fresh install system, nautilus started for the loop mounted iso9660 tree
BUT on the upgraded system it DID NOT start. I killed magicdev in case that was
I am suspicious of nautilus.
There are three possible reasons I know of why a filesystem is considered to be
1. A process is using it (either current directory or an open file).
'/sbin/fuser -muv /the/mount/point' run as root will tell you what that process is.
2. Another filesystem is mounted in a subdirectory of the filesystem
3. The filesystem is being exported by knfsd (not the case here).
Please let me know which the problem is.
1. The sequence of commands are the only ones executed before the last umount
2. I tried logging off and then logging in as root but still cannot umount.
3. running /sbin/fuser -muv /mnt/images
produced no output.
4. df shows only /, /dev/shm, and /mnt/images (nfs mount)
5. /etc/exports is a null file.
Run 'mount' or 'cat /proc/mounts' to show all mounted filesystems, just in case.
But, I can't reproduce the problem here with a beta3-ish system. I need evidence
that the problem isn't just an oversight on your end.
I am going to put something out on the public beta mailing list to see if anyone
else sees this problem.
Created attachment 50005 [details]
output from mount on the fresh-install system
Created attachment 50006 [details]
output from mount on the 7.2->beta3 install system
Sorry but this is a bug. See testers mailing list.
I was FINALLY able to get rid of the mount by going to single user mode --
I have been playing with this some more. Sometimes it happens and then
sometimes it does not.
I get get the mount to go away by switching to telinit 1 or telinit 2. During
the switch, the first message from nfs says that "umount2 failed because devise
busy" and is immediately followed by a message from "mount" that "/mnt/images
This is followed by a message that nfs (retry) OK. But at the end of the
switch, there are three more nfs (retry) OK messages.
Any suggestions on determining what this is will be helpful.
If it is a bug, I need information on reproducing it on my system. So far that
hasn't been possible :(
OK, I can get this problem to occur on two separate systems -- one a fresh
install and the other a 7.2->beta3 upgrade. I will attach the install and
upgrade logs for the two installs, the anaconda-ks.cfg files from the two
systems, and strace output from the failing umount for each system.
I was hoping you could have me do something to help isolate the problem. I
expect it is something strange since I assume that redhat uses nfs a lot.
Please note that it is only after I access some file on the loop mounted
filesystem (pointing to an iso file on the nfs mounted filesystem) that the
Created attachment 50964 [details]
fresh install install.log
Created attachment 50965 [details]
fresh install anaconda-ks.cfg
Created attachment 50966 [details]
fresh install strace umount
Created attachment 50967 [details]
upgrade install 7.2 install.log
Created attachment 50968 [details]
upgrade install beta3 upgrade.log
Created attachment 50969 [details]
upgrade install anaconda-ks.cfg
Created attachment 50970 [details]
upgrade install strace umount output
It's clear that the umount is failing because _something_ is still using the
filesystem, because there's no reason yet to assume that the kernel is lying :)
I think it just requires sufficient poking with fuser, etc. If going to runlevel
1 solves the problem, and you can't get fuser to produce useful information, try
stopping services/processes one by one until the umount succeeds, etc. etc.
OK, switching to run level 1 (telinit 1) does free up the device according to
"df". However, /proc/mounts still shows the mount. In fact, it shows a whole
bunch of them (I did a bunch of runlevel switching).
Executing /etc/init.d/netfs stop fails but the runlevel switch says it fails the
first time but the succeeds. However, it keeps doing retrys since there are
still entries in /proc/mounts.
I have tried stopping everything in sight with no effect. In all cases, fuser
-vamu /mnt/images shows NOTHING.
OK, the good news is that we are able to replicate this now. The bad news is
that it is a kernel bug. More good news, there's a workaround in the mean time.
Just to clarify, here are the steps to reproduce:
1) mount an NFS export
2) on your local machine, navigate to an ISO image in that NFS export
3) loopback mount this ISO (so you have loopback mounted a NFS-exported file)
4) touch a file in the loopback mount with rpm or one of its derivatives
5) unmount the loopback mount
6) attempt to unmount the NFS export
You will get a message that the device or resource is busy, but fuser shows
nothing for that filesystem.
The workaround is to just remount the loopback mount, then immediately unmount
it. You will then be able to unmount the NFS export.
jkt reproduced it, the trick seems to be to use rpm to access some files
inside the loopback-mounted ISO.
Clearly a kernel bug, SEP.
Confirmed it is still present in beta4
Something odd happened. Running under kde, I did the nfs mount, loop mount iso
image thing and everything accessed fine. I then umounted the loop mount and
remounted it but this time the kde equivalent of nautilus did not come up. I
could umount the loop mount but the nfs mount was "busy".
I then logged off and logged in as root under gnome. I remounted the loop mount
and nautilus came up (closed it) and umounted the loop mount. I could then
umount the nfs mount.
rpm -e autorun
help for the kde case ?
Ok culprit found, identified and exterminated.
Should appear in rawhide as 2.4.18-0.25