Bug 652786

Summary: nfs won't stop on reboot or halt
Product: [Fedora] Fedora Reporter: Gus Wirth <gwirth79>
Component: nfs-utilsAssignee: Steve Dickson <steved>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: low Docs Contact:
Priority: low    
Version: 14CC: jlayton, pspencer, sandro.bonazzola, steved
Target Milestone: ---   
Target Release: ---   
Hardware: i686   
OS: Linux   
Fixed In Version: nfs-utils-1.2.3-2.fc14 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 663045 (view as bug list) Environment:
Last Closed: 2010-12-10 15:31:24 EST Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On:    
Bug Blocks: 663045    
Description Flags
patch to fix nfs shutdown problem
patch to fix rpcidmapd shutdown problem none

Description Gus Wirth 2010-11-12 13:50:53 EST
Description of problem:

When rebooting or halting a system with the NFS service running, the NFS service won't stop.

Version-Release number of selected component (if applicable):


How reproducible:

Operate system normally with NFS services running. 

Steps to Reproduce:
1. Have NFS start either on runlevel or manual start
2. Reboot or halt system

Actual results:

NFS fails to stop. This leads to error messages that /var is in use.

Expected results:

NFS service stops. Clean reboot or halt without error messages.

Additional info:

If NFS is stopped manually before reboot/halt like so:

# service nfs stop

it will stop cleanly and show appropriate messages for various components. However, when the system is rebooting or halting no NFS messages are seen, seeming to indicate that it never shuts down.
Comment 1 Steve Dickson 2010-11-23 14:54:08 EST
what does is the output of 'ps ax | grep nfsd' after the service is stopped?
Comment 2 Gus Wirth 2010-11-23 15:29:02 EST
(In reply to comment #1)
> what does is the output of 'ps ax | grep nfsd' after the service is stopped?

Here is the result of stopping the nfs service:

[gus@falcon ~]$ sudo service nfs stop
Shutting down NFS mountd:                                  [  OK  ]
Shutting down NFS daemon:                                  [  OK  ]
Shutting down NFS quotas:                                  [  OK  ]
Shutting down NFS services:                                [  OK  ]
[gus@falcon ~]$ ps ax|grep nfs
28923 pts/7    S+     0:00 grep --color=auto nfs
[gus@falcon ~]$

As you can see, if the nfs service is stopped from the command line before the system is shut down, it stops cleanly. When the system is shut down with nfs running, I never see the "Shutting down ..." messages. It's as if the init script never gets called.

I did all my testing from run level 3 to eliminate GUI problems, and I don't have any clients connected. Because the system is shutting down, I'm not sure how to capture the console messages. I could try to set up a serial console and capture the shutdown sequence.

I also turned off almost all other services. I'll see what the bare minimum is to trigger the problem.
Comment 3 Gus Wirth 2010-11-24 16:45:14 EST
I found the bug. It exists because the subsystem lock file name does not match the init script name. The /etc/rc.d/rc script checks to see if a subsytem is running by using the name of the init script for the subsystem file. In the case of the NFS subsystem, the NFS init script creates a subsysetm file /var/lock/subsys/nfsd. It should be nfs (no d on the end).

I noticed that this subsystem check affects several other systems also, for example CUPS and rsyslog. I wrote a tiny little program to check the subsystem status (derived from /etc/rc.d/rc):

# check if a subsystem is running based on what needs to be killed

# First, run the KILL scripts.
for i in /etc/rc$runlevel.d/K* ; do

	# Check if the subsystem is already up.
	echo "Checking for $subsys"
	[ -f /var/lock/subsys/$subsys ] || [ -f /var/lock/subsys/$subsys.init ] || echo "$subsys does not exist"


Patch for /etc/init.d/nfs:

--- nfs.orig    2010-11-24 10:24:15.191119091 -0800
+++ nfs 2010-11-24 13:26:52.369559897 -0800
@@ -124,7 +124,7 @@
        touch /var/lock/subsys/rpc.mountd
-       touch /var/lock/subsys/nfsd
+       touch /var/lock/subsys/nfs
        # Let rpc.idmapd know that rpc.mountd just started
        [ -x /usr/sbin/rpc.idmapd ] && /sbin/service rpcidmapd condstart
Comment 4 Gus Wirth 2010-11-24 18:25:07 EST
Created attachment 462775 [details]
patch to fix nfs shutdown problem
Comment 5 Gus Wirth 2010-11-24 18:27:32 EST
I missed a few places. The attached patch gets all the remaining nfs vs nfsd substitutions.
Comment 6 Steve Dickson 2010-12-01 12:07:35 EST
*** Bug 655726 has been marked as a duplicate of this bug. ***
Comment 7 Fedora Update System 2010-12-02 08:56:00 EST
nfs-utils-1.2.3-2.fc14 has been submitted as an update for Fedora 14.
Comment 8 Fedora Update System 2010-12-02 14:17:19 EST
nfs-utils-1.2.3-2.fc14 has been pushed to the Fedora 14 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update nfs-utils'.  You can provide feedback for this update here: https://admin.fedoraproject.org/updates/nfs-utils-1.2.3-2.fc14
Comment 9 Gus Wirth 2010-12-02 17:06:21 EST
There is a similar problem in rpc.idmapd which is also part of nfs-utils. Simple patch attached to fix.
Comment 10 Gus Wirth 2010-12-02 17:08:09 EST
Created attachment 464390 [details]
patch to fix rpcidmapd shutdown problem
Comment 11 Fedora Update System 2010-12-10 15:31:15 EST
nfs-utils-1.2.3-2.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 12 Philip Spencer 2011-01-24 10:54:27 EST
This affects nfs-utils-1.2.2-6.fc13 in Fedora 13 too.

Can an update for Fedora 13 please be pushed as well? This bug often prevents a server from being remotely rebootable, as the failure of NFS to come down cleanly during the reboot process causes files to be left open and the system locks up trying to unmount them.