Bug 613437 - RDMA/OpenIB shutdown hangs if using NFSoRDMA
Summary: RDMA/OpenIB shutdown hangs if using NFSoRDMA
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: rdma
Version: 13
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Doug Ledford
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-07-11 15:32 UTC by David Dillow
Modified: 2011-05-02 16:49 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-05-02 16:49:47 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
Patch to shut down and unload NFSoRDMA modules during RDMA stop (1.03 KB, patch)
2010-07-11 15:32 UTC, David Dillow
no flags Details | Diff

Description David Dillow 2010-07-11 15:32:02 UTC
Created attachment 431006 [details]
Patch to shut down and unload NFSoRDMA modules during RDMA stop

Description of problem:
If using NFSoRDMA, the reboot will hang while trying to stop the rdma service. This is due to not unloading the modules for NFSoRDMA prior to trying to unload the rest of the stack. It doesn't matter if the use of NFSoRDMA is as a client or as a server.


Version-Release number of selected component (if applicable):
1.0-7.fc13
1.0-7.fc12

How reproducible:
Always

The attached patch fixes this issue for me during shutdown.

Comment 1 David Dillow 2010-07-11 16:27:31 UTC
Well, it is a partial fix -- I thought I had tested the patch with clients mounted, but it appears I did not. In that case, I still get the hangs, as svcrdma/xprtrdma do not unload.

So, more work to be done here.

Comment 2 Doug Ledford 2010-08-03 05:01:46 UTC
I'm creating a new service, nfs-rdma, that's intended to be started after nfs
and stopped before nfs that enables and disables rdma support.  The rdma init
script now checks to make sure that nfs-rdma support is disabled before downing
the rdma service.  If the nfs-rdma service fails to stop (because things are in
use) then you can down the nfs service and then down the nfs-rdma service
(which is backward order, but should work as downing the nfs service will free
up the kernel modules to be unloaded).  Will be present in rdma-1.0-8 or later.

Comment 3 David Dillow 2010-08-03 14:30:43 UTC
That will let me remove my modifications to get NFSoRDMA starting, but I don't think it is going to help when stopping the rdma service -- I've looked around a bit, but I don't see a way to forcibly disconnect clients from the NFS server, which keeps the use count non-zero on the nfsd and svcrdma modules, preventing their removal. There may be a way; I just haven't found it yet -- or it may be a different issue than I think.

I've also seen a similar problem with the NFS client, as even after unmounting all of the NFS filesystems, I still cannot rmmod xprtrdma.

I'm starting to wonder if the best workaround at the moment is to see if the scripts can detect we're going to runlevel 0, 1, or 6 and just not try to remove the modules.

Comment 4 Doug Ledford 2011-05-02 16:49:47 UTC
Fixed long ago but bug was not autoclosed.


Note You need to log in before you can comment on or make changes to this bug.