Bug 857555
| Summary: | nfs: fix potential slabcache leaks when cache allocations fail | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | Jeff Layton <jlayton> |
| Component: | kernel | Assignee: | Steve Dickson <steved> |
| Status: | CLOSED ERRATA | QA Contact: | Petr Beňas <pbenas> |
| Severity: | medium | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 6.4 | CC: | eguan, jcpunk, nfs-maint, pbenas, pstehlik, rwheeler, steved, yanwang |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | kernel-2.6.32-335.el6 | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2013-02-21 06:37:44 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
This request was evaluated by Red Hat Product Management for inclusion in a Red Hat Enterprise Linux release. Product Management has requested further review of this request by Red Hat Engineering, for potential inclusion in a Red Hat Enterprise Linux release for currently deployed products. This request is not yet committed for inclusion in a release. hi Jeff, Do you know any test steps/reproducer QE could use to test the patch? Thanks in advance. No. There are 2 bugs at play here and only one is easily reproducible... The first problem is that when allocating any of the caches fails, the nfs code doesn't clean up any that were already allocated. That's a long standing bug that we ought to fix, but that will be hard to reproduce without adding some sort of fault injection. The second problem is that we don't ever clean up the nfs_commit_data cache. This is easier to reproduce, but that bug is introduced in one of steved's pnfs "waves". No released kernel contains that bug (yet). Most likely steve should just take the patch I sent upstream into his pnfs waves and close this bug as a dup. (In reply to comment #5) > No. There are 2 bugs at play here and only one is easily reproducible... > > The first problem is that when allocating any of the caches fails, the nfs > code doesn't clean up any that were already allocated. That's a long > standing bug that we ought to fix, but that will be hard to reproduce > without adding some sort of fault injection. > > The second problem is that we don't ever clean up the nfs_commit_data cache. > This is easier to reproduce, but that bug is introduced in one of steved's > pnfs "waves". No released kernel contains that bug (yet). hi Jeff, Thanks for your useful info. As per the second problem you mentioned above, could you tell me how do I know the nfs_commit_data cache cleaned up when uploading nfs module? QE could cover this at least, thanks. (In reply to comment #6) > (In reply to comment #5) > > No. There are 2 bugs at play here and only one is easily reproducible... > > > > The first problem is that when allocating any of the caches fails, the nfs > > code doesn't clean up any that were already allocated. That's a long > > standing bug that we ought to fix, but that will be hard to reproduce > > without adding some sort of fault injection. > > > > The second problem is that we don't ever clean up the nfs_commit_data cache. > > This is easier to reproduce, but that bug is introduced in one of steved's > > pnfs "waves". No released kernel contains that bug (yet). > hi Jeff, > Thanks for your useful info. > As per the second problem you mentioned above, could you tell me how do I > know the nfs_commit_data cache cleaned up when uploading nfs module? QE > could cover this at least, thanks. sorry, s/uploading/unloading/ $ grep nfs_commit_data /proc/slabinfo If that cache no longer exists after unloading the module then things are working correctly. (In reply to comment #8) > $ grep nfs_commit_data /proc/slabinfo > > If that cache no longer exists after unloading the module then things are > working correctly. Thank you, Jeff! QE could write test case to cover this. Actually...I found that I could oops the kernel pretty trivially with steved's backport that didn't have this patch: # modprobe nfs # rmmod nfs # cat /proc/slabinfo ...and boom, it oopses because the string describing the slabcache lives inside of nfs.ko. Once you rmmod, that pointer is no longer good. Adding the above patch fixes the issue. Patch(es) available on kernel-2.6.32-335.el6 Reproduced in 2.6.32-330.el6.x86_64 and verified in 2.6.32-331.el6.x86_64. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHSA-2013-0496.html |
I recently pushed this patch upstream: commit 3dd4765fce04c0b4af1e0bc4c0b10f906f95fabc Author: Jeff Layton <jlayton> Date: Thu Aug 2 14:30:56 2012 -0400 nfs: tear down caches in nfs_init_writepagecache when allocation fails ...which I think we'll want in rhel6. Ideally Steve D can just pull it in as part of his pnfs DIO backport, but opening this bug to make sure it doesn't fall through the cracks.