Bug 642680

Summary: XFS: accounting of reclaimable inodes is incorrect [rhel-6.0.z]
Product: Red Hat Enterprise Linux 6 Reporter: RHEL Program Management <pm-rhel>
Component: kernelAssignee: Frantisek Hrbata <fhrbata>
Status: CLOSED ERRATA QA Contact: Boris Ranto <branto>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 6.0CC: branto, dchinner, dhoward, esandeen, jweiner, kzhang, pbenas, pm-eus, qcai, rwheeler, snagar, syeghiay
Target Milestone: rcKeywords: RHELNAK, ZStream
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: kernel-2.6.32-71.5.1.el6 Doc Type: Bug Fix
Doc Text:
Previously, accounting of reclaimable inodes did not work correctly. When an inode was reclaimed it was only deleted from the per-AG (per Allocation Group) tree. Neither the counter was decreased, nor was the parent tree's AG entry untagged properly. This caused the system to hang indefinitely. With this update, the accounting of reclaimable inodes works properly and the system remains responsive.
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-11-10 19:11:33 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 641764    
Bug Blocks:    
Attachments:
Description Flags
Systemtap test that outputs numbers of reclaimable inodes none

Description RHEL Program Management 2010-10-13 14:51:11 UTC
This bug has been copied from bug #641764 and has been proposed
to be backported to 6.0 z-stream (EUS).

Comment 2 Frantisek Hrbata 2010-10-20 06:22:35 UTC
in 2.6.32-71.5.1.el6

Comment 4 Boris Ranto 2010-10-26 12:07:21 UTC
Since it is very hard/rare to reproduce this issue using reproducer steps in #641764 I've had to use different technique.
I used systemtap to watch the number of reclaimable inodes when xfs shrinker is called while generating a read (of millions of empty files) on a xfs formatted partition.
In 2.6.32-71.el6, the number almost always grows (meaning that it lowers quite rarely by a small number (usually just 1 or so)) resulting in very high number of reclaimable nodes with long term run (tens of millions even with ram limited to 512M). When huge file (tens of gigabytes) is read, the number won't lower or will lower very insignificantly (as described previously). This could later cause the problems described in #641764.
In 2.6.32-71.6.1 (with 512M ram), the number stays within a reasonable range (up to tens of thousands) and lowers quite often (and quite significantly). Read of a huge file causes the number of reclaimable nodes to get within a thousand. Therefore the number keeps within a reasonable range even in a long term test and necessary conditions for reproduction of #641764 are not met.
Based on previous I'm moving this to verified.

Comment 5 Boris Ranto 2010-10-26 12:10:44 UTC
Created attachment 455747 [details]
Systemtap test that outputs numbers of reclaimable inodes

It'll print the number of reclaimable inodes every time xfs_reclaim_inode_shrink is called and the number is higher than the previous one. It'll finish when the number lowers.

Comment 7 errata-xmlrpc 2010-11-10 19:11:33 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2010-0842.html

Comment 8 Martin Prpič 2010-11-11 12:05:05 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Previously, accounting of reclaimable inodes did not work correctly. When an inode was reclaimed it was only deleted from the per-AG (per Allocation Group) tree. Neither the counter was decreased, nor was the parent tree's AG entry untagged properly. This caused the system to hang indefinitely. With this update, the accounting of reclaimable inodes works properly and the system remains responsive.