Description of problem: After rename operation on files, gluster v quota <volname> list-objects gives incorrect number of files in output Version-Release number of selected component (if applicable): glusterfs-3.7.1-7.el6rhs.x86_64 How reproducible: 100% Steps to Reproduce: 1. Create 2*2 distribute replicate volume 2. Mount volume as fuse 3. Enable quota and limit-objects 4. Create some files on mount point 5. Rename all the files from the mount point [root@darkknightrises ~]# gluster v quota vol0 limit-object / 1000 volume quota : success Output after object limit exceeds [root@darkknightrises ~]# gluster v quota vol0 list-objects Path Hard-limit Soft-limit Files Dirs Available Soft-limit exceeded? Hard-limit exceeded? --------------------------------------------------------------------------------------------------------------------------------------------- / 1000 80% 997 3 0 Yes Yes ===================================== Output after rename of file [root@darkknightrises ~]# gluster v quota vol0 list-objects Path Hard-limit Soft-limit Files Dirs Available Soft-limit exceeded? Hard-limit exceeded? --------------------------------------------------------------------------------------------------------------------------------------------- / 1000 80% 1427 3 0 Yes Yes Files on the mount point: ========================= [root@client glusterfs]# ls -ltrh | wc -l 1007 Actual results: gluster v quota vol0 list-objects show wrong number of files. Expected results: gluster v quota vol0 list-objects should show the correct number of files available on mount point Additional info: [root@darkknightrises ~]# gluster v info vol0 Volume Name: vol0 Type: Distributed-Replicate Volume ID: 1e9ab9f1-19ca-4595-9e47-ee11f8f6a171 Status: Started Number of Bricks: 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: 10.70.33.214:/rhs/brick1/b001 Brick2: 10.70.33.219:/rhs/brick1/b002 Brick3: 10.70.33.225:/rhs/brick1/b003 Brick4: 10.70.44.13:/rhs/brick1/b004 Options Reconfigured: features.quota-deem-statfs: on features.inode-quota: on features.quota: on features.scrub: Active features.bitrot: on features.uss: enable performance.readdir-ahead: on
*** Bug 1250361 has been marked as a duplicate of this bug. ***
Here are the problems with marker rename: Problem 1) 1) mq_reduce_parent_size is not handling inode-quota contribution 2) When dest files exists and IO is happening Now renaming will overwrite existing file mq_reduce_parent_size called on dest file with saved contribution, this can be a problem is IO is still happening contribution might have changed Problem 2) There is a small race between rename and in-progress write Consider below scenario 1) rename FOP invoked on file 'x' 2) write is still in progress for file 'x' 3) rename takes a lock on old-parent 4) write-update txn blocked on old-parent to acquire lock 5) in rename_cbk, contri xattrs are removed and contribution is deleted and lock is released 6) now write-update txn gets the lock and updates the wrong parent as it was holding lock on old parent so validate parent once the lock is acquired Problem 3) when a rename operation is performed, a lock is held on old parent. This lock is release before unwinding the rename operation. This can be a problem if there are in-progress writes happening during rename, where update txn can take a lock and update the old parent as inode table is not updated with new parent
https://code.engineering.redhat.com/gerrit/#/c/55711/1
Able to reproduce bugs on latest build glusterfs-3.7.1-13. Files on mount point [root@rhsqa7 fuse]# ll | wc -l 1999 list-objects before rename operations [root@darkknight ~]# gluster v quota testvol list-objects Path Hard-limit Soft-limit Files Dirs Available Soft-limit exceeded? Hard-limit exceeded? --------------------------------------------------------------------------------------------------------------------------------------------- / 2000 80% 1998 3 0 Yes Yes list-objects after rename operations [root@darkknight ~]# gluster v quota testvol list-objects Path Hard-limit Soft-limit Files Dirs Available Soft-limit exceeded? Hard-limit exceeded? --------------------------------------------------------------------------------------------------------------------------------------------- / 2000 80% 2872 3 0 Yes Yes
DHT linkto were also accounted, hecce inode-count was shown incorrect. Upstream patch 'http://review.gluster.org/#/c/12113/' fixes the issue
upstream patch: http://review.gluster.org/#/c/12113/ release-3.7 patch: http://review.gluster.org/#/c/12204/ This is fixed as part of re-base from 3.7.5
Before Rename [root@localhost newtest]# gluster v quota testvol list-object Path Hard-limit Soft-limit Files Dirs Available Soft-limit exceeded? Hard-limit exceeded? ----------------------------------------------------------------------------------------------------------------------------------------------- / 500 80%(400) 496 4 0 Yes Yes /test 1000 80%(800) 496 1 503 No No After Rename [root@localhost newtest]# gluster v quota testvol list-object Path Hard-limit Soft-limit Files Dirs Available Soft-limit exceeded? Hard-limit exceeded? ----------------------------------------------------------------------------------------------------------------------------------------------- / 500 80%(400) 496 4 0 Yes Yes /test 1000 80%(800) 496 1 503 Bug verified on build glusterfs-3.7.5-6.el7rhgs.x86_64
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-0193.html