Description of problem: On a four node samba ctdb setup enable quota and set the quota limit. When we do a windows mount it shows disc size full and does not allow to right any data to it where as the share is completely empty without any data present in it. gluster volume quota volname list is showing used space as 16384.0PB. Even on cifs mount right is not possible because of the disc size full issue. Version-Release number of selected component (if applicable): samba-4.4.6-2.el7rhgs.x86_64 glusterfs-fuse-3.8.4-5.el7rhgs.x86_64 How reproducible: 1/1 Steps to Reproduce: 1.Enable quota on a samba-ctdb setup 2.Provide quota limit 3.Mount share on windows and check properties 4.Try to write data any data to it Actual results: Disc size is full and does not allow any data Expected results: Should show proper info and should not block IO Additional info:
Moving on, i disable and enable quota after removing all quota limitation. Now when i set quota limitation again on root share & subfolder. The "gluster volume quota VOLNAME list" command showed no data i.e NA in all the sections (soft limit, hard limit, used etc) of the list. After using the setup for almost 15minutes that is creating files in the share and doing basic operation the "gluster volume quota VOLNAME list" command started showing correct data and is working fine for now. So somewhere the quota accounting is getting messed up. Sanoj can you please look in to this and provide information regarding how did it showed 16384 PB of used space in first place.
As we saw that initial bits of trusted.glusterfs.quota.size xattr on the bricks were FFFFFFFFFFF, Its evident that the accounting is wrong. The remaining bits representing file count/dir count were correct. We need to know if the issue is with quota crawler (which populates xattrs initially) or the xattr update or was there a race between these two? Since Quota disable followed by enable resolved the issue - looks like crawler alone may not have led to the issue. To RCA further, we need to know, 1) Was there any io in progess while quota was being enabled ? 2) Was there any operations done between step 1 and 2 ?
(In reply to Sanoj Unnikrishnan from comment #5) > As we saw that initial bits of trusted.glusterfs.quota.size xattr on the > bricks were FFFFFFFFFFF, Its evident that the accounting is wrong. The > remaining bits representing file count/dir count were correct. > We need to know if the issue is with quota crawler (which populates xattrs > initially) or the xattr update or was there a race between these two? > Since Quota disable followed by enable resolved the issue - looks like > crawler alone may not have led to the issue. > To RCA further, we need to know, > 1) Was there any io in progess while quota was being enabled ? No there was no IO in progress. > 2) Was there any operations done between step 1 and 2 ? No operations were performed between step1 & step2. There was just 1 empty directory present in the share way before enabling quota.
We don't have a clear RCA or reproducer yet for this.