Bug 1318093

Hi Oonkwee,

The error "ctx for the node ... is NULL" can be ignored.

Regarding Quota list showing wrong usage, was any rename operations performed?

There was an issue related to rename operation, where quota usage would go wrong and this is been fixed in 3.1.2.

Here is the upstream patch: http://review.gluster.org/#/c/11578/


Thanks,
Vijay

For quota error "ctx for the node ... is NULL", we have submitted a patch upstream: http://review.gluster.org/#/c/13748/2

Bug# 1318170 filed to track quota error "ctx for the node ... is NULL"

Hi Oonkwee,

There are 3 different issues mentioned in bug:

Issue-1)
Error in the brick logs "ctx for the node ... is NULL".
This is actually not an error and has no impact with quota accounting. Can be ignored.
Patch "http://review.gluster.org/#/c/13748/" will fix this error message

Issue-2)
Quota list show wrong usage:
Some how quota accounting is miscalculated, I suspect this be same issue as:
bug# 1240991


Issue-3)
You have mentioned in comment# 6
>> The client suspects there is some data corruption, hence, their application is 
>> facing issues while retrieving files.

Quota only set/gets metadata (xattrs), could ypu please provide more information what kind of data corruption is seen?

I suspect, because the usage is miscalculated to "16384.0PB", quota-enforcer may not be allowing the writes to happen.


Thanks,
Vijay

Hi Oonkwee,

Regarding quota usage showing, we have couple of issues related to quota miscalculation fixed in 3.1.2. I will find the list of patches submitted related to this and update the bug with the patches.

Please let us know if you need a workaround to correct the quota usage?


Thanks,
Vijay

Hi Oonkwee,


We have couple of fixes related to quota accounting:

Fixed in 3.1.1
    http://review.gluster.org/#/c/11863/
    http://review.gluster.org/#/c/11995/
    http://review.gluster.org/#/c/12032/
    http://review.gluster.org/#/c/11403/

Fixed in 3.1.2
    http://review.gluster.org/#/c/11578/


Thanks,
Vijay

Hi Vijay,

They are open to a workaround to the quota issue if you can get one.

Hi Oonkwee Lim,

What is the gluster version the customer is running. If they are running 3.1.0,
then there are multiple fixes went to 3.1.1 on quota size miscalculation.
Is there any plan for the customer on upgrading Gluster? If yes, then it is good to apply workaround after the upgrade.


Here is the workaround to correct quota size:
 
1) Find a directory whose size is calculated incorrectly
   In the description there is once directory which is showing wrong incorrect 
value:

Path                   Hard-limit Soft-limit   Used  Available  Soft-limit exceeded? Hard-limit exceeded? 
/1002976973               3.0TB       80%   16384.0PB   3.0TB              No                   No


2) Execute below command on all the nodes and for all the bricks
   #find /brickpath/1002976973/ -type d | xargs /usr/bin/setfattr -n trusted.glusterfs.quota.dirty -v 0x3100


3) Mount a volume with fuse and no-readdirp options
   #mount -t glusterfs -o use-readdirp=no localhost:/volname /mnt


4) send lookup on all sub-dirs of dir 1234
   #find /mnt/1002976973 -type d -exec stat {} \;


5) After completion of lookup, verify that quota usage is showing correct values
   #gluster volume quota volname list /1002976973


Thanks,
Vijay

Hi Bipin,

Workaround provided is for correcting Quota size.
Regarding data corruption, we are working on the RCA.

Thanks,
Vijay