Bug 1402297 - Enabling Quota on volumes leads inconsistent data
Summary: Enabling Quota on volumes leads inconsistent data
Keywords:
Status: CLOSED EOL
Alias: None
Product: GlusterFS
Classification: Community
Component: quota
Version: 3.7.16
Hardware: x86_64
OS: Linux
unspecified
low
Target Milestone: ---
Assignee: Sanoj Unnikrishnan
QA Contact:
URL:
Whiteboard:
: 1402296 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-12-07 08:34 UTC by PankaJ
Modified: 2020-04-30 06:08 UTC (History)
3 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2017-03-08 11:03:57 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description PankaJ 2016-12-07 08:34:15 UTC
Description of problem:

After enabling quota, mounted data are not showing on client side. We have 8 volume of varying sizes. We are enabling quota on 3 volumes of 4TB each in sizes.
We have installed GlusterFS community version 3.7.16 on AWS. But once we mount GlusterFS Volumes to the client side with quota enabled then we are not able to get consistent data. Sometime it shows and sometime it doesn't. If shown then we are not able to create or write on the files. Please help us to resolve this problem 

Version-Release number of selected component (if applicable):
3.7.16

How reproducible:
Install glusterfs on AWS and create few volumes having lots of varying size of data. Enable quota and then mount your Volumes on client. after few moment or days filesystem becomes inconsistent. 

Steps to Reproduce:
1. Install GlusterFS on AWS 2 ec2-instances replicated with transport type socket
2. Now enable quota on few volumes and mount them onto client
3. After few moment it will become inconsistent. 

Actual results:
Data on client side become inconsistent over the time of period. 

Expected results:
It should work perfectly fine without any inconsistency in data at client side by enabling quota on volumes.

Additional info:
We keep monitoring directory based quota limit and all are under our control but still we are facing this issue. Some strange behavior regarding file system attributes. We see some duplicate quota attributes attached on our files under same volume which have quota enabled. If we clear those particular attributes by "setfattr -x " command then data become consistent. But we can't clear thousand of file manually on both glusetrfs server nodes.  Please help us to resolve this issue or tell us if we have to do some explicit configuration for same. We are following GlusterFS documentation for configuration.

Comment 1 Sanoj Unnikrishnan 2016-12-14 07:02:03 UTC
Regarding Duplicate Quota Attrs:
Quota attributes are versioned, so we can have duplicate attributes belonging to different version. We always use latest version for checking/updating usage.

The cleaning up of quota attributes is a background task and can continue even after the quota disable is issued. So, If the user issues a quota enable prior to completion of this task, we may have duplicate attributes, but the new attributes will be of a higher version.

Regarding inconsistency:
Pasting comments from gluster IRC channel (Dec 7)

sanoj > What exactly is the issue when u say "not able to get consistent data"
pank_ > This means that on client side we are facing some r/rw issue on files. Also some dir and files are not showing. After cleaning quota, gfid attributes then we are able to get data on client side.

Raghavendra Could you please comment on this ^^( in particular what can cause the files to not show in listing )

Comment 2 Sanoj Unnikrishnan 2017-01-31 09:53:43 UTC

We should not be cleaning gfid under any scenario.

1) Since its been a while. Do you still hit this issue?
   Is it reproducible? with/without quota (as we don't see reason  for quota effecting file visibility). 
2) Were files created only after the volume was created (or was there any old brick being reused).
3) Is it that you have multiple volumes with / without quota and you see the issue only on quota enabled volumes?
4) Did you try a quota disable prior to manual xattr cleanup. were the files missing then?
If you are seeing this issue could u please share the logs

Comment 3 Sanoj Unnikrishnan 2017-01-31 09:56:45 UTC
*** Bug 1402296 has been marked as a duplicate of this bug. ***

Comment 4 Kaushal 2017-03-08 11:03:57 UTC
This bug is getting closed because GlusteFS-3.7 has reached its end-of-life.

Note: This bug is being closed using a script. No verification has been performed to check if it still exists on newer releases of GlusterFS.
If this bug still exists in newer GlusterFS releases, please reopen this bug against the newer release.

Comment 5 PankaJ 2020-04-30 06:08:00 UTC
This has been closed due to EOL.
No further action required.


Note You need to log in before you can comment on or make changes to this bug.