Bug 1402297

Summary:	Enabling Quota on volumes leads inconsistent data
Product:	[Community] GlusterFS	Reporter:	PankaJ <pankaj>
Component:	quota	Assignee:	Sanoj Unnikrishnan <sunnikri>
Status:	CLOSED EOL	QA Contact:
Severity:	low	Docs Contact:
Priority:	unspecified
Version:	3.7.16	CC:	bugs, pankaj, rgowdapp
Target Milestone:	---	Keywords:	Triaged
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-03-08 11:03:57 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description PankaJ 2016-12-07 08:34:15 UTC

Description of problem:

After enabling quota, mounted data are not showing on client side. We have 8 volume of varying sizes. We are enabling quota on 3 volumes of 4TB each in sizes.
We have installed GlusterFS community version 3.7.16 on AWS. But once we mount GlusterFS Volumes to the client side with quota enabled then we are not able to get consistent data. Sometime it shows and sometime it doesn't. If shown then we are not able to create or write on the files. Please help us to resolve this problem

Version-Release number of selected component (if applicable):
3.7.16

How reproducible:
Install glusterfs on AWS and create few volumes having lots of varying size of data. Enable quota and then mount your Volumes on client. after few moment or days filesystem becomes inconsistent.

Steps to Reproduce:
1. Install GlusterFS on AWS 2 ec2-instances replicated with transport type socket
2. Now enable quota on few volumes and mount them onto client
3. After few moment it will become inconsistent.

Actual results:
Data on client side become inconsistent over the time of period.

Expected results:
It should work perfectly fine without any inconsistency in data at client side by enabling quota on volumes.

Additional info:
We keep monitoring directory based quota limit and all are under our control but still we are facing this issue. Some strange behavior regarding file system attributes. We see some duplicate quota attributes attached on our files under same volume which have quota enabled. If we clear those particular attributes by "setfattr -x " command then data become consistent. But we can't clear thousand of file manually on both glusetrfs server nodes. Please help us to resolve this issue or tell us if we have to do some explicit configuration for same. We are following GlusterFS documentation for configuration.

Comment 1 Sanoj Unnikrishnan 2016-12-14 07:02:03 UTC

Regarding Duplicate Quota Attrs:
Quota attributes are versioned, so we can have duplicate attributes belonging to different version. We always use latest version for checking/updating usage.

The cleaning up of quota attributes is a background task and can continue even after the quota disable is issued. So, If the user issues a quota enable prior to completion of this task, we may have duplicate attributes, but the new attributes will be of a higher version.

Regarding inconsistency:
Pasting comments from gluster IRC channel (Dec 7)

sanoj > What exactly is the issue when u say "not able to get consistent data"
pank_ > This means that on client side we are facing some r/rw issue on files. Also some dir and files are not showing. After cleaning quota, gfid attributes then we are able to get data on client side.

Raghavendra Could you please comment on this ^^( in particular what can cause the files to not show in listing )

Comment 2 Sanoj Unnikrishnan 2017-01-31 09:53:43 UTC


We should not be cleaning gfid under any scenario.

1) Since its been a while. Do you still hit this issue?
   Is it reproducible? with/without quota (as we don't see reason  for quota effecting file visibility). 
2) Were files created only after the volume was created (or was there any old brick being reused).
3) Is it that you have multiple volumes with / without quota and you see the issue only on quota enabled volumes?
4) Did you try a quota disable prior to manual xattr cleanup. were the files missing then?
If you are seeing this issue could u please share the logs

Comment 3 Sanoj Unnikrishnan 2017-01-31 09:56:45 UTC

*** Bug 1402296 has been marked as a duplicate of this bug. ***

Comment 4 Kaushal 2017-03-08 11:03:57 UTC

This bug is getting closed because GlusteFS-3.7 has reached its end-of-life.

Note: This bug is being closed using a script. No verification has been performed to check if it still exists on newer releases of GlusterFS.
If this bug still exists in newer GlusterFS releases, please reopen this bug against the newer release.

Comment 5 PankaJ 2020-04-30 06:08:00 UTC

This has been closed due to EOL.
No further action required.