Bug 1447269

Summary:	Duplicate files in same folder
Product:	[Community] GlusterFS	Reporter:	bart.denotte
Component:	distribute	Assignee:	Nithya Balachandran <nbalacha>
Status:	CLOSED EOL	QA Contact:
Severity:	high	Docs Contact:
Priority:	high
Version:	3.8	CC:	anraj, bugs, nbalacha
Target Milestone:	---	Keywords:	Triaged
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-11-07 10:37:57 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description bart.denotte 2017-05-02 09:22:48 UTC

Hello,


I'm currently using glusterfs-server-3.7.5-1.el6 on centos 6.
I have a distributed setup with 4 nodes. We use posix acl. I recently added node 4 to the environment (was almost full) and did a fix-layout after adding it.

In certain folders I can create two files with the same name. 
If i copy an already existing file from local storage into that specific location on gluster volume, it gets two entries. If I do the same action to a parent folder i don't see this behaviour anymore.

One file contains the special T flag and has no data.

-rw-r----T+  1 root    root       0 May  2 09:40 testfile
-rw-r-----+  1 root    root      47 May  2 09:40 testfile


Both files exist at two bricks, one with data one without data.
The getfattr (getfattr -m. -d) output from the file on both bricks is:


- File with data
security.selinux="unconfined_u:object_r:file_t:s0"
system.posix_acl_access=0sAgAAAAEABgD/////AgAHANcxhW8EAAcA/////wgAAwCJAwAAEAAEAP////8gAAAA/////w==
trusted.SGI_ACL_FILE=0sAAAABgAAAAH/////AAYAAAAAAAJvhTHXAAcAAAAAAAT/////AAcAAAAAAAgAAAOJAAMAAAAAABD/////AAQAAAAAACD/////AAAAAA==
trusted.bit-rot.version=0sAgAAAAAAAABY+yYpAAvC+w==
trusted.gfid=0stCuARz7zTXWTBi7bAsE9Vg==

- File without data
security.selinux="unconfined_u:object_r:file_t:s0"
system.posix_acl_access=0sAgAAAAEABgD/////AgAHANcxhW8EAAcA/////wgAAwCJAwAAEAAEAP////8gAAAA/////w==
trusted.SGI_ACL_FILE=0sAAAABgAAAAH/////AAYAAAAAAAJvhTHXAAcAAAAAAAT/////AAcAAAAAAAgAAAOJAAMAAAAAABD/////AAQAAAAAACD/////AAAAAA==
trusted.gfid=0stCuARz7zTXWTBi7bAsE9Vg==
trusted.glusterfs.dht.linkto="ams03data02-client-3"

The getfattr output from the dir containing the files on both bricks is:

- Folder with file containing data
security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a66696c655f743a733000
system.posix_acl_access=0x0200000001000700ffffffff02000700d731856f04000700ffffffff080003008903000010000500ffffffff20000000ffffffff
system.posix_acl_default=0x0200000001000700ffffffff02000700d731856f04000700ffffffff080003008903000010000700ffffffff20000000ffffffff
trusted.SGI_ACL_DEFAULT=0x0000000600000001ffffffff00070000000000026f8531d70007000000000004ffffffff0007000000000008000003890003000000000010ffffffff0007000000000020ffffffff00000000
trusted.SGI_ACL_FILE=0x0000000600000001ffffffff00070000000000026f8531d70007000000000004ffffffff0007000000000008000003890003000000000010ffffffff0005000000000020ffffffff00000000
trusted.gfid=0x496c62dfc615430c985299b7bc7d36fe
trusted.glusterfs.dht=0x00000001000000007fa11578bfac6033

- Folder with empty file
security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a66696c655f743a733000
trusted.gfid=0x496c62dfc615430c985299b7bc7d36fe
trusted.glusterfs.dht=0x0000000100000000bfac6034ffffffff

If any additional information is required, please let me know.

Thanks in advance
Bart

Comment 1 bart.denotte 2017-05-05 08:27:06 UTC

Hi,

Apparently the ACL rights got mixed up on certain bricks for these folders. 
I did some few tests, and by making the ACL's consistent over all bricks (setfacl -m on a folder on each brick individually) made it work.

Setting the ACL's via the mount did make them consistent over all the bricks, which is strange.

Best regards
Bart

Comment 2 Nithya Balachandran 2017-05-09 17:23:52 UTC

From the IRC chat:


<bartden> Prio 1: Why did ACL rights get mixed up on individual bricks
<bartden> Prio 2: Reapplying ACL’s on dir does not work without adding a new or deleting an old

Comment 3 Niels de Vos 2017-11-07 10:37:57 UTC

This bug is getting closed because the 3.8 version is marked End-Of-Life. There will be no further updates to this version. Please open a new bug against a version that still receives bugfixes if you are still facing this issue in a more current release.