Bug 761850 (GLUSTER-118)

Summary:	Problem with renaming of files
Product:	[Community] GlusterFS	Reporter:	Raghavendra G <raghavendra>
Component:	distribute	Assignee:	Anand Avati <aavati>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	medium	Docs Contact:
Priority:	low
Version:	2.0.2	CC:	aavati, chrisw, gluster-bugs, jdarcy, raghavendra
Target Milestone:	---
Target Release:	---
Hardware:	All
OS:	Linux
URL:	http://lists.gnu.org/archive/html/gluster-devel/2009-06/msg00052.html
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Raghavendra G 2009-07-08 08:14:15 UTC

Comments from He Xiaobin <allreol>

I have tracked the file renaming problem in my configuration ( I have mentioned in http://xiaoyou.qq.com/index.php?mod=home ). It seems that the problem coming because the file has the wrong position. For example, In my configuration(distribute->client->server->lock->posix), I have two servers, on client. The problem behaved as following:
In CLIENT side:
  [root@client home] ls /mnt/glusterfs/src
  total 0
In Servre side:
  [root@server1 home] ls /tank/export/src
2
 [root@server2 home] ls /tank/export/src
1
I once thought there should be two files (1 and 2), but as you have seen, it shown nothing. If you create a file named 1, it sould be placed on sever1, and server2 for file 2. As the gluster placed file on severs by the file's name(Though I not quite sure),  I think there might too reasons invoking this problem:
  1) the hash function works in a wrong way sometimes;
  2) the sever disconneted when client was creating files.
I can provide more information if it helps. Thanks.

Comment 1 Raghavendra G 2009-07-08 11:12:06 UTC

reported by Brent A Nelson <brent.edu> on gluster-devel.
http://lists.gnu.org/archive/html/gluster-devel/2009-06/msg00052.html

At least with 2.0.2, my GlusterFS filesystems (io-threads=>distribute=>replicate=>ha=>client=>server=>io-threads=>posix-locks=>posix) seem to have a serious problem with file renaming:

root@node1:/data# mkdir test
root@node1:/data# cd test
root@node1:/data/test# cp /bin/ls .
root@node1:/data/test# mv ls ls.bak
mv: cannot move `ls' to a subdirectory of itself, `ls.bak'
root@node1:/data/test# ls -al
total 240
drwxr-xr-x 2 root root  32768 2009-06-24 10:45 .
drwxr-xr-x 4 root root  32768 2009-06-24 10:45 ..
-rwxr-xr-x 1 root root 106176 2009-06-24 10:45 ls
root@node1:/data/test# cp ls ls.bak2
root@node1:/data/test# ls -al
total 352
drwxr-xr-x 2 root root  32768 2009-06-24 10:46 .
drwxr-xr-x 4 root root  32768 2009-06-24 10:45 ..
-rwxr-xr-x 1 root root 106176 2009-06-24 10:45 ls
-rwxr-xr-x 1 root root 106176 2009-06-24 10:46 ls.bak2

This leaves residue on the underlying shares (eonly odd nodes are listed,
as the even ones are replicas and show the same contents):

root@node1:/data/test# ls -la /disk?/glusterfs/test
/disk3/glusterfs/test:
total 128
drwxr-xr-x 2 root root   4096 2009-06-24 10:46 .
drwxr-xr-x 4 root root   4096 2009-06-24 10:45 ..
-rwxr-xr-x 1 root root 106176 2009-06-24 10:46 ls.bak2

/disk4/glusterfs/test:
total 16
drwxr-xr-x 2 root root 4096 2009-06-24 10:45 .
drwxr-xr-x 4 root root 4096 2009-06-24 10:45 ..

root@node3:~# ls -la /disk?/glusterfs/test
/disk3/glusterfs/test:
total 20
drwxr-xr-x 2 root root 4096 2009-06-24 10:45 .
drwxr-xr-x 4 root root 4096 2009-06-24 10:45 ..
---------T 1 root root    0 2009-06-24 10:45 ls

/disk4/glusterfs/test:
total 16
drwxr-xr-x 2 root root 4096 2009-06-24 10:45 .
drwxr-xr-x 4 root root 4096 2009-06-24 10:45 ..

root@node5:~# ls -la /disk?/glusterfs/test
/disk3/glusterfs/test:
total 16
drwxr-xr-x 2 root root 4096 2009-06-24 10:45 .
drwxr-xr-x 4 root root 4096 2009-06-24 10:45 ..

/disk4/glusterfs/test:
total 128
drwxr-xr-x 2 root root   4096 2009-06-24 10:45 .
drwxr-xr-x 4 root root   4096 2009-06-24 10:45 ..
-rwxr-xr-x 1 root root 106176 2009-06-24 10:45 ls

root@node7:~# ls -la /disk?/glusterfs/test
/disk3/glusterfs/test:
total 16
drwxr-xr-x 2 root root 4096 2009-06-24 10:45 .
drwxr-xr-x 4 root root 4096 2009-06-24 10:45 ..

/disk4/glusterfs/test:
total 16
drwxr-xr-x 2 root root 4096 2009-06-24 10:45 .
drwxr-xr-x 4 root root 4096 2009-06-24 10:45 ..

The file I tried to rename left an extra, empty version of the file with a+T permissions, in addition to the file itself.  When I end up with these a+T files, GlusterFS does not show them, but operations can indicate their presence (e.g., you can't delete a directory containing these files from GlusterFS, as GlusterFS won't delete the file, and then the directory is not empty and can't be unlinked).

I know I have seen these a+T files in 2.0.1 and earlier, as well (I believe I mentioned them when I was doing NFS testing).

Comment 2 Jeff Darcy 2010-01-13 11:14:21 UTC

The extra copy of the file with the strange mode is a distribute-translator link file, sort of like a symbolic link, used to facilitate renaming without actually moving the data.  It's created on the file's new home subvolume (according to a hash of the new filename) with an extended attribute identifying its original home subvolume where it actually still lives.  You can examine this attribute with the following command to verify.

    getfattr -d -e text -m trusted.glusterfs.dht.linkto filename

Link files should only get created for renames, and should be invisible to users, but (as you pointed out) sometimes they're not quite invisible enough.  The real issue seems to be that the rename attempt left a link file even though it failed, which probably shouldn't be the case.

Comment 3 Anand Avati 2010-01-23 18:34:04 UTC

This bug was fixed long back (before migration to git. hence cannot provide commmit id url)

Avati