Version-Release number of selected component (if applicable): mainline How reproducible: often Steps to Reproduce: 1.create a volume volume1 with one brick1 2.start the volume1 3.mount from the client.(glusterfs, nfs mounts) 4.execute: dd if=/dev/urandom of=file1 bs=1M count=1024 5.while dd in progress, replace-brick brick1 with brick2 6.replace-brick should be successful 7.add-brick <brick1> to the same volume 'volume1' 8.add-brick successful 9.from nfs mount perform 'ls' Lists duplicate entries of the same file. 'file1' is present on both bricks with same gfid.The file size differs on the bricks. Actual results: Client output on NFS Mount:- ---------------------------- [root@Shwetha-Laptop nfsc1]# ls -lai total 2097164 1 drwxr-xr-x. 3 root root 82 Feb 24 2012 . 1835009 drwxr-xr-x. 13 root root 4096 Feb 23 17:01 .. 4031882641594738110 -rwxrwxrwx. 1 root root 1073741824 Feb 24 2012 file1 4031882641594738110 -rwxrwxrwx. 1 root root 1073741824 Feb 24 2012 file1 [root@Shwetha-Laptop nfsc1]# ls -la total 622604 drwxr-xr-x. 3 root root 82 Feb 24 2012 . drwxr-xr-x. 13 root root 4096 Feb 23 17:01 .. -rw-r--r--. 1 root root 318767104 Feb 24 2012 file1 -rw-r--r--. 1 root root 318767104 Feb 24 2012 file1 -rwxrwxrwx. 1 root root 0 Feb 24 2012 file2 Client output on FUSE Mount:- ------------------------------ [root@Shwetha-Laptop c1]# ls -lia total 1048588 1 drwxr-xr-x. 3 root root 35 Feb 24 2012 . 1835009 drwxr-xr-x. 13 root root 4096 Feb 23 17:01 .. 13719461539811947575 -rwxrwxrwx. 1 root root 1073741835 Feb 24 2012 file1 Expected results: Should list only one entry of <file1> Additional info: Server1:- --------- [root@APP-SERVER1 ~]# getfattr -R -d -m . -e hex /export1/ getfattr: Removing leading '/' from absolute path names # file: export1/ trusted.afr.dist-io-threads=0x000000000000000000000000 trusted.afr.dist-replace-brick=0x000000000000000000000000 trusted.gfid=0x00000000000000000000000000000001 trusted.glusterfs.dht=0x00000001000000007fffffffffffffff trusted.glusterfs.volume-id=0x5cbb49f6c6bf42caaa398ac9cad1bc3a # file: export1//.glusterfs/00/00/00000000-0000-0000-0000-000000000001 trusted.afr.dist-io-threads=0x000000000000000000000000 trusted.afr.dist-replace-brick=0x000000000000000000000000 trusted.gfid=0x00000000000000000000000000000001 trusted.glusterfs.dht=0x00000001000000007fffffffffffffff trusted.glusterfs.volume-id=0x5cbb49f6c6bf42caaa398ac9cad1bc3a # file: export1//.glusterfs/0e/b6/0eb648b3-b42c-4d7e-be65-5194e61ff437 trusted.afr.dist-io-threads=0x000000000000000000000000 trusted.afr.dist-replace-brick=0x000000000000000000000000 trusted.gfid=0x0eb648b3b42c4d7ebe655194e61ff437 # file: export1//.glusterfs/6b/32/6b32f2dc-79d6-4569-8d01-7ddd2e4ae2eb trusted.gfid=0x6b32f2dc79d645698d017ddd2e4ae2eb # file: export1//file1 trusted.afr.dist-io-threads=0x000000000000000000000000 trusted.afr.dist-replace-brick=0x000000000000000000000000 trusted.gfid=0x0eb648b3b42c4d7ebe655194e61ff437 # file: export1//file2 trusted.gfid=0x6b32f2dc79d645698d017ddd2e4ae2eb [root@APP-SERVER1 ~]# ls -lh /export1/file1 -rw-r--r-- 2 root root 304M Feb 24 22:21 /export1/file1 Server2:- --------- [root@APP-SERVER2 ~]# getfattr -m . -d -e hex /export1/file1 getfattr: Removing leading '/' from absolute path names # file: export1/file1 trusted.afr.dist-io-threads=0x000000000000000000000000 trusted.afr.dist-replace-brick=0x000000000000000000000000 trusted.gfid=0x0eb648b3b42c4d7ebe655194e61ff437 [root@APP-SERVER2 ~]# getfattr -m . -d -e hex /export1 getfattr: Removing leading '/' from absolute path names # file: export1 trusted.afr.dist-io-threads=0x000000000000000000000000 trusted.afr.dist-replace-brick=0x000000000000000000000000 trusted.gfid=0x00000000000000000000000000000001 trusted.glusterfs.dht=0x0000000100000000000000007ffffffe trusted.glusterfs.volume-id=0x5cbb49f6c6bf42caaa398ac9cad1bc3a [root@APP-SERVER2 ~]# getfattr -m . -d -e hex /export1/ getfattr: Removing leading '/' from absolute path names # file: export1/ trusted.afr.dist-io-threads=0x000000000000000000000000 trusted.afr.dist-replace-brick=0x000000000000000000000000 trusted.gfid=0x00000000000000000000000000000001 trusted.glusterfs.dht=0x0000000100000000000000007ffffffe trusted.glusterfs.volume-id=0x5cbb49f6c6bf42caaa398ac9cad1bc3a [root@APP-SERVER2 ~]# getfattr -m . -d -e hex /export1/file1 getfattr: Removing leading '/' from absolute path names # file: export1/file1 trusted.afr.dist-io-threads=0x000000000000000000000000 trusted.afr.dist-replace-brick=0x000000000000000000000000 trusted.gfid=0x0eb648b3b42c4d7ebe655194e61ff437 [root@APP-SERVER2 ~]# ls -lh /export1/file1 -rwxrwxrwx 2 root root 1.0G Feb 24 23:16 /export1/file1
This used to happen before 3.3 release, because then, adding a used brick was permitted. Now(3.3+), add-brick does not allow previously used bricks to be added.