Bug 797163

Summary: ls on the nfs mount shows double entries of files after a replace-brick and add-brick operation of the same brick
Product: [Community] GlusterFS Reporter: Shwetha Panduranga <shwetha.h.panduranga>
Component: nfsAssignee: Rajesh <rajesh>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: mainlineCC: gluster-bugs, rajesh, vagarwal, vbellur
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 852574 (view as bug list) Environment:
Last Closed: 2012-11-16 01:49:56 EST Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On:    
Bug Blocks: 852574, 858483    

Description Shwetha Panduranga 2012-02-24 07:34:05 EST
Version-Release number of selected component (if applicable):
mainline

How reproducible:
often

Steps to Reproduce:
1.create a volume volume1 with one brick1
2.start the volume1
3.mount from the client.(glusterfs, nfs mounts)
4.execute: dd if=/dev/urandom of=file1 bs=1M count=1024
5.while dd in progress, replace-brick brick1 with brick2
6.replace-brick should be successful
7.add-brick <brick1> to the same volume 'volume1'
8.add-brick successful
9.from nfs mount perform 'ls'

Lists duplicate entries of the same file. 'file1' is present on both bricks with same gfid.The file size differs on the bricks. 

Actual results:
Client output on NFS Mount:-
----------------------------
[root@Shwetha-Laptop nfsc1]# ls -lai
total 2097164
                   1 drwxr-xr-x.  3 root root         82 Feb 24  2012 .
             1835009 drwxr-xr-x. 13 root root       4096 Feb 23 17:01 ..
 4031882641594738110 -rwxrwxrwx.  1 root root 1073741824 Feb 24  2012 file1
 4031882641594738110 -rwxrwxrwx.  1 root root 1073741824 Feb 24  2012 file1

[root@Shwetha-Laptop nfsc1]# ls -la
total 622604
drwxr-xr-x.  3 root root        82 Feb 24  2012 .
drwxr-xr-x. 13 root root      4096 Feb 23 17:01 ..
-rw-r--r--.  1 root root 318767104 Feb 24  2012 file1
-rw-r--r--.  1 root root 318767104 Feb 24  2012 file1
-rwxrwxrwx.  1 root root         0 Feb 24  2012 file2


Client output on FUSE Mount:-
------------------------------

[root@Shwetha-Laptop c1]# ls -lia 
total 1048588
                   1 drwxr-xr-x.  3 root root         35 Feb 24  2012 .
             1835009 drwxr-xr-x. 13 root root       4096 Feb 23 17:01 ..
13719461539811947575 -rwxrwxrwx.  1 root root 1073741835 Feb 24  2012 file1

Expected results:
Should list only one entry of <file1>

Additional info:
Server1:-
---------
[root@APP-SERVER1 ~]# getfattr -R -d -m . -e hex /export1/
getfattr: Removing leading '/' from absolute path names
# file: export1/
trusted.afr.dist-io-threads=0x000000000000000000000000
trusted.afr.dist-replace-brick=0x000000000000000000000000
trusted.gfid=0x00000000000000000000000000000001
trusted.glusterfs.dht=0x00000001000000007fffffffffffffff
trusted.glusterfs.volume-id=0x5cbb49f6c6bf42caaa398ac9cad1bc3a

# file: export1//.glusterfs/00/00/00000000-0000-0000-0000-000000000001
trusted.afr.dist-io-threads=0x000000000000000000000000
trusted.afr.dist-replace-brick=0x000000000000000000000000
trusted.gfid=0x00000000000000000000000000000001
trusted.glusterfs.dht=0x00000001000000007fffffffffffffff
trusted.glusterfs.volume-id=0x5cbb49f6c6bf42caaa398ac9cad1bc3a

# file: export1//.glusterfs/0e/b6/0eb648b3-b42c-4d7e-be65-5194e61ff437
trusted.afr.dist-io-threads=0x000000000000000000000000
trusted.afr.dist-replace-brick=0x000000000000000000000000
trusted.gfid=0x0eb648b3b42c4d7ebe655194e61ff437

# file: export1//.glusterfs/6b/32/6b32f2dc-79d6-4569-8d01-7ddd2e4ae2eb
trusted.gfid=0x6b32f2dc79d645698d017ddd2e4ae2eb

# file: export1//file1
trusted.afr.dist-io-threads=0x000000000000000000000000
trusted.afr.dist-replace-brick=0x000000000000000000000000
trusted.gfid=0x0eb648b3b42c4d7ebe655194e61ff437

# file: export1//file2
trusted.gfid=0x6b32f2dc79d645698d017ddd2e4ae2eb

[root@APP-SERVER1 ~]# ls -lh /export1/file1
-rw-r--r-- 2 root root 304M Feb 24 22:21 /export1/file1

Server2:-
---------
[root@APP-SERVER2 ~]# getfattr -m . -d -e hex /export1/file1
getfattr: Removing leading '/' from absolute path names
# file: export1/file1
trusted.afr.dist-io-threads=0x000000000000000000000000
trusted.afr.dist-replace-brick=0x000000000000000000000000
trusted.gfid=0x0eb648b3b42c4d7ebe655194e61ff437

[root@APP-SERVER2 ~]# getfattr -m . -d -e hex /export1
getfattr: Removing leading '/' from absolute path names
# file: export1
trusted.afr.dist-io-threads=0x000000000000000000000000
trusted.afr.dist-replace-brick=0x000000000000000000000000
trusted.gfid=0x00000000000000000000000000000001
trusted.glusterfs.dht=0x0000000100000000000000007ffffffe
trusted.glusterfs.volume-id=0x5cbb49f6c6bf42caaa398ac9cad1bc3a

[root@APP-SERVER2 ~]# getfattr -m . -d -e hex /export1/
getfattr: Removing leading '/' from absolute path names
# file: export1/
trusted.afr.dist-io-threads=0x000000000000000000000000
trusted.afr.dist-replace-brick=0x000000000000000000000000
trusted.gfid=0x00000000000000000000000000000001
trusted.glusterfs.dht=0x0000000100000000000000007ffffffe
trusted.glusterfs.volume-id=0x5cbb49f6c6bf42caaa398ac9cad1bc3a

[root@APP-SERVER2 ~]# getfattr -m . -d -e hex /export1/file1 
getfattr: Removing leading '/' from absolute path names
# file: export1/file1
trusted.afr.dist-io-threads=0x000000000000000000000000
trusted.afr.dist-replace-brick=0x000000000000000000000000
trusted.gfid=0x0eb648b3b42c4d7ebe655194e61ff437

[root@APP-SERVER2 ~]# ls -lh /export1/file1
-rwxrwxrwx 2 root root 1.0G Feb 24 23:16 /export1/file1
Comment 1 Rajesh 2012-11-16 01:49:56 EST
This used to happen before 3.3 release, because then, adding a used brick was permitted. Now(3.3+), add-brick does not allow previously used bricks to be added.