Bug 1032034

Summary: DHT :- able to create two files having same name at same directory level
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Rachana Patel <racpatel>
Component: glusterfsAssignee: Vijaikumar Mallikarjuna <vmallika>
Status: CLOSED ERRATA QA Contact: amainkar
Severity: high Docs Contact:
Priority: high    
Version: 2.1CC: grajaiya, psriniva, smohan, vagarwal, vbellur, vmallika
Target Milestone: ---Keywords: Regression, ZStream
Target Release: RHGS 2.1.2   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: glusterfs-3.4.0.53rhs Doc Type: Bug Fix
Doc Text:
Previously, when a cached sub-volume went off-line and rm -rf command was executed and subsequently if you create files with the same filenames as before, duplicate files are seen when the cached sub-volume is back on-line. With this fix, files with the same filenames cannot be created to avoid duplicate files.
Story Points: ---
Clone Of:
: 1042725 (view as bug list) Environment:
Last Closed: 2014-02-25 08:04:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1042725    

Description Rachana Patel 2013-11-19 12:27:12 UTC
Description of problem:
in case of DHT volume, able to create two files having same name at same directory level if cached sub-vol for that file is down and hashed sub-volume is up.

If cached sub-volume is down for file 'x' then 'rm -rf' operation on parent directory is deleting dht-link file of file 'x' from hashed sub volume and after that User is able to create new file having same name 'x'.

when down sub-volume comes up, User can see two files having same name.

FUSE mount shows, two files as different entity(different inode, actual size) while NFS mount shows it as same entity.(same indoe)

FUSE mount:-
[root@rhs-client22 flat]# ls -il dir1
total 1
11426505971835458909 -rw-r--r--. 1 root root   0 Nov 19 05:17 mv13
13466909972681817225 -rw-r--r--. 1 root root 597 Nov 19 02:58 mv16
 9787038189907361861 -rw-r--r--. 1 root root   0 Nov 19 05:17 mv16

NFS mount:-
[root@rhs-client22 flat-nfs]# ls -il dir1
total 0
11426505971835458909 -rw-r--r--. 1 root root 0 Nov 19 05:17 mv13
 9787038189907361861 -rw-r--r--. 1 root root 0 Nov 19 05:17 mv16
 9787038189907361861 -rw-r--r--. 1 root root 0 Nov 19 05:17 mv16


Version-Release number of selected component (if applicable):
3.4.0.44rhs-1.el6_4.x86_64

How reproducible:
most of the time(tried 4 times and got it thrice)

Steps to Reproduce:
1.create and mount DHT volume
[root@7-VM4 bricks]# gluster v status flat
Status of volume: flat
Gluster process						Port	Online	Pid
------------------------------------------------------------------------------
Brick 10.70.36.130:/rhs/brick1/f			49158	Y	1818
Brick 10.70.36.132:/rhs/brick1/f			49161	Y	13151
Brick 10.70.36.133:/rhs/brick1/f			49152	Y	24449
Brick 10.70.36.133:/rhs/brick2/f			49153	Y	10156
NFS Server on localhost					2049	Y	24463
NFS Server on 10.70.36.132				2049	Y	9385
NFS Server on 10.70.36.130				2049	Y	3954
NFS Server on 10.70.36.131				2049	Y	28110
 
There are no active volume tasks

[root@rhs-client22 flat]# mount | grep flat
10.70.36.130:/flat on /mnt/flat-nfs type nfs (rw,addr=10.70.36.130)
10.70.36.130:/flat on /mnt/flat type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072)


2. create few directories and files from mount point.
3. perform rename operation so you will have few files having hashed and cached sub volume different.

[root@rhs-client22 flat]# mkdir dir1
[root@rhs-client22 flat]# touch dir1/f{1..40}
[root@rhs-client22 flat]# for i in {1..40}; do mv  dir1/f$i dir1/mv$i; done

4. choose one such file(in over case it's dir1/mv16) and bring cached sub-volume down by killing brick process

[root@7-VM4 bricks]# gluster v status flat
Status of volume: flat
Gluster process						Port	Online	Pid
------------------------------------------------------------------------------
Brick 10.70.36.130:/rhs/brick1/f			49158	Y	1818
Brick 10.70.36.132:/rhs/brick1/f			49161	Y	13151
Brick 10.70.36.133:/rhs/brick1/f			N/A	N	23599
Brick 10.70.36.133:/rhs/brick2/f			49153	Y	10156
NFS Server on localhost					2049	Y	23764
NFS Server on 10.70.36.131				2049	Y	27559
NFS Server on 10.70.36.132				2049	Y	8846
NFS Server on 10.70.36.130				2049	Y	3358
 
There are no active volume tasks

5. from mount point  perform 'rm -rf' on parent directory of chosen file and after that create that file again

[root@rhs-client22 flat]# rm -rf dir1
rm: cannot remove `dir1': Transport endpoint is not connected
[root@rhs-client22 flat]# ls > dir1/mv16
[root@rhs-client22 flat]# ls dir1
mv16

6. bring all sub volume up by start force option and check content of directory

FUSE:-
[root@rhs-client22 flat]# ls -il dir1
total 1
11426505971835458909 -rw-r--r--. 1 root root   0 Nov 19 05:17 mv13
13466909972681817225 -rw-r--r--. 1 root root 597 Nov 19 02:58 mv16
 9787038189907361861 -rw-r--r--. 1 root root   0 Nov 19 05:17 mv16
 9895225446184983754 -rw-r--r--. 1 root root   0 Nov 19 05:17 mv21
11759662711799292840 -rw-r--r--. 1 root root   0 Nov 19 05:17 mv25
13061044463268719119 -rw-r--r--. 1 root root   0 Nov 19 05:17 mv26
12308587649156665583 -rw-r--r--. 1 root root   0 Nov 19 05:17 mv27
12790191986129576517 -rw-r--r--. 1 root root   0 Nov 19 05:17 mv30
11178764831929195927 -rw-r--r--. 1 root root   0 Nov 19 05:17 mv32
10297353596715392646 -rw-r--r--. 1 root root   0 Nov 19 05:17 mv34
10470029047883916841 -rw-r--r--. 1 root root   0 Nov 19 05:17 mv35
11924441097350016938 -rw-r--r--. 1 root root   0 Nov 19 05:17 mv36
12012688160682546453 -rw-r--r--. 1 root root   0 Nov 19 05:17 mv40
 9240225271435750434 -rw-r--r--. 1 root root   0 Nov 19 05:17 mv6
12517604682329201207 -rw-r--r--. 1 root root   0 Nov 19 05:17 mv7
13249352854442342936 -rw-r--r--. 1 root root   0 Nov 19 05:17 mv9

NFS:-
[root@rhs-client22 flat-nfs]# ls -il dir1
total 0
11426505971835458909 -rw-r--r--. 1 root root 0 Nov 19 05:17 mv13
 9787038189907361861 -rw-r--r--. 1 root root 0 Nov 19 05:17 mv16
 9787038189907361861 -rw-r--r--. 1 root root 0 Nov 19 05:17 mv16
 9895225446184983754 -rw-r--r--. 1 root root 0 Nov 19 05:17 mv21
11759662711799292840 -rw-r--r--. 1 root root 0 Nov 19 05:17 mv25
13061044463268719119 -rw-r--r--. 1 root root 0 Nov 19 05:17 mv26
12308587649156665583 -rw-r--r--. 1 root root 0 Nov 19 05:17 mv27
12790191986129576517 -rw-r--r--. 1 root root 0 Nov 19 05:17 mv30
11178764831929195927 -rw-r--r--. 1 root root 0 Nov 19 05:17 mv32
10297353596715392646 -rw-r--r--. 1 root root 0 Nov 19 05:17 mv34
10470029047883916841 -rw-r--r--. 1 root root 0 Nov 19 05:17 mv35
11924441097350016938 -rw-r--r--. 1 root root 0 Nov 19 05:17 mv36
12012688160682546453 -rw-r--r--. 1 root root 0 Nov 19 05:17 mv40
 9240225271435750434 -rw-r--r--. 1 root root 0 Nov 19 05:17 mv6
12517604682329201207 -rw-r--r--. 1 root root 0 Nov 19 05:17 mv7
13249352854442342936 -rw-r--r--. 1 root root 0 Nov 19 05:17 mv9

backend
server 2
[root@7-VM3 ~]# ls -l /rhs/brick1/f/dir1
total 4
-rw-r--r-- 2 root root 597 Nov 19 13:28 mv16


server3
[root@7-VM4 ~]# ls -l /rhs/brick1/f/dir1/mv16
-rw-r--r-- 2 root root 0 Nov 19  2013 /rhs/brick1/f/dir1/mv16



Actual results:
able to create two files with same name and NFS and FUSE mount shows different inode size

Expected results:
User should not be able to create two files having same name at same level

Additional info:

Comment 3 Vijaikumar Mallikarjuna 2013-12-12 09:50:54 UTC
When I was trying to recreate the problem mentioned in the description I found another issue:

If there is a hard link say 'mv17' exists.
Remove file 'mv16', two entries of mv16 is removed. Now If I try to remove 'mv17' I get error 'rm: cannot remove ‘mv17’: Invalid argument'

Comment 5 Vijaikumar Mallikarjuna 2013-12-30 13:12:02 UTC
Patch: https://code.engineering.redhat.com/gerrit/17921

Comment 6 Pavithra 2014-01-03 06:58:37 UTC
Please verify the edited doc text for technical accuracy.

Comment 7 Vijaikumar Mallikarjuna 2014-01-03 07:23:55 UTC
Doc-text looks good to me.

Comment 8 Rachana Patel 2014-01-06 10:51:03 UTC
verified with build 3.4.0.53rhs-1.el6_4.x86_64

not able to reproduce hence moving it to verified

Comment 10 errata-xmlrpc 2014-02-25 08:04:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2014-0208.html