Description of problem: DHT : files are stored on directory which doesn't have hash range(hash layout) Version-Release number of selected component (if applicable): 3.4.0.12rhs.beta3-1.el6rhs.x86_64 How reproducible: always Steps to Reproduce: 1. create a distributed volume, start volume, FUSE mount that volume and kill one brick process [root@wall ~]# kill -9 2441 [root@wall ~]# gluster v status test Status of volume: test Gluster process Port Online Pid ------------------------------------------------------------------------------ Brick 10.70.42.158:/rhs/brick1/t1 49156 Y 420 Brick 10.70.42.246:/rhs/brick1/t1 49156 Y 22375 Brick 10.70.42.191:/rhs/brick1/t1 N/A N 2441 Brick 10.70.42.158:/rhs/brick1/t2 49157 Y 431 Brick 10.70.42.246:/rhs/brick1/t2 49157 Y 22386 Brick 10.70.42.191:/rhs/brick1/t2 49157 Y 2351 NFS Server on localhost 2049 Y 2497 NFS Server on 5dddc52f-259a-4b45-ad60-8d1a917624ce 2049 Y 17269 NFS Server on 50a95d83-f6fe-4996-9287-3005131c948b 2049 Y 531 NFS Server on cf1941b8-7d06-482b-bacc-930b5a1401f4 2049 Y 22480 There are no active volume tasks 2. from mount, create few dir and files cd /mnt/test;for i in {1..100} ; do mkdir d$i; done;touch file{1..1000}; 3.bring brick up by start force [root@wall ~]# gluster v start test force volume start: test: success [root@wall ~]# gluster v status test Status of volume: test Gluster process Port Online Pid ------------------------------------------------------------------------------ Brick 10.70.42.158:/rhs/brick1/t1 49156 Y 420 Brick 10.70.42.246:/rhs/brick1/t1 49156 Y 22375 Brick 10.70.42.191:/rhs/brick1/t1 49156 Y 2619 Brick 10.70.42.158:/rhs/brick1/t2 49157 Y 431 Brick 10.70.42.246:/rhs/brick1/t2 49157 Y 22386 Brick 10.70.42.191:/rhs/brick1/t2 49157 Y 2351 NFS Server on localhost 2049 Y 2633 NFS Server on 50a95d83-f6fe-4996-9287-3005131c948b 2049 Y 652 NFS Server on 5dddc52f-259a-4b45-ad60-8d1a917624ce 2049 Y 17347 NFS Server on cf1941b8-7d06-482b-bacc-930b5a1401f4 2049 Y 22566 There are no active volume tasks 4. from same mount, creat few files and dirs, inside any dir cd d1;for i in {1..10} ; do mkdir new$i; done;touch new{1..100}; 5. newley created files should not be present on the brick which was down previously(rebalance was not executed) verify the same [root@wall ~]# getfattr -d -m . -e hex /rhs/brick1/t1/d1 getfattr: Removing leading '/' from absolute path names # file: rhs/brick1/t1/d1 trusted.gfid=0x9cbebcdc8a5a4bc89fa0b1284b84723d [root@wall ~]# ls -l /rhs/brick1/t1/d1 total 0 drwxr-xr-x 2 root root 6 Jul 7 07:33 new1 drwxr-xr-x 2 root root 6 Jul 7 07:33 new10 -rw-r--r-- 2 root root 0 Jul 7 07:33 new100 -rw-r--r-- 2 root root 0 Jul 7 07:33 new11 -rw-r--r-- 2 root root 0 Jul 7 07:33 new12 -rw-r--r-- 2 root root 0 Jul 7 07:33 new13 -rw-r--r-- 2 root root 0 Jul 7 07:33 new15 -rw-r--r-- 2 root root 0 Jul 7 07:33 new16 -rw-r--r-- 2 root root 0 Jul 7 07:33 new17 -rw-r--r-- 2 root root 0 Jul 7 07:33 new18 -rw-r--r-- 2 root root 0 Jul 7 07:33 new19 drwxr-xr-x 2 root root 6 Jul 7 07:33 new2 -rw-r--r-- 2 root root 0 Jul 7 07:33 new20 -rw-r--r-- 2 root root 0 Jul 7 07:33 new21 -rw-r--r-- 2 root root 0 Jul 7 07:33 new22 -rw-r--r-- 2 root root 0 Jul 7 07:33 new23 -rw-r--r-- 2 root root 0 Jul 7 07:33 new25 -rw-r--r-- 2 root root 0 Jul 7 07:33 new26 -rw-r--r-- 2 root root 0 Jul 7 07:33 new27 -rw-r--r-- 2 root root 0 Jul 7 07:33 new28 -rw-r--r-- 2 root root 0 Jul 7 07:33 new29 drwxr-xr-x 2 root root 6 Jul 7 07:33 new3 -rw-r--r-- 2 root root 0 Jul 7 07:33 new31 -rw-r--r-- 2 root root 0 Jul 7 07:33 new32 -rw-r--r-- 2 root root 0 Jul 7 07:33 new33 -rw-r--r-- 2 root root 0 Jul 7 07:33 new34 -rw-r--r-- 2 root root 0 Jul 7 07:33 new35 -rw-r--r-- 2 root root 0 Jul 7 07:33 new36 -rw-r--r-- 2 root root 0 Jul 7 07:33 new37 -rw-r--r-- 2 root root 0 Jul 7 07:33 new38 -rw-r--r-- 2 root root 0 Jul 7 07:33 new39 drwxr-xr-x 2 root root 6 Jul 7 07:33 new4 -rw-r--r-- 2 root root 0 Jul 7 07:33 new40 -rw-r--r-- 2 root root 0 Jul 7 07:33 new41 -rw-r--r-- 2 root root 0 Jul 7 07:33 new42 -rw-r--r-- 2 root root 0 Jul 7 07:33 new45 -rw-r--r-- 2 root root 0 Jul 7 07:33 new47 -rw-r--r-- 2 root root 0 Jul 7 07:33 new48 drwxr-xr-x 2 root root 6 Jul 7 07:33 new5 -rw-r--r-- 2 root root 0 Jul 7 07:33 new50 -rw-r--r-- 2 root root 0 Jul 7 07:33 new52 -rw-r--r-- 2 root root 0 Jul 7 07:33 new53 -rw-r--r-- 2 root root 0 Jul 7 07:33 new54 -rw-r--r-- 2 root root 0 Jul 7 07:33 new55 -rw-r--r-- 2 root root 0 Jul 7 07:33 new56 -rw-r--r-- 2 root root 0 Jul 7 07:33 new57 -rw-r--r-- 2 root root 0 Jul 7 07:33 new59 drwxr-xr-x 2 root root 6 Jul 7 07:33 new6 -rw-r--r-- 2 root root 0 Jul 7 07:33 new60 -rw-r--r-- 2 root root 0 Jul 7 07:33 new61 -rw-r--r-- 2 root root 0 Jul 7 07:33 new62 -rw-r--r-- 2 root root 0 Jul 7 07:33 new63 -rw-r--r-- 2 root root 0 Jul 7 07:33 new64 -rw-r--r-- 2 root root 0 Jul 7 07:33 new65 -rw-r--r-- 2 root root 0 Jul 7 07:33 new66 -rw-r--r-- 2 root root 0 Jul 7 07:33 new67 -rw-r--r-- 2 root root 0 Jul 7 07:33 new69 drwxr-xr-x 2 root root 6 Jul 7 07:33 new7 -rw-r--r-- 2 root root 0 Jul 7 07:33 new70 -rw-r--r-- 2 root root 0 Jul 7 07:33 new71 -rw-r--r-- 2 root root 0 Jul 7 07:33 new72 -rw-r--r-- 2 root root 0 Jul 7 07:33 new73 -rw-r--r-- 2 root root 0 Jul 7 07:33 new74 -rw-r--r-- 2 root root 0 Jul 7 07:33 new75 -rw-r--r-- 2 root root 0 Jul 7 07:33 new76 -rw-r--r-- 2 root root 0 Jul 7 07:33 new77 -rw-r--r-- 2 root root 0 Jul 7 07:33 new79 drwxr-xr-x 2 root root 6 Jul 7 07:33 new8 -rw-r--r-- 2 root root 0 Jul 7 07:33 new80 -rw-r--r-- 2 root root 0 Jul 7 07:33 new82 -rw-r--r-- 2 root root 0 Jul 7 07:33 new83 -rw-r--r-- 2 root root 0 Jul 7 07:33 new84 -rw-r--r-- 2 root root 0 Jul 7 07:33 new85 -rw-r--r-- 2 root root 0 Jul 7 07:33 new86 -rw-r--r-- 2 root root 0 Jul 7 07:33 new87 -rw-r--r-- 2 root root 0 Jul 7 07:33 new89 drwxr-xr-x 2 root root 6 Jul 7 07:33 new9 -rw-r--r-- 2 root root 0 Jul 7 07:33 new90 -rw-r--r-- 2 root root 0 Jul 7 07:33 new91 -rw-r--r-- 2 root root 0 Jul 7 07:33 new92 -rw-r--r-- 2 root root 0 Jul 7 07:33 new93 -rw-r--r-- 2 root root 0 Jul 7 07:33 new94 -rw-r--r-- 2 root root 0 Jul 7 07:33 new95 -rw-r--r-- 2 root root 0 Jul 7 07:33 new96 -rw-r--r-- 2 root root 0 Jul 7 07:33 new98 -rw-r--r-- 2 root root 0 Jul 7 07:33 new99 Actual results: dir does not have layout but files are stored on that brick/dir Expected results: if dir does not have layout than newly created files should not be stored on that volume Additional info:
Fix sent for review upstream @ http://review.gluster.org/5299
verified with 3.4.0.18rhs-1.el6rhs.x86_64
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2013-1262.html