Hide Forgot
Description of problem: ========================= On a 1 x 2 replica volume when running ping_pong on a file added a brick to the volume to increase the replica count to 3 and self-heal is initiated to self-heal the files to newly added brick. 1. Only one brick is marked as source. 2. Entry self-heal succeeds . 3. Data on the file is not self-healed onto newly added brick. Version-Release number of selected component (if applicable): ============================================================= glusterfs 3.4.0.36rhs built on Oct 22 2013 10:56:18 How reproducible: =================== Often Steps to Reproduce: ===================== 1. Create a 1 x 2 replicate volume. Start the volume 2. Create 2 fuse mount. Start ping_pong on the mount points. "ping_pong -rw ping_pong_testfile 6" 3. add-brick to the increase the replica count to 3. 4. Execute : "gluster volume heal <volume_name> full" Actual results: ================ 1 . Only one node is set as the source. i.e the extended attribute "trusted.afr.vol_rep-client-2" is not set on file on both the nodes. 2. Data self-heal is not triggered by the "heal full" command. The root@rhs-client11 [Oct-29-2013-13:38:04] >gluster v add-brick vol_rep replica 3 rhs-client13:/rhs/bricks/b3 volume add-brick: success root@rhs-client11 [Oct-29-2013-13:38:14] > root@rhs-client11 [Oct-29-2013-13:38:15] >getfattr -d -e hex -m . /rhs/bricks/b1/ping_pong_testfile getfattr: Removing leading '/' from absolute path names # file: rhs/bricks/b1/ping_pong_testfile trusted.afr.vol_rep-client-0=0x000000010000000000000000 trusted.afr.vol_rep-client-1=0x000000010000000000000000 trusted.gfid=0x3d720f45e8264e5b99aa33c237c66ef6 root@rhs-client11 [Oct-29-2013-13:38:21] >gluster v heal vol_rep full Launching heal operation to perform full self heal on volume vol_rep has been successful Use heal info commands to check status Node1 extended attributes after the heal: =============================================================== root@rhs-client11 [Oct-29-2013-13:38:29] >getfattr -d -e hex -m . /rhs/bricks/b1/ping_pong_testfile getfattr: Removing leading '/' from absolute path names # file: rhs/bricks/b1/ping_pong_testfile trusted.afr.vol_rep-client-0=0x000000010000000000000000 trusted.afr.vol_rep-client-1=0x000000010000000000000000 "=0x000000010000000100000000 trusted.gfid=0x3d720f45e8264e5b99aa33c237c66ef6 Node2 extended attributes after the heal: =============================================================== root@rhs-client12 [Oct-29-2013-13:38:49] >getfattr -d -e hex -m . /rhs/bricks/b2/ping_pong_testfile getfattr: Removing leading '/' from absolute path names # file: rhs/bricks/b2/ping_pong_testfile trusted.afr.vol_rep-client-0=0x000000060000000000000000 trusted.afr.vol_rep-client-1=0x000000010000000000000000 trusted.gfid=0x3d720f45e8264e5b99aa33c237c66ef6 Node3 extended attrbutes after the heal: =============================================================== root@rhs-client13 [Oct-29-2013-13:38:21] >getfattr -d -e hex -m . /rhs/bricks/b3/ping_pong_testfile getfattr: Removing leading '/' from absolute path names # file: rhs/bricks/b3/ping_pong_testfile trusted.gfid=0x3d720f45e8264e5b99aa33c237c66ef6 Expected results: ================== 1. "trusted.afr.vol_rep-client-2" extended attribute should also be created on Node2 after the heal. 2. Data-self-heal should also be successful on the file after add-brick and execution of "heal full" command.
Executed the case on build " glusterfs 3.4.0.35.1u2rhs built on Oct 21 2013 14:00:58 ". Issue exist on this build too. Also, the "volume heal <volume_name> info healed" doesn't report the entry self-heal of the file to the newly-added brick. root@rhs-client11 [Nov-18-2013- 7:15:46] >gluster v heal vol_rep info healed Gathering list of healed entries on volume vol_rep has been successful Brick rhs-client11:/rhs/bricks/b1 Number of entries: 0 Brick rhs-client12:/rhs/bricks/b1-rep1 Number of entries: 0 Brick rhs-client13:/rhs/bricks/b1-rep2 Number of entries: 0 root@rhs-client11 [Nov-18-2013- 7:16:04] > root@rhs-client11 [Nov-18-2013- 7:16:39] > root@rhs-client11 [Nov-18-2013- 7:16:39] > Brick1:- ========== root@rhs-client11 [Nov-18-2013- 7:16:39] >ls -l /rhs/bricks/b1/ping_pong_testfile -rw------- 2 root root 7 Nov 18 07:16 /rhs/bricks/b1/ping_pong_testfile Brick2:- ========== root@rhs-client12 [Nov-18-2013- 7:16:39] >ls -l /rhs/bricks/b1-rep1/ping_pong_testfile -rw------- 2 root root 7 Nov 18 07:16 /rhs/bricks/b1-rep1/ping_pong_testfile Brick3:- ========== root@rhs-client13 [Nov-18-2013- 7:16:39] >ls -l /rhs/bricks/b1-rep2/ping_pong_testfile -rw------- 2 root root 0 Nov 18 07:12 /rhs/bricks/b1-rep2/ping_pong_testfile
https://code.engineering.redhat.com/gerrit/#/c/16778/
Verified the fix on the build " glusterfs 3.4.0.52rhs built on Dec 19 2013 12:20:16 " with the steps specified in the description. Bug is fixed. Moving the bug to verified state.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHEA-2014-0208.html