Bug 1024371

Description of problem:
=========================
On a 1 x 2 replica volume when running ping_pong on a file added a brick to the volume to increase the replica count to 3 and self-heal is initiated to self-heal the files to newly added brick. 

1. Only one brick is marked as source. 

2. Entry self-heal succeeds .

3. Data on the file is not self-healed onto newly added brick. 


Version-Release number of selected component (if applicable):
=============================================================
glusterfs 3.4.0.36rhs built on Oct 22 2013 10:56:18

How reproducible:
===================
Often

Steps to Reproduce:
=====================
1. Create a 1 x 2 replicate volume. Start the volume

2. Create 2 fuse mount. Start ping_pong on the mount points. "ping_pong -rw ping_pong_testfile 6" 

3. add-brick to the increase the replica count to 3. 

4. Execute : "gluster volume heal <volume_name> full"

Actual results:
================
1 . Only one node is set as the source. i.e the extended attribute "trusted.afr.vol_rep-client-2" is not set on file on both the nodes.

2. Data self-heal is not triggered by the "heal full" command. The 

root@rhs-client11 [Oct-29-2013-13:38:04] >gluster v add-brick vol_rep replica 3 rhs-client13:/rhs/bricks/b3
volume add-brick: success
root@rhs-client11 [Oct-29-2013-13:38:14] >
root@rhs-client11 [Oct-29-2013-13:38:15] >getfattr -d -e hex -m . /rhs/bricks/b1/ping_pong_testfile
getfattr: Removing leading '/' from absolute path names
# file: rhs/bricks/b1/ping_pong_testfile
trusted.afr.vol_rep-client-0=0x000000010000000000000000
trusted.afr.vol_rep-client-1=0x000000010000000000000000
trusted.gfid=0x3d720f45e8264e5b99aa33c237c66ef6

root@rhs-client11 [Oct-29-2013-13:38:21] >gluster v heal vol_rep full
Launching heal operation to perform full self heal on volume vol_rep has been successful 
Use heal info commands to check status

Node1 extended attributes after the heal:
===============================================================
root@rhs-client11 [Oct-29-2013-13:38:29] >getfattr -d -e hex -m . /rhs/bricks/b1/ping_pong_testfile
getfattr: Removing leading '/' from absolute path names
# file: rhs/bricks/b1/ping_pong_testfile
trusted.afr.vol_rep-client-0=0x000000010000000000000000
trusted.afr.vol_rep-client-1=0x000000010000000000000000
"=0x000000010000000100000000
trusted.gfid=0x3d720f45e8264e5b99aa33c237c66ef6

Node2 extended attributes after the heal:
===============================================================
root@rhs-client12 [Oct-29-2013-13:38:49] >getfattr -d -e hex -m . /rhs/bricks/b2/ping_pong_testfile
getfattr: Removing leading '/' from absolute path names
# file: rhs/bricks/b2/ping_pong_testfile
trusted.afr.vol_rep-client-0=0x000000060000000000000000
trusted.afr.vol_rep-client-1=0x000000010000000000000000
trusted.gfid=0x3d720f45e8264e5b99aa33c237c66ef6

Node3 extended attrbutes after the heal:
===============================================================
root@rhs-client13 [Oct-29-2013-13:38:21] >getfattr -d -e hex -m . /rhs/bricks/b3/ping_pong_testfile
getfattr: Removing leading '/' from absolute path names
# file: rhs/bricks/b3/ping_pong_testfile
trusted.gfid=0x3d720f45e8264e5b99aa33c237c66ef6

Expected results:
==================
1. "trusted.afr.vol_rep-client-2" extended attribute should also be created on Node2 after the heal. 

2. Data-self-heal should also be successful on the file after add-brick and execution of "heal full" command.