+++ This bug was initially created as a clone of Bug #1198056 +++
Description of problem:
Some of the files which were present before the geo-rep session was created. The files didn't sync to slave until setfattr was performed on those files.
Master volume had the following 4 files:
[root@wingo master]# ls
host.conf hosts hosts.allow hosts.deny
1. Create 4 node cluster (master cluster)
2. Create a 2x2 volume (gr)
3. Start the volume
4. Mount the volume on client (/mnt/master)
5. cp -rf /etc/host* /mnt/master/
6. Create another 4 node cluster (slave cluster)
7. Create a 1x2 volume (gr) from node 1 and node 2 of slave cluster
8. Start the volume
9. Mount the volume on client (/mnt/slave)
10. Create password less ssh between node 1 of master and node 2 of slave.
11. Execute gluster system:: execute gsec_create on node 1 of master
12. Create a geo-rep session between node 1 of master to node 1 of slave.
gluster volume geo-replication <master_volume> <slave_host>::<slave_volume> create push-pem [force]
13. Start a geo-rep session
gluster volume geo-replication <master_volume> <slave_host>::<slave_volume> start
14. ls to the slave volume (ls /mnt/slave)
2 files "host.conf and hosts" were synced to slave and 2 files "hosts.allow and hosts.deny" didn't sync to slave volume.
Performed "setfattr -n glusterfs.geo-rep.trigger-sync -v "1" /mnt/master/hosts.deny" and
"setfattr -n glusterfs.geo-rep.trigger-sync -v "1" /mnt/master/hosts.allow"
after which the files were synced to /mnt/slave
All the files should be successfully synced to slave volume.
Version-Release number of selected component (if applicable):
REVIEW: http://review.gluster.org/9855 (geo-rep: Do not use xsync_upper_limit for change detection) posted (#1) for review on master by Aravinda VK (email@example.com)
Root caused the issue:
If files created before Geo-rep create, xtime xattr will not be available. After geo-rep create it enables indexing. Marker will update xtime for all the files which are created/modified after indexing is enabled.
Geo-rep uses changelog register time as upper limit for xsync to decide a file should be synced or not. It compares xtime of each file >= upper_limit, If xtime is not found it updates xtime with current time. But this updated xtime is greater than upper_limit, xsync skips this file by thinking that changelog will pick this. But changelog will not have CREATE info for this file only SETXATTR/DATA is recorded.
Patch sent to fix this issue:
REVIEW: http://review.gluster.org/9855 (geo-rep: Do not use xsync_upper_limit for change detection) posted (#2) for review on master by Aravinda VK (firstname.lastname@example.org)
COMMIT: http://review.gluster.org/9855 committed in master by Vijay Bellur (email@example.com)
Author: Aravinda VK <firstname.lastname@example.org>
Date: Wed Mar 11 13:31:09 2015 +0530
geo-rep: Do not use xsync_upper_limit for change detection
Use register time(xsync_upper_limit) only for stime update, do not
use for change detection.
If a file created before geo-rep, xtime xattr does not exist.
Geo-rep updates xtime of the file to current time if not exists.
xtime > upper_limit so geo-rep will not pick those files. Changelog
either will have SETXATTR, and fails to sync the file.
If a file is created before geo-rep create and updated after
geo-rep start. xtime of the file is greater than upper limit(geo-rep
start time/changelog register time). Geo-rep(XSync) will not pick this
file for syncing. Changelog will have only DATA recorded for that file.
Geo-rep tries DATA without any ENTRY ops and fails with rsync error.
Signed-off-by: Aravinda VK <email@example.com>
Reviewed-by: Kotresh HR <firstname.lastname@example.org>
Reviewed-by: Saravanakumar Arumugam <email@example.com>
Tested-by: Saravanakumar Arumugam <firstname.lastname@example.org>
Tested-by: Gluster Build System <email@example.com>
Reviewed-by: Vijay Bellur <firstname.lastname@example.org>
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.
glusterfs-3.7.0 has been announced on the Gluster mailinglists , packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist  and the update infrastructure for your distribution.