Red Hat Bugzilla – Bug 1222776
[geo-rep]: With tarssh the file is created at slave but it doesnt get sync
Last modified: 2015-07-29 00:43:27 EDT
Description of problem: ======================= When we are using sync option as tarssh "use-tarssh true". Whenever the file is created at master volume, the respective entry gets created at the slave but the actual sync doesn't happen. At master volume: ================= [root@wingo master]# pwd /mnt/master [root@wingo master]# ls hosts hosts.allow [root@wingo master]# cat * 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 # # hosts.allow This file contains access rules which are used to # allow or deny connections to network services that # either use the tcp_wrappers library or that have been # started through a tcp_wrappers-enabled xinetd. # # See 'man 5 hosts_options' and 'man 5 hosts_access' # for information on rule syntax. # See 'man tcpd' for information on tcp_wrappers # [root@wingo master]# At slave volume: ================ [root@wingo slave]# pwd /mnt/slave [root@wingo slave]# ls hosts hosts.allow [root@wingo slave]# cat * [root@wingo slave]# cat hosts [root@wingo slave]# cat hosts.allow [root@wingo slave]# Version-Release number of selected component (if applicable): ============================================================= glusterfs-3.7.0-2.el6rhs.x86_64 How reproducible: ================= Always Steps to Reproduce: =================== 1. Create a master cluster 2. Create and Start the master volume 3. Create a slave cluster 4. Create ans Start the slave volume 5. Create and start gluster_shared_storage volume 6. Mount gluster_shared_storage at /var/run/gluster/shared_storage on all the master nodes participating in master volume. 7. Create the geo-rep session 8. Set the config option use_meta_volume to true 9. Set the config option use_tarssh to true 10. Start the geo-rep session 11. Mount the master volume to client and create a file 12. Mount the slave volume 13. Check the files on master and slave volume Actual results: =============== Files that are in master gets created at slave with no content. (The first phase of creation the entry is successful but the actual sync doesn't happen) Expected results: ================= The files should actually sync.
The issue is with recent enhancement to sync extended attributes in geo-replication. The --overwrite option being usind with tar command is not in effect when --xattrs option of tar command is used to sync xattrs. The consequence being if file exists on destination, it does not overwrite, it tries to unlink (which fails in gfid-access translator as entries are banned on aux-gfid-mount) hence fails. NOTE: --overwrite option is used in tar over ssh mode to keep gfids intact. I am looking into the issue. And will update the same with further findings.
As a workaround, set sync_xattrs to false using geo-rep config command. gluster volume geo-replication <MASTERVOL> SLAVEHOST>::<SLAVEVOL> config sync_xattrs false
There is nothing to do in gluster side to sync xattrs and acl using tar over ssh as tar is not respecting --overwrite option when used with --xattrs option. Hence decided to not support acls and xattr syncing with tar over ssh and this will be documented. But syncing xattrs and acls works fine with 'rsync' as sync engine. Sent patch upstream: http://review.gluster.org/#/c/10873/
Patch already in downstream build from upstream 3.7. Upstream Patch (master): http://review.gluster.org/#/c/10873 Upstream Patch (3.7): http://review.gluster.org/#/c/10899/
Verified with build: glusterfs-3.7.0-3.el6rhs.x86_64 [root@georep1 scripts]# gluster volume geo-replication master 10.70.46.154::slave config use_tarssh true [root@georep1 scripts]# Master: ======= [root@wingo master]# pwd /mnt/master [root@wingo master]# ls hosts hosts.allow [root@wingo master]# cat * 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 # # hosts.allow This file contains access rules which are used to # allow or deny connections to network services that # either use the tcp_wrappers library or that have been # started through a tcp_wrappers-enabled xinetd. # # See 'man 5 hosts_options' and 'man 5 hosts_access' # for information on rule syntax. # See 'man tcpd' for information on tcp_wrappers # [root@wingo master]# Slave: ====== [root@wingo slave]# pwd /mnt/slave [root@wingo slave]# ls hosts hosts.allow [root@wingo slave]# cat * 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 # # hosts.allow This file contains access rules which are used to # allow or deny connections to network services that # either use the tcp_wrappers library or that have been # started through a tcp_wrappers-enabled xinetd. # # See 'man 5 hosts_options' and 'man 5 hosts_access' # for information on rule syntax. # See 'man tcpd' for information on tcp_wrappers # [root@wingo slave]# Arequal: ======== [root@wingo ~]# ./scripts/arequal-checksum -p /mnt/master Entry counts Regular files : 2 Directories : 3 Symbolic links : 0 Other : 0 Total : 5 Metadata checksums Regular files : 3e9 Directories : 24d74c Symbolic links : 3e9 Other : 3e9 Checksums Regular files : 6b4cb7a6384c32b8811cf8591fc6145b Directories : 616328010401341a Symbolic links : 0 Other : 0 Total : 8b3367fe238b12f9 [root@wingo ~]# ./scripts/arequal-checksum -p /mnt/slave Entry counts Regular files : 2 Directories : 3 Symbolic links : 0 Other : 0 Total : 5 Metadata checksums Regular files : 3e9 Directories : 24d74c Symbolic links : 3e9 Other : 3e9 Checksums Regular files : 6b4cb7a6384c32b8811cf8591fc6145b Directories : 616328010401341a Symbolic links : 0 Other : 0 Total : 8b3367fe238b12f9 [root@wingo ~]# Moving the bug to verified state
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2015-1495.html