Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1176934

Summary:	Changing mtime using touch doesn't copy master file to slave with tar+ssh
Product:	[Community] GlusterFS	Reporter:	Aravinda VK <avishwan>
Component:	geo-replication	Assignee:	Aravinda VK <avishwan>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	mainline	CC:	aavati, avishwan, bugs, csaba, cww, gluster-bugs, khiremat, nlevinki, storage-qa-internal
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:	glusterfs-3.7.0beta1	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:	1176275	Environment:
Last Closed:	2015-05-14 17:26:18 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1176275
Bug Blocks:

Description Aravinda VK 2014-12-23 15:49:18 UTC

+++ This bug was initially created as a clone of Bug #1176275 +++

Description of problem:

During Changelog mode,

  # for i in $(cat path_list.txt); do touch /mount-point/$i; done

While we saw this issue we trying touching the top level directories

  # for i in $(cat path_list.txt); do touch /mount-point/$(dirname $i); done

We waiting, but seems like nothing would change or get copied over. 

So then as a next attempt we stopped the session and started again voila - everything was copied over.

How reproducible:
Always

Steps to Reproduce:
1. Delete a file on slave from brick and its relevant gfid 
2. Touch a file on master 
3. File doesn't get copied over

Actual results:
File doesn't get copied over

Expected results:
File should get copied over, session restart shouldn't be a necessary step

--- Additional comment from Aravinda VK on 2014-12-22 01:23:25 EST ---

In Changelog mode, touch is recorded as SETATTR, and as of now geo-rep is syncing only chmod and chown. So data operation is not triggered.

Since Xsync is FS crawl, it detects the new change and triggers both SETATTR and also DATA. tar+ssh will sync that file.

If we need to sync mtime along with chown and chmod, we need to enhance the geo-rep code. We can't add data when only SETATTR is recorded, since doing rsync/tar+ssh when SETATTR is overhead.

If the expectation is to initiate the data sync for selected files, then we can provide a virtual xattr, which changelog records as SETATTR and DATA fop.

Comment 1 Anand Avati 2014-12-23 15:52:36 UTC

REVIEW: http://review.gluster.org/9331 (geo-rep: Sync atime, mtime to slave when SETATTR) posted (#1) for review on master by Aravinda VK (avishwan)

Comment 2 Anand Avati 2014-12-24 18:41:07 UTC

REVIEW: http://review.gluster.org/9337 (feature/changelog: Virtual xattr to trigger explicit sync in geo-rep.) posted (#1) for review on master by Kotresh HR (khiremat)

Comment 3 Kotresh HR 2014-12-24 18:43:18 UTC

Since touch doesn't initiate a data transfer in geo-replication, A virtual
xattr "glusterfs.geo-rep.trigger-sync" is provided to initiate data sync explicitly. 

Usage:
setfattr -n glusterfs.geo-rep.trigger-sync <file-path>

Comment 4 Anand Avati 2014-12-25 17:49:02 UTC

COMMIT: http://review.gluster.org/9331 committed in master by Venky Shankar (vshankar) 
------
commit 92cc5d4f64d55675b45b85f612f513500d4e4fc7
Author: Aravinda VK <avishwan>
Date:   Tue Dec 23 21:13:32 2014 +0530

    geo-rep: Sync atime, mtime to slave when SETATTR
    
    Existing geo-rep only syncs chown and chmod, with this
    patch geo-rep also syncs atime and mtime.
    
    Change-Id: Iea52d86682873bb4a47eeb0d325f5b9ddf2de2cf
    Signed-off-by: Aravinda VK <avishwan>
    BUG: 1176934
    Reviewed-on: http://review.gluster.org/9331
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Venky Shankar <vshankar>

Comment 5 Anand Avati 2014-12-29 17:08:10 UTC

COMMIT: http://review.gluster.org/9337 committed in master by Venky Shankar (vshankar) 
------
commit 5e7e12694e3898ef7bb5425827122c434a465ce8
Author: Kotresh HR <khiremat>
Date:   Wed Dec 24 21:26:28 2014 +0530

    feature/changelog: Virtual xattr to trigger explicit sync in geo-rep.
    
    A virtual xattr "glusterfs.geo-rep.trigger-sync" is provided
    in glusterfs through changelog translator. Geo-rep triggers
    a explicit data sync on setting this xattr on a file.
    Changelog captures a DATA entry on file's gfid on setting this
    virtual xattr on a file. This is supported only for files. It
    doesn't support directories.
    
    Usage: setfattr -n glusterfs.geo-rep.trigger-sync <file-path>
    
    Change-Id: Ia689326ac2dcb31035ffbecad2c548eda4eb9245
    BUG: 1176934
    Signed-off-by: Kotresh HR <khiremat>
    Reviewed-on: http://review.gluster.org/9337
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Venky Shankar <vshankar>
    Tested-by: Venky Shankar <vshankar>

Comment 6 Anand Avati 2014-12-30 05:56:16 UTC

REVIEW: http://review.gluster.org/9366 (test: Make gfid-access.py more generic.) posted (#1) for review on master by Kotresh HR (khiremat)

Comment 7 Anand Avati 2014-12-31 09:13:30 UTC

REVIEW: http://review.gluster.org/9370 (feature/changelog: Logging entry as well for explicit sync virtual xattr.) posted (#1) for review on master by Kotresh HR (khiremat)

Comment 8 Anand Avati 2014-12-31 09:20:41 UTC

COMMIT: http://review.gluster.org/9366 committed in master by Vijay Bellur (vbellur) 
------
commit 0250eb0698e567134ddd897b134bbe4f8da003a2
Author: Kotresh HR <khiremat>
Date:   Tue Dec 30 10:47:49 2014 +0530

    test: Make gfid-access.py more generic.
    
    Few of the hard coded values, uid, gid and file permissions
    are made as arguments to make the script more generic.
    Also fixes the permission issue which was hard coded
    as integer instead of octal.
    
    Change-Id: Icec700770de2b7cae00e02f783d072860e6d5e2b
    BUG: 1176934
    Signed-off-by: Kotresh HR <khiremat>
    Reviewed-on: http://review.gluster.org/9366
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Aravinda VK <avishwan>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 9 Anand Avati 2015-01-06 10:23:13 UTC

REVIEW: http://review.gluster.org/9370 (feature/changelog: Logging entry as well for explicit sync virtual xattr.) posted (#2) for review on master by Kotresh HR (khiremat)

Comment 10 Anand Avati 2015-01-08 12:42:52 UTC

REVIEW: http://review.gluster.org/9370 (feature/changelog: Logging entry as well for explicit sync virtual xattr.) posted (#3) for review on master by Kotresh HR (khiremat)

Comment 11 Anand Avati 2015-01-09 09:17:33 UTC

COMMIT: http://review.gluster.org/9370 committed in master by Venky Shankar (vshankar) 
------
commit bbb4b7679be26ed5460524bcf26e779f6907d299
Author: Kotresh HR <khiremat>
Date:   Wed Dec 31 12:47:47 2014 +0530

    feature/changelog: Logging entry as well for explicit sync virtual xattr.
    
    This is an improvement over the patch 'http://review.gluster.org/9337'
    to trigger explicit geo-rep sync on regular files even if entry is not
    present on the slave. An attempt is made to find the pargfid and
    if available captures CREATE along with DATA in changelog.
    CREATE is captured with default file permissions. Setting this virtual
    setxattr on directories captures MKDIR in changelog. The value of
    setxattr can be as follows.
    
    If value = "1"       : Both CREATE and DATA is captured in changelog if
                           pargfid is available, else on DATA is captured.
       value = "any other: ENOTSUP is returned.
    
    Usage:
      setfattr -n glusterfs.geo-rep.trigger-sync -v "1" <file-path>
    
    NOTE: This patch supports explicit record of entries only for
          directories and regular files.
    
    Change-Id: Iedde8b2c8bc3b78db524050d8c866ff664811d01
    BUG: 1176934
    Signed-off-by: Kotresh HR <khiremat>
    Reviewed-on: http://review.gluster.org/9370
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Venky Shankar <vshankar>
    Tested-by: Venky Shankar <vshankar>

Comment 12 Niels de Vos 2015-05-14 17:26:18 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Comment 13 Niels de Vos 2015-05-14 17:28:15 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Comment 14 Niels de Vos 2015-05-14 17:35:14 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user