Bug 1486120 - symlinks trigger faulty geo-replication state (rsnapshot usecase)
Summary: symlinks trigger faulty geo-replication state (rsnapshot usecase)
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: geo-replication
Version: 3.12
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Kotresh HR
QA Contact:
URL:
Whiteboard:
Depends On: 1432046 1503174
Blocks: 1431081
TreeView+ depends on / blocked
 
Reported: 2017-08-29 05:38 UTC by Kotresh HR
Modified: 2017-10-17 13:32 UTC (History)
5 users (show)

Fixed In Version: glusterfs-3.12.0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1432046
Environment:
Last Closed: 2017-09-05 17:40:02 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description Kotresh HR 2017-08-29 05:38:57 UTC
+++ This bug was initially created as a clone of Bug #1432046 +++

+++ This bug was initially created as a clone of Bug #1431081 +++
+++                                                           +++
+++ Use this bug to get a fix in the master branch before     +++
+++ backporting it to the maintained versions.                +++

Description of problem:
operations as done by rsnapshot easily trigger faulty geo-replication state

Version-Release number of selected component (if applicable):
3.10, but 3.8 affected the same way

How reproducible:
100%

Steps to Reproduce:
0) create volume on master and slave and setup geo-replication between them
1) mount master volume and cd to it (nfs or fuse doesn't make a difference)
2) simulate a rsnapshot run that updates a symlink like this:

mkdir /tmp/symlinkbug
ln -f -s /does/not/exist /tmp/symlinkbug/a_symlink
rsync -a /tmp/symlinkbug ./
cp -al symlinkbug symlinkbug.0
ln -f -s /does/not/exist2 /tmp/symlinkbug/a_symlink
rsync -a /tmp/symlinkbug ./
cp -al symlinkbug symlinkbug.1

(rsnapshot uses hardlinks between rotations, that's why it is using cp -al)

Actual results:
geo-replication goes to faulty state, and symlinkbug/a_symlink still points to the old location

Expected results:
geo-replication should update the link destination, and not choke/go into faulty state on the second cp -al

Additional info:
you can pause in between the steps, add checkpoints and verify those to have all steps synced separately - doesn't make a difference.

--- Additional comment from Worker Ant on 2017-08-09 06:36:51 EDT ---

REVIEW: https://review.gluster.org/18011 (geo-rep: Fix syncing of hardlink of symlink) posted (#2) for review on master by Kotresh HR (khiremat)

--- Additional comment from Worker Ant on 2017-08-23 23:13:56 EDT ---

COMMIT: https://review.gluster.org/18011 committed in master by Aravinda VK (avishwan) 
------
commit e893962deaabab8e934813f8a0443a8f94e009f2
Author: Kotresh HR <khiremat>
Date:   Tue Aug 8 10:12:14 2017 -0400

    geo-rep: Fix syncing of hardlink of symlink
    
    Problem:
    If there is a hardlink to a symlink on master
    and if the symlink file is deleted on master,
    geo-rep fails to sync the hardlink.
    
    Typical Usecase:
    It's easily hit with rsnapshot use case where
    it uses hardlinks.
    
    Example Reproducer:
    Setup geo-replication between master and slave
    volume and in master mount point, do the following.
    
     1. mkdir /tmp/symlinkbug
     2. ln -f -s /does/not/exist /tmp/symlinkbug/a_symlink
     3. rsync -a /tmp/symlinkbug ./
     4. cp -al symlinkbug symlinkbug.0
     5. ln -f -s /does/not/exist2 /tmp/symlinkbug/a_symlink
     6. rsync -a /tmp/symlinkbug ./
     7. cp -al symlinkbug symlinkbug.1
    
    Cause:
    If the source was not present while syncing hardlink,
    it was always packing the blob as regular file.
    
    Fix:
    If the source was not present while syncing hardlink,
    pack the blob based on the mode.
    
    Change-Id: Iaa12d6f99de47b18e0650e7c4eb455f23f8390f2
    BUG: 1432046
    Signed-off-by: Kotresh HR <khiremat>
    Reported-by: Christian Lohmaier <lohmaier+rhbz>
    Reviewed-on: https://review.gluster.org/18011
    Smoke: Gluster Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>
    Reviewed-by: Aravinda VK <avishwan>

Comment 1 Worker Ant 2017-08-29 06:27:04 UTC
REVIEW: https://review.gluster.org/18127 (geo-rep: Fix syncing of hardlink of symlink) posted (#1) for review on release-3.12 by Kotresh HR (khiremat)

Comment 2 Worker Ant 2017-08-29 12:42:35 UTC
COMMIT: https://review.gluster.org/18127 committed in release-3.12 by Shyamsundar Ranganathan (srangana) 
------
commit b04b3dd4dc8fcff9ad44e31d1da810eba223cd08
Author: Kotresh HR <khiremat>
Date:   Tue Aug 8 10:12:14 2017 -0400

    geo-rep: Fix syncing of hardlink of symlink
    
    Problem:
    If there is a hardlink to a symlink on master
    and if the symlink file is deleted on master,
    geo-rep fails to sync the hardlink.
    
    Typical Usecase:
    It's easily hit with rsnapshot use case where
    it uses hardlinks.
    
    Example Reproducer:
    Setup geo-replication between master and slave
    volume and in master mount point, do the following.
    
     1. mkdir /tmp/symlinkbug
     2. ln -f -s /does/not/exist /tmp/symlinkbug/a_symlink
     3. rsync -a /tmp/symlinkbug ./
     4. cp -al symlinkbug symlinkbug.0
     5. ln -f -s /does/not/exist2 /tmp/symlinkbug/a_symlink
     6. rsync -a /tmp/symlinkbug ./
     7. cp -al symlinkbug symlinkbug.1
    
    Cause:
    If the source was not present while syncing hardlink,
    it was always packing the blob as regular file.
    
    Fix:
    If the source was not present while syncing hardlink,
    pack the blob based on the mode.
    
    > Change-Id: Iaa12d6f99de47b18e0650e7c4eb455f23f8390f2
    > BUG: 1432046
    > Signed-off-by: Kotresh HR <khiremat>
    > Reported-by: Christian Lohmaier <lohmaier+rhbz>
    > Reviewed-on: https://review.gluster.org/18011
    > Smoke: Gluster Build System <jenkins.org>
    > CentOS-regression: Gluster Build System <jenkins.org>
    > Reviewed-by: Aravinda VK <avishwan>
    (cherry picked from commit e893962deaabab8e934813f8a0443a8f94e009f2)
    
    Change-Id: Iaa12d6f99de47b18e0650e7c4eb455f23f8390f2
    BUG: 1486120
    Signed-off-by: Kotresh HR <khiremat>
    Reviewed-on: https://review.gluster.org/18127
    Smoke: Gluster Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>
    Reviewed-by: Shyamsundar Ranganathan <srangana>

Comment 3 Shyamsundar 2017-09-05 17:40:02 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.12.0, please open a new bug report.

glusterfs-3.12.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/announce/2017-September/000082.html
[2] https://www.gluster.org/pipermail/gluster-users/


Note You need to log in before you can comment on or make changes to this bug.