Bug 1426952 - [geo-rep]: extended attributes are not synced if the entry and extended attributes are done within changelog roleover/or entry sync
Summary: [geo-rep]: extended attributes are not synced if the entry and extended attri...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: geo-replication
Version: rhgs-3.2
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
: RHGS 3.3.0
Assignee: Mohammed Rafi KC
QA Contact: Rahul Hinduja
URL:
Whiteboard:
Depends On:
Blocks: 1417147 1448914 1463513
TreeView+ depends on / blocked
 
Reported: 2017-02-26 18:11 UTC by Rahul Hinduja
Modified: 2017-09-21 04:57 UTC (History)
6 users (show)

Fixed In Version: glusterfs-3.8.4-26
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1448914 (view as bug list)
Environment:
Last Closed: 2017-09-21 04:33:25 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:2774 0 normal SHIPPED_LIVE glusterfs bug fix and enhancement update 2017-09-21 08:16:29 UTC

Description Rahul Hinduja 2017-02-26 18:11:36 UTC
Description of problem:
=======================

Have observed a scenario where extended attributes are not syncing to slave via rsync/fuse if the entry and setxattr is done within 15 seconds. 

Master:
=======

[root@dj master]# touch r.1 ; setfattr  -n user.rtest1 -v 100112 r.1 
[root@dj master]# getfattr -d -e hex -m . r.1
# file: r.1
security.selinux=0x73797374656d5f753a6f626a6563745f723a6675736566735f743a733000
user.rtest1=0x313030313132

[root@dj master]# touch r.2 ; sleep 20 ; setfattr  -n user.rtest1 -v 100113 r.2
[root@dj master]# touch r.3 ; sleep 20 ; setfattr  -n user.rtest1 -v 100113 r.3
[root@dj master]# touch r.4 ; setfattr  -n user.rtest1 -v 100112 r.4
[root@dj master]# 


Slave:
======

[root@dj slave]# getfattr -d -e hex -m . r.1
# file: r.1
security.selinux=0x73797374656d5f753a6f626a6563745f723a6675736566735f743a733000

#root@dj slave]# 
[root@dj slave]# getfattr -d -e hex -m . r.2
# file: r.2
security.selinux=0x73797374656d5f753a6f626a6563745f723a6675736566735f743a733000

[root@dj slave]# 
[root@dj slave]# 
[root@dj slave]# getfattr -d -e hex -m . r.2
# file: r.2
security.selinux=0x73797374656d5f753a6f626a6563745f723a6675736566735f743a733000
user.rtest1=0x313030313133

[root@dj slave]# getfattr -d -e hex -m . r.1
# file: r.1
security.selinux=0x73797374656d5f753a6f626a6563745f723a6675736566735f743a733000

[root@dj slave]# getfattr -d -e hex -m . r.3
# file: r.3
security.selinux=0x73797374656d5f753a6f626a6563745f723a6675736566735f743a733000
user.rtest1=0x313030313133

[root@dj slave]# getfattr -d -e hex -m . r.4
# file: r.4
security.selinux=0x73797374656d5f753a6f626a6563745f723a6675736566735f743a733000

[root@dj slave]# 
[root@dj slave]# getfattr -d -e hex -m . r.4
# file: r.4
security.selinux=0x73797374656d5f753a6f626a6563745f723a6675736566735f743a733000

[root@dj slave]# 

Looked into the .processed directory, it doesn't contain any information yet. Looked into the changelogs and .processing directory. When extended attribute are applied after 15 sec, it is rightly logged into the new changelog which is in processing directory with (SETXATTR). If it is set with 15 secs, it is not recorded in the changelog but only Entry and SETATTR is only recorded and no record for SETXATTR.

Version-Release number of selected component (if applicable):
=============================================================

glusterfs-geo-replication-3.8.4-15.el7rhgs.x86_64


How reproducible:
=================

Always

Steps to Reproduce:
===================
1. Create geo-rep session between master and slave volume
2. Mount the volumes 
3. Create a file and wait for 20 secs
4. Set the extended attribute on the file. It will properly sync to slave.
5. Create another file and immediately set the extended attribute on the file

Actual results:
===============

extended attributes are not synced to the slave


Expected results:
=================

extended attributes should be synced to the slave

Comment 4 Kotresh HR 2017-02-27 09:01:08 UTC
Analysis:

This is not a regression. This is a day one bug. There is an optimization in changelog to not record multiple METADATA ops on single gfid with in a changelog.
I think this was because there was no extended attribute support during that time and only supported SETATTR. Now since we also support SETXATTR, if SETATTR is already recorded for gfid1, we would miss recording SETXATTR on gfid1. Hence
setxattr would not get synced. This bug will affect the user set extended attributes if any.

The solution would be to remove the optimization for METADATA fops and keep it only for DATA fops.

Comment 8 Mohammed Rafi KC 2017-05-08 16:01:51 UTC
upstream master patch : https://review.gluster.org/#/c/17205/

Comment 11 Rahul Hinduja 2017-08-08 17:30:57 UTC
Verified with build: glusterfs-geo-replication-3.8.4-38.el7rhgs.x86_64

Carried the steps mentioned in the description. Now even if the xattrs are set just after entry, they are properly logged into the same change logs and gets synced to slave properly. 

Master:
=======

[root@dhcp37-115 master]# touch r.3 ; setfattr  -n user.rtest1 -v 100112 r.3
[root@dhcp37-115 master]#  getfattr -d -e hex -m . r.3
# file: r.3
security.selinux=0x73797374656d5f753a6f626a6563745f723a6675736566735f743a733000
user.rtest1=0x313030313132

[root@dhcp37-115 master]# 


Slave:
======

[root@dhcp37-115 slave]# getfattr -d -e hex -m . r.3
# file: r.3
security.selinux=0x73797374656d5f753a6f626a6563745f723a6675736566735f743a733000
user.rtest1=0x313030313132

[root@dhcp37-115 slave]# 


Processed changelog:
====================

[root@dhcp42-79 .processed]# cat CHANGELOG.1502213056
E 69e37a25-ad03-4dbb-9e59-625831db93e6 CREATE 33188 0 0 00000000-0000-0000-0000-000000000001%2Fr.3
M 69e37a25-ad03-4dbb-9e59-625831db93e6 SETATTR
M 69e37a25-ad03-4dbb-9e59-625831db93e6 SETXATTR
[root@dhcp42-79 .processed]# 

Moving this bug to verified state.

Comment 13 errata-xmlrpc 2017-09-21 04:33:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2774

Comment 14 errata-xmlrpc 2017-09-21 04:57:48 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2774


Note You need to log in before you can comment on or make changes to this bug.