Bug 1230694 - [Backup]: Glusterfind pre fails with htime xattr updation error resulting in historical changelogs not available
Summary: [Backup]: Glusterfind pre fails with htime xattr updation error resulting in ...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: glusterfind
Version: 3.7.1
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
Assignee: Kotresh HR
QA Contact: bugs@gluster.org
URL:
Whiteboard:
Depends On: 1228495 1230015
Blocks: 1223636
TreeView+ depends on / blocked
 
Reported: 2015-06-11 11:50 UTC by Kotresh HR
Modified: 2015-10-01 02:10 UTC (History)
6 users (show)

Fixed In Version: glusterfs-3.7.2
Clone Of: 1230015
Environment:
Last Closed: 2015-06-20 09:50:52 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description Kotresh HR 2015-06-11 11:50:07 UTC
+++ This bug was initially created as a clone of Bug #1230015 +++

+++ This bug was initially created as a clone of Bug #1228495 +++

Description of problem:

Had a 2node cluster with 2 volumes 2*2 dist-rep 'ozone' and 4*1 distribute 'nash',  with glusterfind sessions for both of them. Glutserfind command on ozone resulted in a crash (bug 1228017), post which I carried on my testing with nash volume.

Did creates, renames, file moves, metadata changes and operations of those kind at teh fuse mountpoint of nash. Ran glusterfind commands pre and post everytime there was a change. Left the setup untouched for close to 3 hours. 

Ran the glusterfind pre command again, without doing any changes on the mountpoint, and it resulted in an error on node2, stating 'Historical changelogs not available'. It managed to get the changelogs on node1. glutserfind list displayed healthy session entries on node1 but 'session corrupted' entries on node2. 

Brick logs displayed the following error on node2:

[2015-06-04 12:37:06.887081] W [socket.c:642:__socket_rwv] 0-nash-changelog: readv on /var/run/gluster/.7fa2c65ef2768e33fb16e3a06d07afa73795.sock failed (No data available)
[2015-06-04 12:45:22.519142] W [socket.c:642:__socket_rwv] 0-nash-changelog: readv on /var/run/gluster/.7fa2c65ef2768e33fb16e3a06d07afa73942.sock failed (No data available)
[2015-06-04 16:29:18.732545] E [changelog-helpers.c:333:htime_update] 0-nash-changelog: Htime xattr updation failed, reason (No data available)
[2015-06-04 16:29:33.749876] E [changelog-helpers.c:333:htime_update] 0-nash-changelog: Htime xattr updation failed, reason (No data available)
[2015-06-04 16:29:48.767266] E [changelog-helpers.c:333:htime_update] 0-nash-changelog: Htime xattr updation failed, reason (No data available)
[2015-06-04 16:30:03.784468] E [changelog-helpers.c:333:htime_update] 0-nash-changelog: Htime xattr updation failed, reason (No data available)



Version-Release number of selected component (if applicable):


How reproducible: 1:1

--- Additional comment from Anand Avati on 2015-06-10 02:51:26 EDT ---

REVIEW: http://review.gluster.org/11150 (features/changelog: Do htime setxattr without XATTR_REPLACE flag) posted (#1) for review on master by Kotresh HR (khiremat)

Comment 1 Anand Avati 2015-06-12 06:54:17 UTC
REVIEW: http://review.gluster.org/11181 (features/changelog: Do htime setxattr without XATTR_REPLACE flag) posted (#3) for review on release-3.7 by Kotresh HR (khiremat)

Comment 2 Anand Avati 2015-06-12 18:38:09 UTC
COMMIT: http://review.gluster.org/11181 committed in release-3.7 by Vijay Bellur (vbellur) 
------
commit 1b7721817a7807bc0af51d89d61fbb723769b2d3
Author: Kotresh HR <khiremat>
Date:   Tue Jun 9 10:44:44 2015 +0530

    features/changelog: Do htime setxattr without XATTR_REPLACE flag
    
    HTIME_KEY marks the last changelog rolled over. The xattr is
    maintained on .glusterfs/changelog/htime/HTIME.TSTAMP file.
    On every rollover of the changelog file, the xattr is updated.
    It is being updated with XATTR_REPLACE flag as xattr gets
    created during changelog enable. But it is once found that
    the xattrs on the file is cleared and is not reproduced later
    on. This patch protects that case, if it happens by setting
    xattr without XATTR_REPLACE flag in failure case.
    
    The reason behind doing this in failure case is not to mask
    the actual cause of xattrs getting cleared. This provides
    the log message if the original issue still exists but the
    consequential effects are fixed.
    
    Also changed the log messages to depict the events happened
    during changelog enable.
    
    Change-Id: I699ed09a03667fd823d01d65c9c360fa7bc0e455
    BUG: 1230694
    Reviewed-On: http://review.gluster.org/#/c/11150/
    Signed-off-by: Kotresh HR <khiremat>
    Reviewed-on: http://review.gluster.org/11181
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 3 Niels de Vos 2015-06-20 09:50:52 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.2, please reopen this bug report.

glusterfs-3.7.2 has been announced on the Gluster Packaging mailinglist [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://www.gluster.org/pipermail/packaging/2015-June/000006.html
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Comment 4 BugMasta 2015-09-30 06:38:16 UTC
I am still seeing this bug in 3.7.4

Comment 5 BugMasta 2015-10-01 02:10:33 UTC
Well, we're seeing an error that historical changelogs are unavailable. The cause looks like it's different.


Note You need to log in before you can comment on or make changes to this bug.