Bug 1207624

Summary: BitRot :- scrubber is not detecting rotten data and not marking file as 'BAD' file
Product: [Community] GlusterFS Reporter: Rachana Patel <racpatel>
Component: bitrotAssignee: Venky Shankar <vshankar>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: high Docs Contact: bugs <bugs>
Priority: unspecified    
Version: mainlineCC: annair, bugs, hchiramm, mzywusko, nsathyan, vagarwal
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: glusterfs-3.7.0beta1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-05-14 17:27:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1186580    

Description Rachana Patel 2015-03-31 10:43:48 UTC
Description of problem:
=======================
once file is signed with proper hash value by bitd, modify file in backend but scrubber was unable to detect changes and didn't mark file as 'BAD'

Version-Release number of selected component (if applicable):
=============================================================
0.803.gitf64666f.el6.x86_64

How reproducible:
=================
always


Steps to Reproduce:
===================
1. create and mount gluster volume. 
2. Enable bitrot and create files.
3. Once files have been signed go to bricks and modify file from backend.


Actual results:
===============
scrubber is failed to detect checksum mismatch. no xattr set and log doesnt have any error reported for the same

Expected results:
================
Scrubber should mark file as 'BAD' in case of checksum mismtach

Comment 1 Anand Avati 2015-04-02 10:35:57 UTC
REVIEW: http://review.gluster.org/10118 (bitrot/scrub: Scrubber fixes) posted (#1) for review on master by Venky Shankar (vshankar)

Comment 2 Anand Avati 2015-04-06 14:18:28 UTC
REVIEW: http://review.gluster.org/10118 (bitrot/scrub: Scrubber fixes) posted (#2) for review on master by Venky Shankar (vshankar)

Comment 3 Anand Avati 2015-04-07 08:39:20 UTC
REVIEW: http://review.gluster.org/10118 (bitrot/scrub: Scrubber fixes) posted (#3) for review on master by Venky Shankar (vshankar)

Comment 4 Anand Avati 2015-04-07 10:38:36 UTC
REVIEW: http://review.gluster.org/10118 (bitrot/scrub: Scrubber fixes) posted (#4) for review on master by Venky Shankar (vshankar)

Comment 5 Anand Avati 2015-04-08 07:03:11 UTC
COMMIT: http://review.gluster.org/10118 committed in master by Vijay Bellur (vbellur) 
------
commit f0cd1d73c63001740cd7691a77df7631c9b8e8dc
Author: Venky Shankar <vshankar>
Date:   Tue Mar 31 15:48:18 2015 +0530

    bitrot/scrub: Scrubber fixes
    
    This patch fixes a handful of problem with scrubber which
    are detailed below.
    
    Scrubber used to skip objects for verification due to missing
    fd iterface to fetch versioning extended attributes. Similar
    to the inode interface, an fd based interface in POSIX is now
    introduced.
    
    Moreover, this patch also fixes potential false reporting by
    scrubber due to:
    
      An object gets dirtied and signed when scrubber is busy
      calculatingobject checksum. This is fixed by caching the
      signed version when an object is first inspected for
      stalenes, i.e., during pre-compute stage. This version is
      used to verify checksum in the post-compute stage when the
      signatures are compared for possible corruption.
    
      Side effect of _not_ sending signature length during signing
      resulted in "truncated" signature to be set for an object.
      Now, at the time of signing, the signature length is sent
      and is used in place of invoking strlen() to get signature
      length (which could have possible 00s). The signature length
      itself is not persisted in the signature xattr, but is
      calculated on-the-fly by substracting the xattr length by
      the "structure" header size.
    
    Some of the log entries are made more meaningful (as and aid
    for debugging).
    
    Change-Id: I938bee5aea6688d5d99eb2640053613af86d6269
    BUG: 1207624
    Signed-off-by: Venky Shankar <vshankar>
    Reviewed-on: http://review.gluster.org/10118
    Reviewed-by: Raghavendra Bhat <raghavendra>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 6 Anand Avati 2015-04-08 10:50:20 UTC
REVIEW: http://review.gluster.org/10162 (test/bitrot-stub: fix spurious devrpm build failure) posted (#1) for review on master by Gaurav Kumar Garg (ggarg)

Comment 8 Vivek Agarwal 2015-04-23 10:56:03 UTC
Patch in comment 6 abandoned

Comment 9 Niels de Vos 2015-05-14 17:27:10 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Comment 10 Niels de Vos 2015-05-14 17:28:39 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Comment 11 Niels de Vos 2015-05-14 17:35:18 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user