Bug 1021686 - refactor AFR module
refactor AFR module
Status: CLOSED CURRENTRELEASE
Product: GlusterFS
Classification: Community
Component: replicate (Show other bugs)
mainline
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Anand Avati
:
: 951800 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-10-21 15:27 EDT by Anand Avati
Modified: 2015-09-01 19:06 EDT (History)
5 users (show)

See Also:
Fixed In Version: glusterfs-3.6.0beta1
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-11-11 03:24:16 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Anand Avati 2013-10-21 15:27:37 EDT
Refactor AFR module to remove accumulated kruft code and general cleanup
Comment 1 Anand Avati 2013-10-21 15:29:27 EDT
REVIEW: http://review.gluster.org/6112 (syncops: expose @flags in syncop_rmdir()) posted (#3) for review on master by Anand Avati (avati@redhat.com)
Comment 2 Anand Avati 2013-10-21 15:29:35 EDT
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#5) for review on master by Anand Avati (avati@redhat.com)
Comment 3 Anand Avati 2013-10-21 15:29:41 EDT
REVIEW: http://review.gluster.org/6031 (replace-brick: remove code from cli and glusterd) posted (#5) for review on master by Anand Avati (avati@redhat.com)
Comment 4 Anand Avati 2013-10-21 15:41:22 EDT
REVIEW: http://review.gluster.org/6112 (syncops: expose @flags in syncop_rmdir()) posted (#4) for review on master by Anand Avati (avati@redhat.com)
Comment 5 Anand Avati 2013-10-21 15:41:30 EDT
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#6) for review on master by Anand Avati (avati@redhat.com)
Comment 6 Anand Avati 2013-10-21 15:41:38 EDT
REVIEW: http://review.gluster.org/6031 (replace-brick: remove code from cli and glusterd) posted (#6) for review on master by Anand Avati (avati@redhat.com)
Comment 7 Anand Avati 2014-01-17 17:04:46 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#7) for review on master by Anand Avati (avati@redhat.com)
Comment 8 Anand Avati 2014-01-19 16:22:29 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#8) for review on master by Anand Avati (avati@redhat.com)
Comment 9 Anand Avati 2014-01-20 00:36:57 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#9) for review on master by Anand Avati (avati@redhat.com)
Comment 10 Anand Avati 2014-01-21 00:38:52 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#10) for review on master by Anand Avati (avati@redhat.com)
Comment 11 Anand Avati 2014-01-23 02:28:41 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#11) for review on master by Anand Avati (avati@redhat.com)
Comment 12 Anand Avati 2014-01-25 02:45:50 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#12) for review on master by Anand Avati (avati@redhat.com)
Comment 13 Anand Avati 2014-02-03 02:45:56 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#13) for review on master by Anand Avati (avati@redhat.com)
Comment 14 Anand Avati 2014-02-03 02:46:05 EST
REVIEW: http://review.gluster.org/6888 (syncops: add support for custom PID) posted (#1) for review on master by Anand Avati (avati@redhat.com)
Comment 15 Anand Avati 2014-02-04 14:22:30 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#14) for review on master by Anand Avati (avati@redhat.com)
Comment 16 Anand Avati 2014-02-04 14:22:40 EST
REVIEW: http://review.gluster.org/6888 (syncops: add support for custom PID) posted (#2) for review on master by Anand Avati (avati@redhat.com)
Comment 17 Anand Avati 2014-02-06 00:04:58 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#15) for review on master by Anand Avati (avati@redhat.com)
Comment 18 Anand Avati 2014-02-06 00:05:08 EST
REVIEW: http://review.gluster.org/6888 (syncops: add support for custom PID) posted (#3) for review on master by Anand Avati (avati@redhat.com)
Comment 19 Anand Avati 2014-02-06 15:34:20 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#16) for review on master by Anand Avati (avati@redhat.com)
Comment 20 Anand Avati 2014-02-06 15:34:54 EST
REVIEW: http://review.gluster.org/6888 (syncops: add support for custom PID) posted (#4) for review on master by Anand Avati (avati@redhat.com)
Comment 21 Anand Avati 2014-02-06 15:35:01 EST
REVIEW: http://review.gluster.org/6932 (storage/posix: calculate checksum only over read data) posted (#1) for review on master by Anand Avati (avati@redhat.com)
Comment 22 Anand Avati 2014-02-07 20:35:39 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#17) for review on master by Anand Avati (avati@redhat.com)
Comment 23 Anand Avati 2014-02-07 20:35:51 EST
REVIEW: http://review.gluster.org/6932 (storage/posix: calculate checksum only over read data) posted (#2) for review on master by Anand Avati (avati@redhat.com)
Comment 24 Anand Avati 2014-02-07 20:35:58 EST
REVIEW: http://review.gluster.org/6888 (syncops: add support for custom PID) posted (#5) for review on master by Anand Avati (avati@redhat.com)
Comment 25 Anand Avati 2014-02-07 20:36:06 EST
REVIEW: http://review.gluster.org/6941 (core: add @xdata parameter to syncop_[f]removexattr()) posted (#1) for review on master by Anand Avati (avati@redhat.com)
Comment 26 Anand Avati 2014-02-07 23:37:53 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#18) for review on master by Anand Avati (avati@redhat.com)
Comment 27 Anand Avati 2014-02-07 23:38:01 EST
REVIEW: http://review.gluster.org/6932 (storage/posix: calculate checksum only over read data) posted (#3) for review on master by Anand Avati (avati@redhat.com)
Comment 28 Anand Avati 2014-02-07 23:38:09 EST
REVIEW: http://review.gluster.org/6888 (syncops: add support for custom PID) posted (#6) for review on master by Anand Avati (avati@redhat.com)
Comment 29 Anand Avati 2014-02-07 23:38:16 EST
REVIEW: http://review.gluster.org/6941 (core: add @xdata parameter to syncop_[f]removexattr()) posted (#2) for review on master by Anand Avati (avati@redhat.com)
Comment 30 Anand Avati 2014-02-08 04:11:14 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#19) for review on master by Anand Avati (avati@redhat.com)
Comment 31 Anand Avati 2014-02-08 04:11:23 EST
REVIEW: http://review.gluster.org/6888 (syncops: add support for custom PID) posted (#7) for review on master by Anand Avati (avati@redhat.com)
Comment 32 Anand Avati 2014-02-08 04:11:31 EST
REVIEW: http://review.gluster.org/6932 (storage/posix: calculate checksum only over read data) posted (#4) for review on master by Anand Avati (avati@redhat.com)
Comment 33 Anand Avati 2014-02-08 04:11:40 EST
REVIEW: http://review.gluster.org/6941 (core: add @xdata parameter to syncop_[f]removexattr()) posted (#3) for review on master by Anand Avati (avati@redhat.com)
Comment 34 Anand Avati 2014-02-08 04:40:25 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#20) for review on master by Anand Avati (avati@redhat.com)
Comment 35 Anand Avati 2014-02-08 04:40:35 EST
REVIEW: http://review.gluster.org/6932 (storage/posix: calculate checksum only over read data) posted (#5) for review on master by Anand Avati (avati@redhat.com)
Comment 36 Anand Avati 2014-02-08 04:40:43 EST
REVIEW: http://review.gluster.org/6888 (syncops: add support for custom PID) posted (#8) for review on master by Anand Avati (avati@redhat.com)
Comment 37 Anand Avati 2014-02-08 04:40:51 EST
REVIEW: http://review.gluster.org/6941 (core: add @xdata parameter to syncop_[f]removexattr()) posted (#4) for review on master by Anand Avati (avati@redhat.com)
Comment 38 Anand Avati 2014-02-09 03:19:27 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#21) for review on master by Anand Avati (avati@redhat.com)
Comment 39 Anand Avati 2014-02-09 03:19:36 EST
REVIEW: http://review.gluster.org/6932 (storage/posix: calculate checksum only over read data) posted (#6) for review on master by Anand Avati (avati@redhat.com)
Comment 40 Anand Avati 2014-02-09 03:19:44 EST
REVIEW: http://review.gluster.org/6888 (syncops: add support for custom PID) posted (#9) for review on master by Anand Avati (avati@redhat.com)
Comment 41 Anand Avati 2014-02-09 03:19:51 EST
REVIEW: http://review.gluster.org/6941 (core: add @xdata parameter to syncop_[f]removexattr()) posted (#5) for review on master by Anand Avati (avati@redhat.com)
Comment 42 Anand Avati 2014-02-09 17:29:07 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#22) for review on master by Anand Avati (avati@redhat.com)
Comment 43 Anand Avati 2014-02-09 17:29:15 EST
REVIEW: http://review.gluster.org/6888 (syncops: add support for custom PID) posted (#10) for review on master by Anand Avati (avati@redhat.com)
Comment 44 Anand Avati 2014-02-09 17:29:23 EST
REVIEW: http://review.gluster.org/6932 (storage/posix: calculate checksum only over read data) posted (#7) for review on master by Anand Avati (avati@redhat.com)
Comment 45 Anand Avati 2014-02-09 17:29:31 EST
REVIEW: http://review.gluster.org/6941 (core: add @xdata parameter to syncop_[f]removexattr()) posted (#6) for review on master by Anand Avati (avati@redhat.com)
Comment 46 Anand Avati 2014-02-09 19:27:53 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#23) for review on master by Anand Avati (avati@redhat.com)
Comment 47 Anand Avati 2014-02-09 19:28:01 EST
REVIEW: http://review.gluster.org/6932 (storage/posix: calculate checksum only over read data) posted (#8) for review on master by Anand Avati (avati@redhat.com)
Comment 48 Anand Avati 2014-02-09 19:28:09 EST
REVIEW: http://review.gluster.org/6888 (syncops: add support for custom PID) posted (#11) for review on master by Anand Avati (avati@redhat.com)
Comment 49 Anand Avati 2014-02-09 19:28:17 EST
REVIEW: http://review.gluster.org/6941 (core: add @xdata parameter to syncop_[f]removexattr()) posted (#7) for review on master by Anand Avati (avati@redhat.com)
Comment 50 Anand Avati 2014-02-10 02:22:23 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#24) for review on master by Anand Avati (avati@redhat.com)
Comment 51 Anand Avati 2014-02-10 02:22:32 EST
REVIEW: http://review.gluster.org/6888 (syncops: add support for custom PID) posted (#12) for review on master by Anand Avati (avati@redhat.com)
Comment 52 Anand Avati 2014-02-10 02:22:41 EST
REVIEW: http://review.gluster.org/6932 (storage/posix: calculate checksum only over read data) posted (#9) for review on master by Anand Avati (avati@redhat.com)
Comment 53 Anand Avati 2014-02-10 02:22:49 EST
REVIEW: http://review.gluster.org/6941 (core: add @xdata parameter to syncop_[f]removexattr()) posted (#8) for review on master by Anand Avati (avati@redhat.com)
Comment 54 Anand Avati 2014-02-11 03:19:41 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#25) for review on master by Anand Avati (avati@redhat.com)
Comment 55 Anand Avati 2014-02-11 03:19:49 EST
REVIEW: http://review.gluster.org/6888 (syncops: add support for custom PID) posted (#13) for review on master by Anand Avati (avati@redhat.com)
Comment 56 Anand Avati 2014-02-11 03:20:02 EST
REVIEW: http://review.gluster.org/6932 (storage/posix: calculate checksum only over read data) posted (#10) for review on master by Anand Avati (avati@redhat.com)
Comment 57 Anand Avati 2014-02-11 03:20:18 EST
REVIEW: http://review.gluster.org/6941 (core: add @xdata parameter to syncop_[f]removexattr()) posted (#9) for review on master by Anand Avati (avati@redhat.com)
Comment 58 Anand Avati 2014-02-11 04:04:44 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#26) for review on master by Anand Avati (avati@redhat.com)
Comment 59 Anand Avati 2014-02-11 04:04:52 EST
REVIEW: http://review.gluster.org/6888 (syncops: add support for custom PID) posted (#14) for review on master by Anand Avati (avati@redhat.com)
Comment 60 Anand Avati 2014-02-11 04:05:06 EST
REVIEW: http://review.gluster.org/6932 (storage/posix: calculate checksum only over read data) posted (#11) for review on master by Anand Avati (avati@redhat.com)
Comment 61 Anand Avati 2014-02-11 04:05:13 EST
REVIEW: http://review.gluster.org/6941 (core: add @xdata parameter to syncop_[f]removexattr()) posted (#10) for review on master by Anand Avati (avati@redhat.com)
Comment 62 Anand Avati 2014-02-12 01:11:49 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#27) for review on master by Anand Avati (avati@redhat.com)
Comment 63 Anand Avati 2014-02-12 01:11:59 EST
REVIEW: http://review.gluster.org/6888 (syncops: add support for custom PID) posted (#15) for review on master by Anand Avati (avati@redhat.com)
Comment 64 Anand Avati 2014-02-12 01:12:07 EST
REVIEW: http://review.gluster.org/6932 (storage/posix: calculate checksum only over read data) posted (#12) for review on master by Anand Avati (avati@redhat.com)
Comment 65 Anand Avati 2014-02-12 01:12:16 EST
REVIEW: http://review.gluster.org/6941 (core: add @xdata parameter to syncop_[f]removexattr()) posted (#11) for review on master by Anand Avati (avati@redhat.com)
Comment 66 Anand Avati 2014-02-12 03:39:52 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#28) for review on master by Anand Avati (avati@redhat.com)
Comment 67 Anand Avati 2014-02-12 03:40:03 EST
REVIEW: http://review.gluster.org/6888 (syncops: add support for custom PID) posted (#16) for review on master by Anand Avati (avati@redhat.com)
Comment 68 Anand Avati 2014-02-12 03:40:12 EST
REVIEW: http://review.gluster.org/6932 (storage/posix: calculate checksum only over read data) posted (#13) for review on master by Anand Avati (avati@redhat.com)
Comment 69 Anand Avati 2014-02-12 03:40:24 EST
REVIEW: http://review.gluster.org/6941 (core: add @xdata parameter to syncop_[f]removexattr()) posted (#12) for review on master by Anand Avati (avati@redhat.com)
Comment 70 Anand Avati 2014-02-12 14:19:11 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#29) for review on master by Anand Avati (avati@redhat.com)
Comment 71 Anand Avati 2014-02-12 14:19:19 EST
REVIEW: http://review.gluster.org/6888 (syncops: add support for custom PID) posted (#17) for review on master by Anand Avati (avati@redhat.com)
Comment 72 Anand Avati 2014-02-12 14:19:27 EST
REVIEW: http://review.gluster.org/6932 (storage/posix: calculate checksum only over read data) posted (#14) for review on master by Anand Avati (avati@redhat.com)
Comment 73 Anand Avati 2014-02-12 14:19:35 EST
REVIEW: http://review.gluster.org/6941 (core: add @xdata parameter to syncop_[f]removexattr()) posted (#13) for review on master by Anand Avati (avati@redhat.com)
Comment 74 Anand Avati 2014-02-13 14:17:18 EST
COMMIT: http://review.gluster.org/6941 committed in master by Anand Avati (avati@redhat.com) 
------
commit 0cab34b3a5e94267bf6b39669b6e85af1fab8f3d
Author: Anand Avati <avati@redhat.com>
Date:   Fri Feb 7 14:29:34 2014 -0800

    core: add @xdata parameter to syncop_[f]removexattr()
    
    To be used in afr metadata self-heal
    
    Change-Id: I8dac4b19d61e331702427eeb5b606aab3d20b328
    BUG: 1021686
    Signed-off-by: Anand Avati <avati@redhat.com>
    Reviewed-on: http://review.gluster.org/6941
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Comment 75 Anand Avati 2014-02-13 14:18:53 EST
COMMIT: http://review.gluster.org/6932 committed in master by Anand Avati (avati@redhat.com) 
------
commit 7b4d6e3ec61e9cb0b9d0ffa0aef2117b96e63f65
Author: Anand Avati <avati@redhat.com>
Date:   Thu Feb 6 12:32:01 2014 -0800

    storage/posix: calculate checksum only over read data
    
    If the last block of a file is not aligned to the requested
    size, checksum is calculated over junk data in the iobuf. Or
    it could be zeroes, resulting in a spurious checksum match
    in self-heal.
    
    Change-Id: I41422e08de90013dabfc348ec6fbb8ecdd4f8fb8
    BUG: 1021686
    Signed-off-by: Anand Avati <avati@redhat.com>
    Reviewed-on: http://review.gluster.org/6932
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Comment 76 Anand Avati 2014-02-13 14:19:52 EST
COMMIT: http://review.gluster.org/6888 committed in master by Anand Avati (avati@redhat.com) 
------
commit 3571066deedfe858ef37f09d6ad2160e5dd7b803
Author: Anand Avati <avati@redhat.com>
Date:   Mon Jan 27 00:58:45 2014 -0800

    syncops: add support for custom PID
    
    AFR self-heal needs to issue syncops with special PID. Extend
    the custom UID/GID support to include custom PIDs
    
    Change-Id: I736c0e177f862b029f203acc87f9eb46c8cb839b
    BUG: 1021686
    Signed-off-by: Anand Avati <avati@redhat.com>
    Reviewed-on: http://review.gluster.org/6888
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Comment 77 Anand Avati 2014-02-24 05:33:05 EST
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#30) for review on master by Anand Avati (avati@redhat.com)
Comment 78 Anand Avati 2014-03-11 01:34:15 EDT
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#31) for review on master by Anand Avati (avati@redhat.com)
Comment 79 Anand Avati 2014-03-11 02:15:34 EDT
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#32) for review on master by Anand Avati (avati@redhat.com)
Comment 80 Anand Avati 2014-03-12 03:52:02 EDT
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#33) for review on master by Anand Avati (avati@redhat.com)
Comment 81 Anand Avati 2014-03-12 04:01:52 EDT
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#34) for review on master by Anand Avati (avati@redhat.com)
Comment 82 Anand Avati 2014-03-20 18:07:14 EDT
REVIEW: http://review.gluster.org/6010 (cluster/afr: refactor) posted (#35) for review on master by Anand Avati (avati@redhat.com)
Comment 83 Anand Avati 2014-03-22 08:26:04 EDT
COMMIT: http://review.gluster.org/6010 committed in master by Vijay Bellur (vbellur@redhat.com) 
------
commit 6d3739292b7b51d2ddbab75b5f884fb38925b943
Author: Anand Avati <avati@redhat.com>
Date:   Thu Jan 16 16:14:36 2014 -0800

    cluster/afr: refactor
    
    - Remove client side self-healing completely (opendir, openfd, lookup)
    - Re-work readdir-failover to work reliably in case of NFS
    - Remove unused/dead lock recovery code
    - Consistently use xdata in both calls and callbacks in all FOPs
    - Per-inode event generation, used to force inode ctx refresh
    - Implement dirty flag support (in place of pending counts)
    - Eliminate inode ctx structure, use read subvol bits + event_generation
    - Implement inode ctx refreshing based on event generation
    - Provide backward compatibility in transactions
    - remove unused variables and functions
    - make code more consistent in style and pattern
    - regularize and clean up inode-write transaction code
    - regularize and clean up dir-write transaction code
    - regularize and clean up common FOPs
    - reorganize transaction framework code
    - skip setting xattrs in pending dict if nothing is pending
    - re-write self-healing code using syncops
    - re-write simpler self-heal-daemon
    
    Change-Id: I1e4080c9796c8a2815c2dab4be3073f389d614a8
    BUG: 1021686
    Signed-off-by: Anand Avati <avati@redhat.com>
    Reviewed-on: http://review.gluster.org/6010
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Comment 84 Pranith Kumar K 2014-07-13 03:08:09 EDT
*** Bug 951800 has been marked as a duplicate of this bug. ***
Comment 85 Niels de Vos 2014-09-22 08:32:25 EDT
A beta release for GlusterFS 3.6.0 has been released. Please verify if the release solves this bug report for you. In case the glusterfs-3.6.0beta1 release does not have a resolution for this issue, leave a comment in this bug and move the status to ASSIGNED. If this release fixes the problem for you, leave a note and change the status to VERIFIED.

Packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update (possibly an "updates-testing" repository) infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-September/018836.html
[2] http://supercolony.gluster.org/pipermail/gluster-users/
Comment 86 Niels de Vos 2014-11-11 03:24:16 EST
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.6.1, please reopen this bug report.

glusterfs-3.6.1 has been announced [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-November/019410.html
[2] http://supercolony.gluster.org/mailman/listinfo/gluster-users

Note You need to log in before you can comment on or make changes to this bug.