Created attachment 559625 [details] split brain Description of problem: Mount logs are showing split brain messages while running sanity for stripe-replicate fuse mount Version-Release number of selected component (if applicable): Mainline How reproducible: Steps to Reproduce: 1. created a 2x2 stripe-replicate volume 2. Ran the sanity Actual results: After some time mount point said "transport end point not connected". tests failed Expected results: Additional info: Attached the logs =================== [2012-02-06 04:49:42.811130] I [afr-self-heal-common.c:2022:afr_self_heal_completion_cbk] 0-stripe-rep-replicate-0: background entry self-heal completed on /run27521/p8/d1 [2012-02-06 04:49:42.813440] W [client3_1-fops.c:554:client3_1_rmdir_cbk] 0-stripe-rep-client-2: remote operation failed: Directory not empty ...skipping... [2012-02-06 04:49:50.908865] I [afr-self-heal-common.c:908:afr_sh_missing_entries_done] 0-stripe-rep-replicate-1: split brain found, aborting selfheal of /run27521/pc/d2 [2012-02-06 04:49:50.908884] E [afr-self-heal-common.c:2019:afr_self_heal_completion_cbk] 0-stripe-rep-replicate-1: background gfid self-heal failed on /run27521/pc/d2 [2012-02-06 04:49:50.909437] W [client3_1-fops.c:2287:client3_1_lookup_cbk] 0-stripe-rep-client-0: remote operation failed: Invalid argument. Path: /run27521/pc/d2/c4 [2012-02-06 04:49:50.909456] E [afr-self-heal-common.c:998:afr_sh_common_lookup_resp_handler] 0-stripe-rep-replicate-0: path /run27521/pc/d2/c4 on subvolume stripe-rep-client-0 => -1 (Invalid argument) [2012-02-06 04:49:50.909476] W [client3_1-fops.c:2287:client3_1_lookup_cbk] 0-stripe-rep-client-1: remote operation failed: Invalid argument. Path: /run27521/pc/d2/c4 [2012-02-06 04:49:50.909487] E [afr-self-heal-common.c:998:afr_sh_common_lookup_resp_handler] 0-stripe-rep-replicate-0: path /run27521/pc/d2/c4 on subvolume stripe-rep-client-1 => -1 (Invalid argument) [2012-02-06 04:49:50.909496] E [afr-self-heal-common.c:1275:afr_sh_common_lookup_cbk] 0-stripe-rep-replicate-0: Failed to lookup /run27521/pc/d2/c4, reason Invalid argument [2012-02-06 04:49:50.910145] W [client3_1-fops.c:2287:client3_1_lookup_cbk] 0-stripe-rep-client-1: remote operation failed: Invalid argument. Path: /run27521/pc/d2/c4 [2012-02-06 04:49:50.910164] E [afr-self-heal-common.c:998:afr_sh_common_lookup_resp_handler] 0-stripe-rep-replicate-0: path /run27521/pc/d2/c4 on subvolume stripe-rep-client-1 => -1 (Invalid argument) [2012-02-06 04:49:50.910185] W [client3_1-fops.c:2287:client3_1_lookup_cbk] 0-stripe-rep-client-0: remote operation failed: Invalid argument. Path: /run27521/pc/d2/c4 [2012-02-06 04:49:50.910196] E [afr-self-heal-common.c:998:afr_sh_common_lookup_resp_handler] 0-stripe-rep-replicate-0: path /run27521/pc/d2/c4 on subvolume stripe-rep-client-0 => -1 (Invalid argument) [2012-02-06 04:49:50.910205] E [afr-self-heal-common.c:1275:afr_sh_common_lookup_cbk] 0-stripe-rep-replicate-0: Failed to lookup /run27521/pc/d2/c4, reason Invalid argument [2012-02-06 04:49:50.910714] E [afr-self-heal-common.c:2019:afr_self_heal_completion_cbk] 0-stripe-rep-replicate-0: background entry self-heal failed on /run27521/pc/d2 [2012-02-06 04:49:50.910756] W [fuse-bridge.c:271:fuse_entry_cbk] 0-glusterfs-fuse: 143839: LOOKUP() /run27521/pc/d2 => -1 (No data available)
It is not a split-brain. The log needs to be fixed.
CHANGE: http://review.gluster.com/3039 (cluster/afr: Fix the split-brain log) merged in master by Vijay Bellur (vijay)
CHANGE: http://review.gluster.com/3041 (cluster/afr: Fix split-brain log) merged in master by Vijay Bellur (vijay)
No log messages are seen now on 3.3.0qa41