Bug 1159119

Summary: glusterd-georep: In multinode slave setup, glusterd crashes during geo-rep create push-pem.
Product: [Community] GlusterFS Reporter: Kotresh HR <khiremat>
Component: glusterdAssignee: Kotresh HR <khiremat>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: unspecified    
Version: mainlineCC: bugs, gluster-bugs
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.7.0 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1159210 (view as bug list) Environment:
Last Closed: 2015-05-14 17:28:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1159210    

Description Kotresh HR 2014-10-31 02:40:27 UTC
Description of problem:
In multinode slave setup, glusterd crashes on slave node other than the one which is used to create geo-rep sessoin during geo-rep create push-pem.


Version-Release number of selected component (if applicable):
mainline master branch.

How reproducible: Always

Steps to Reproduce:
1. Create master and slave volume, with slave volume spread across more than 
   one node.
2. On master: gluster vol geo-rep gsec_create
3. On master: gluster vol geo-rep crate mastervol slave_host:slavevol create push-pem 


Actual results:
glusterd crashed on the slave node other than the one which is used to create goe-rep session.

Expected results:
glusterd should not crash.

Additional info:

Comment 1 Anand Avati 2014-10-31 02:49:54 UTC
REVIEW: http://review.gluster.org/9018 (glusterd/geo-rep: Fix glusterd crash in non-originator slave node.) posted (#1) for review on master by Kotresh HR (khiremat)

Comment 2 Kotresh HR 2014-10-31 02:52:07 UTC
Stack trace of glusterd crash is follows:

#0  0x00007f8cf479ac39 in raise () from /lib64/libc.so.6
#1  0x00007f8cf479c348 in abort () from /lib64/libc.so.6
#2  0x00007f8cf4793b96 in __assert_fail_base () from /lib64/libc.so.6
#3  0x00007f8cf4793c42 in __assert_fail () from /lib64/libc.so.6
#4  0x00007f8cf59b0107 in __gf_free (free_ptr=0x7f8cd40046c0) at mem-pool.c:251
#5  0x00007f8ceb288af7 in glusterd_op_copy_file (dict=dict@entry=0x7f8cd40010dc, op_errstr=op_errstr@entry=0xf0a5f8) at glusterd-geo-rep.c:4444
#6  0x00007f8ceb236e3b in glusterd_op_commit_perform (op=GD_OP_COPY_FILE, dict=dict@entry=0x7f8cd40010dc, op_errstr=op_errstr@entry=0xf0a5f8, 
    rsp_dict=rsp_dict@entry=0x7f8cd4000f6c) at glusterd-op-sm.c:4940
#7  0x00007f8ceb23d25c in glusterd_op_ac_commit_op (event=0x7f8cd4000bc0, ctx=0x7f8cd4002850) at glusterd-op-sm.c:4640
#8  0x00007f8ceb23b649 in glusterd_op_sm () at glusterd-op-sm.c:6575
#9  0x00007f8ceb229456 in __glusterd_handle_commit_op (req=req@entry=0xd06d7c) at glusterd-handler.c:943
#10 0x00007f8ceb2280b0 in glusterd_big_locked_handler (req=0xd06d7c, actor_fn=0x7f8ceb229350 <__glusterd_handle_commit_op>) at glusterd-handler.c:79
#11 0x00007f8cf59bd342 in synctask_wrap (old_task=<optimized out>) at syncop.c:335
#12 0x00007f8cf47aca00 in ?? () from /lib64/libc.so.6
#13 0x0000000000000000 in ?? ()

Comment 3 Anand Avati 2014-11-03 04:56:00 UTC
COMMIT: http://review.gluster.org/9018 committed in master by Krishnan Parthasarathi (kparthas) 
------
commit adfccb57565ff1af0bbb79efc3cf489c55509aba
Author: Kotresh HR <khiremat>
Date:   Fri Oct 31 07:06:52 2014 +0530

    glusterd/geo-rep: Fix glusterd crash in non-originator slave node.
    
    Problem: glusterd crashes in non-originator slave node during geo-rep
             create push-pem.
    
    Cause: In glusterd_op_copy_file, the value of the key "common_pem_contents"
           is freed explicitly even after dict_set is successful when it is
           taken cared by dict_free.
    
    Solution: Free only in failure cases before dict_set.
    
    Change-Id: I65b5f32ee2b946107ad279b1fe3d728ec699bc7e
    BUG: 1159119
    Signed-off-by: Kotresh HR <khiremat>
    Reviewed-on: http://review.gluster.org/9018
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Poornima G <pgurusid>
    Reviewed-by: Krishnan Parthasarathi <kparthas>
    Tested-by: Krishnan Parthasarathi <kparthas>

Comment 4 Niels de Vos 2015-05-14 17:28:15 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Comment 5 Niels de Vos 2015-05-14 17:35:41 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Comment 6 Niels de Vos 2015-05-14 17:38:03 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Comment 7 Niels de Vos 2015-05-14 17:44:21 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user