Bug 1454687 - gluster-blockd crashes when one of the config nodes is not reachable
Summary: gluster-blockd crashes when one of the config nodes is not reachable
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: gluster-block
Version: rhgs-3.3
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: RHGS 3.3.0
Assignee: Pranith Kumar K
QA Contact: surabhi
URL:
Whiteboard:
Depends On:
Blocks: 1417151
TreeView+ depends on / blocked
 
Reported: 2017-05-23 10:37 UTC by Pranith Kumar K
Modified: 2017-09-21 04:19 UTC (History)
3 users (show)

Fixed In Version: gluster-block-0.2.1-1.el7rhgs
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-09-21 04:19:33 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2017:2773 0 normal SHIPPED_LIVE new packages: gluster-block 2017-09-21 08:16:22 UTC

Description Pranith Kumar K 2017-05-23 10:37:28 UTC
Description of problem:
When I executed the following command with 192.168.122.113 gluster-blockd down:
gluster-block create r3/3 ha 3 192.168.122.61,192.168.122.123,192.168.122.113 1GiB

gluster-blockd crashes with following trace:
=================================================================
==2422==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7f8e2f27e046 bp 0x7f8e2b5fe300 sp 0x7f8e2b5fda68 T1)
    #0 0x7f8e2f27e045 in strlen (/lib64/libc.so.6+0x8d045)
    #1 0x7f8e307d8f0b in xdr_string (/lib64/libasan.so.3+0x98f0b)
    #2 0x405555 in xdr_blockResponse /root/gluster-block/rpc/rpcl/block_xdr.c:188
    #3 0x7f8e2f32db1f in xdr_union (/lib64/libc.so.6+0x13cb1f)
    #4 0x7f8e2f325673 in svcunix_reply (/lib64/libc.so.6+0x134673)
    #5 0x7f8e2f32adea in svc_sendreply (/lib64/libc.so.6+0x139dea)
    #6 0x404be8 in gluster_block_cli_1 /root/gluster-block/rpc/rpcl/block_svc.c:133
    #7 0x7f8e2f32b2a0 in svc_getreq_common (/lib64/libc.so.6+0x13a2a0)
    #8 0x7f8e2f32b3e6 in svc_getreq_poll (/lib64/libc.so.6+0x13a3e6)
    #9 0x7f8e2f32ed00 in svc_run (/lib64/libc.so.6+0x13dd00)
    #10 0x4036ef in glusterBlockCliThreadProc /root/gluster-block/daemon/gluster-blockd.c:101
    #11 0x7f8e305296c9 in start_thread (/lib64/libpthread.so.0+0x76c9)
    #12 0x7f8e2f2f8f6e in clone (/lib64/libc.so.6+0x107f6e)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (/lib64/libc.so.6+0x8d045) in strlen
Thread T1 created by T0 here:
    #0 0x7f8e30771488 in __interceptor_pthread_create (/lib64/libasan.so.3+0x31488)
    #1 0x404296 in main /root/gluster-block/daemon/gluster-blockd.c:212
    #2 0x7f8e2f211400 in __libc_start_main (/lib64/libc.so.6+0x20400)


This trace is similar to:
https://bugzilla.redhat.com/show_bug.cgi?id=1452919#c2

RC is that errcode will be -1 but errMsg is NULL and since metadatainfo won't be present it will goto out and reply will have all 0s except errCode and when rpc tries to marshal this as response it crashes.


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 4 Pranith Kumar K 2017-06-04 17:25:33 UTC
https://review.gluster.org/17430

Comment 6 surabhi 2017-06-20 12:58:14 UTC
If gluster-blockd is not running on config nodes and we try to create block using those nodes , it gives proper error message and there is no crash seen.

in the following case, gluster-blockd was not running on two of the nodes so it did not created the block and gave proper message. No crash seen.
gluster-block create blockstor/goodys ha 3 auth enable 10.70.46.152,10.70.46.144,10.70.46.151 1GiB
failed to configure on 10.70.46.144 : Connection refused
failed to configure on 10.70.46.151 : Connection refused
RESULT:FAIL

Moving the BZ to verified.

gluster-block-0.2.1-1.el7rhgs.x86_64
tcmu-runner-1.2.0-4.el7rhgs.x86_64

Comment 8 errata-xmlrpc 2017-09-21 04:19:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:2773


Note You need to log in before you can comment on or make changes to this bug.