Bug 1453179

Summary: gluster-blockd exits on SIGPIPE
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Pranith Kumar K <pkarampu>
Component: gluster-blockAssignee: Prasanna Kumar Kalever <prasanna.kalever>
Status: CLOSED ERRATA QA Contact: Sweta Anandpara <sanandpa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rhgs-3.3CC: amukherj, prasanna.kalever, rhs-bugs, sanandpa, storage-qa-internal
Target Milestone: ---   
Target Release: RHGS 3.3.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: gluster-block-0.2.1-1.el7rhgs Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-09-21 04:19:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1417151    

Description Pranith Kumar K 2017-05-22 10:28:26 UTC 
Description of problem:
If we execute a command like gluster-block create/delete and quickly do a ctrl+c, after gluster-blockd realizes that the client disconnected, it dies on SIGPIPE error:

Thread 2 "gluster-blockd" received signal SIGPIPE, Broken pipe.
[Switching to Thread 0x7ffff1cff700 (LWP 15343)]
0x00007ffff59ef070 in sendmsg () from /lib64/libc.so.6
Missing separate debuginfos, use: dnf debuginfo-install glusterfs-3.10.1-1.fc25.x86_64 glusterfs-api-3.10.1-1.fc25.x86_64 glusterfs-client-xlators-3.10.1-1.fc25.x86_64 glusterfs-libs-3.10.1-1.fc25.x86_64 json-c-0.12-7.fc24.x86_64 keyutils-libs-1.5.9-8.fc24.x86_64 krb5-libs-1.14.4-4.fc25.x86_64 libacl-2.2.52-11.fc24.x86_64 libasan-6.3.1-1.fc25.x86_64 libattr-2.4.47-16.fc24.x86_64 libcom_err-1.43.1-2.fc25.x86_64 libgcc-6.3.1-1.fc25.x86_64 libselinux-2.5-12.fc25.x86_64 libstdc++-6.2.1-2.fc25.x86_64 libuuid-2.28.2-1.fc25.x86_64 openssl-libs-1.0.2k-1.fc25.x86_64 pcre-8.39-6.fc25.x86_64 sssd-client-1.14.2-1.fc25.x86_64 zlib-1.2.8-10.fc24.x86_64
(gdb) c
Continuing.
Couldn't get registers: No such process.
Couldn't get registers: No such process.
Couldn't get registers: No such process.
(gdb) [Thread 0x7fffef9fc700 (LWP 15347) exited]
[Thread 0x7ffff01fd700 (LWP 15346) exited]
[Thread 0x7ffff14fe700 (LWP 15344) exited]
[Thread 0x7ffff1cff700 (LWP 15343) exited]

Program terminated with signal SIGPIPE, Broken pipe.
The program no longer exists.
Quit
(gdb) q
[root@localhost ~]# 


Version-Release number of selected component (if applicable):


How reproducible:
always

Steps to Reproduce:
1.
2.
3.

Actual results:
Process exits

Expected results:
Process should not exit

Additional info:
Comment 2 Pranith Kumar K 2017-05-22 10:32:29 UTC 
Prasanna,
    In glusterfsd, we avoid this with:
signal (SIGPIPE, SIG_IGN);

check:
glusterfs_signals_setup()
Comment 11 Sweta Anandpara 2017-07-20 09:06:55 UTC 
Tested and verified this on the build glusterfs-3.8.4-33 and gluster-block-0.2.1-6. 

Executed multiple rounds of gluster-block create/modify/delete with pressing Ctrl+c soon after executing the command. The cli returned control immediately, with no result printed on CLI. The internal daemon however did what was expected, wrt creates/modify and delete. Validated with the meta information as well, of the respective blocks.

Logs are pasted below. Moving this to verified in 3.3.

[root@dhcp47-114 ~]# gluster v info testvol
 
Volume Name: testvol
Type: Replicate
Volume ID: 35a0b1a7-0dc3-4536-96aa-bd181b91c381
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.47.121:/bricks/brick2/testvol0
Brick2: 10.70.47.113:/bricks/brick2/testvol1
Brick3: 10.70.47.114:/bricks/brick2/testvol2
Options Reconfigured:
nfs.disable: on
transport.address-family: inet
performance.quick-read: off
performance.read-ahead: off
performance.io-cache: off
performance.stat-prefetch: off
performance.open-behind: off
performance.readdir-ahead: off
network.remote-dio: enable
cluster.eager-lock: enable
cluster.quorum-type: auto
cluster.data-self-heal-algorithm: full
cluster.locking-scheme: granular
cluster.shd-max-threads: 8
cluster.shd-wait-qlength: 10000
features.shard: on
user.cifs: off
server.allow-insecure: on
cluster.brick-multiplex: disable
cluster.enable-shared-storage: enable
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# gluster-block create testvol/bk2 ha 1 10.70.47.114 1M
IQN: iqn.2016-12.org.gluster-block:0288af11-603a-42f7-896f-d2fc498b900f
PORTAL(S):  10.70.47.114:3260
RESULT: SUCCESS
[root@dhcp47-114 ~]# gluster-block info testvol/bk2
NAME: bk2
VOLUME: testvol
GBID: 0288af11-603a-42f7-896f-d2fc498b900f
SIZE: 1048576
HA: 1
PASSWORD: 
BLOCK CONFIG NODE(S): 10.70.47.114
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# gluster-block create testvol/bk3 ha 2 10.70.47.114,10.70.47.113 1M
IQN: iqn.2016-12.org.gluster-block:54b06dc2-8961-4a1f-b107-d8b68c93db2d
PORTAL(S):  10.70.47.114:3260 10.70.47.113:3260
RESULT: SUCCESS
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# gluster-block info testvol/bk3
NAME: bk3
VOLUME: testvol
GBID: 54b06dc2-8961-4a1f-b107-d8b68c93db2d
SIZE: 1048576
HA: 2
PASSWORD: 
BLOCK CONFIG NODE(S): 10.70.47.113 10.70.47.114
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# gluster-block create testvol/bk4 auth enable ha 2 10.70.47.114,10.70.47.113 1M
failed to configure on ha : Success
RESULT:FAIL
[root@dhcp47-114 ~]# gluster-block create
Inadequate arguments for create:
gluster-block create <volname/blockname> [ha <count>] [auth enable|disable] <HOST1[,HOST2,...]> <size> [--json*]
[root@dhcp47-114 ~]# gluster-block create testvol/bk4  ha 2 auth enable 10.70.47.114,10.70.47.113 1M
^C
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# gluster-block list testvol
testblock
bk1
bk2
bk3
bk4
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# mkdir /mnt/testvol
[root@dhcp47-114 ~]# mount -t glusterfs 10.70.47.114:testvol /mnt/testvol
[root@dhcp47-114 ~]# cd /mnt/testvol
[root@dhcp47-114 testvol]# 
[root@dhcp47-114 testvol]# 
[root@dhcp47-114 testvol]# ls
block-meta  block-store
[root@dhcp47-114 testvol]# cd block-meta/
[root@dhcp47-114 block-meta]# ls
bk1  bk2  bk3  bk4  meta.lock  testblock
[root@dhcp47-114 block-meta]# cat bk3
VOLUME: testvol
GBID: 54b06dc2-8961-4a1f-b107-d8b68c93db2d
SIZE: 1048576
HA: 2
ENTRYCREATE: INPROGRESS
ENTRYCREATE: SUCCESS
10.70.47.113: CONFIGINPROGRESS
10.70.47.114: CONFIGINPROGRESS
10.70.47.114: CONFIGSUCCESS
10.70.47.113: CONFIGSUCCESS
[root@dhcp47-114 block-meta]# bk4
-bash: bk4: command not found
[root@dhcp47-114 block-meta]# cat bk4
VOLUME: testvol
GBID: 626a71ca-2372-420b-b0b4-b449fa2d6f88
SIZE: 1048576
HA: 2
ENTRYCREATE: INPROGRESS
ENTRYCREATE: SUCCESS
PASSWORD: c18af2c1-fc49-4ec5-bdd4-a1b3b358e905
10.70.47.114: CONFIGINPROGRESS
10.70.47.113: CONFIGINPROGRESS
10.70.47.114: CONFIGSUCCESS
10.70.47.114: AUTHENFORCED
10.70.47.113: CONFIGSUCCESS
10.70.47.113: AUTHENFORCED
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# gluster-block info testvol/bk4
NAME: bk4
VOLUME: testvol
GBID: 626a71ca-2372-420b-b0b4-b449fa2d6f88
SIZE: 1048576
HA: 2
PASSWORD: c18af2c1-fc49-4ec5-bdd4-a1b3b358e905
BLOCK CONFIG NODE(S): 10.70.47.114 10.70.47.113
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# gluster-block delete testvol/bk3
^C
[root@dhcp47-114 block-meta]# gluster-block list testvol
testblock
bk1
bk2
bk4
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# ls
bk1  bk2  bk4  meta.lock  testblock
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# gluster-block modify testvol/bk2 auth enable
^C
[root@dhcp47-114 block-meta]# gluster-block list testvol
testblock
bk1
bk2
bk4
[root@dhcp47-114 block-meta]# gluster-block info testvol/bk2
NAME: bk2
VOLUME: testvol
GBID: 0288af11-603a-42f7-896f-d2fc498b900f
SIZE: 1048576
HA: 1
PASSWORD: 8ed48837-f22b-43f1-911c-6e635ca7a711
BLOCK CONFIG NODE(S): 10.70.47.114
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# ls
bk1  bk2  bk4  meta.lock  testblock
[root@dhcp47-114 block-meta]# cat bk2
VOLUME: testvol
GBID: 0288af11-603a-42f7-896f-d2fc498b900f
SIZE: 1048576
HA: 1
ENTRYCREATE: INPROGRESS
ENTRYCREATE: SUCCESS
10.70.47.114: CONFIGINPROGRESS
10.70.47.114: CONFIGSUCCESS
PASSWORD: 8ed48837-f22b-43f1-911c-6e635ca7a711
10.70.47.114: AUTHENFORCEING
10.70.47.114: AUTHENFORCED
[root@dhcp47-114 block-meta]# 


[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# rpm -qa | grep gluster
glusterfs-cli-3.8.4-33.el7rhgs.x86_64
glusterfs-rdma-3.8.4-33.el7rhgs.x86_64
python-gluster-3.8.4-33.el7rhgs.noarch
vdsm-gluster-4.17.33-1.1.el7rhgs.noarch
glusterfs-client-xlators-3.8.4-33.el7rhgs.x86_64
glusterfs-fuse-3.8.4-33.el7rhgs.x86_64
gluster-nagios-common-0.2.4-1.el7rhgs.noarch
glusterfs-events-3.8.4-33.el7rhgs.x86_64
gluster-block-0.2.1-6.el7rhgs.x86_64
libvirt-daemon-driver-storage-gluster-3.2.0-14.el7.x86_64
gluster-nagios-addons-0.2.9-1.el7rhgs.x86_64
samba-vfs-glusterfs-4.6.3-3.el7rhgs.x86_64
glusterfs-3.8.4-33.el7rhgs.x86_64
glusterfs-debuginfo-3.8.4-26.el7rhgs.x86_64
glusterfs-api-3.8.4-33.el7rhgs.x86_64
glusterfs-geo-replication-3.8.4-33.el7rhgs.x86_64
glusterfs-libs-3.8.4-33.el7rhgs.x86_64
glusterfs-server-3.8.4-33.el7rhgs.x86_64
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# rpm -qa | grep tcmu-runner
tcmu-runner-1.2.0-9.el7rhgs.x86_64
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# gluster pool list
UUID					Hostname                         	State
49610061-1788-4cbc-9205-0e59fe91d842	dhcp47-121.lab.eng.blr.redhat.com	Connected 
a0557927-4e5e-4ff7-8dce-94873f867707	dhcp47-113.lab.eng.blr.redhat.com	Connected 
f828fdfa-e08f-4d12-85d8-2121cafcf9d0	dhcp47-115.lab.eng.blr.redhat.com	Connected 
a96e0244-b5ce-4518-895c-8eb453c71ded	dhcp47-116.lab.eng.blr.redhat.com	Connected 
17eb3cef-17e7-4249-954b-fc19ec608304	dhcp47-117.lab.eng.blr.redhat.com	Connected 
c0dac197-5a4d-4db7-b709-dbf8b8eb0896	localhost                        	Connected 
[root@dhcp47-114 ~]#
Comment 13 errata-xmlrpc 2017-09-21 04:19:33 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:2773