Bug 1453179 - gluster-blockd exits on SIGPIPE
Summary: gluster-blockd exits on SIGPIPE
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: gluster-block
Version: rhgs-3.3
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: RHGS 3.3.0
Assignee: Prasanna Kumar Kalever
QA Contact: Sweta Anandpara
URL:
Whiteboard:
Depends On:
Blocks: 1417151
TreeView+ depends on / blocked
 
Reported: 2017-05-22 10:28 UTC by Pranith Kumar K
Modified: 2017-09-21 04:19 UTC (History)
5 users (show)

Fixed In Version: gluster-block-0.2.1-1.el7rhgs
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-09-21 04:19:33 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2017:2773 0 normal SHIPPED_LIVE new packages: gluster-block 2017-09-21 08:16:22 UTC

Description Pranith Kumar K 2017-05-22 10:28:26 UTC
Description of problem:
If we execute a command like gluster-block create/delete and quickly do a ctrl+c, after gluster-blockd realizes that the client disconnected, it dies on SIGPIPE error:

Thread 2 "gluster-blockd" received signal SIGPIPE, Broken pipe.
[Switching to Thread 0x7ffff1cff700 (LWP 15343)]
0x00007ffff59ef070 in sendmsg () from /lib64/libc.so.6
Missing separate debuginfos, use: dnf debuginfo-install glusterfs-3.10.1-1.fc25.x86_64 glusterfs-api-3.10.1-1.fc25.x86_64 glusterfs-client-xlators-3.10.1-1.fc25.x86_64 glusterfs-libs-3.10.1-1.fc25.x86_64 json-c-0.12-7.fc24.x86_64 keyutils-libs-1.5.9-8.fc24.x86_64 krb5-libs-1.14.4-4.fc25.x86_64 libacl-2.2.52-11.fc24.x86_64 libasan-6.3.1-1.fc25.x86_64 libattr-2.4.47-16.fc24.x86_64 libcom_err-1.43.1-2.fc25.x86_64 libgcc-6.3.1-1.fc25.x86_64 libselinux-2.5-12.fc25.x86_64 libstdc++-6.2.1-2.fc25.x86_64 libuuid-2.28.2-1.fc25.x86_64 openssl-libs-1.0.2k-1.fc25.x86_64 pcre-8.39-6.fc25.x86_64 sssd-client-1.14.2-1.fc25.x86_64 zlib-1.2.8-10.fc24.x86_64
(gdb) c
Continuing.
Couldn't get registers: No such process.
Couldn't get registers: No such process.
Couldn't get registers: No such process.
(gdb) [Thread 0x7fffef9fc700 (LWP 15347) exited]
[Thread 0x7ffff01fd700 (LWP 15346) exited]
[Thread 0x7ffff14fe700 (LWP 15344) exited]
[Thread 0x7ffff1cff700 (LWP 15343) exited]

Program terminated with signal SIGPIPE, Broken pipe.
The program no longer exists.
Quit
(gdb) q
[root@localhost ~]# 


Version-Release number of selected component (if applicable):


How reproducible:
always

Steps to Reproduce:
1.
2.
3.

Actual results:
Process exits

Expected results:
Process should not exit

Additional info:

Comment 2 Pranith Kumar K 2017-05-22 10:32:29 UTC
Prasanna,
    In glusterfsd, we avoid this with:
signal (SIGPIPE, SIG_IGN);

check:
glusterfs_signals_setup()

Comment 11 Sweta Anandpara 2017-07-20 09:06:55 UTC
Tested and verified this on the build glusterfs-3.8.4-33 and gluster-block-0.2.1-6. 

Executed multiple rounds of gluster-block create/modify/delete with pressing Ctrl+c soon after executing the command. The cli returned control immediately, with no result printed on CLI. The internal daemon however did what was expected, wrt creates/modify and delete. Validated with the meta information as well, of the respective blocks.

Logs are pasted below. Moving this to verified in 3.3.

[root@dhcp47-114 ~]# gluster v info testvol
 
Volume Name: testvol
Type: Replicate
Volume ID: 35a0b1a7-0dc3-4536-96aa-bd181b91c381
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.47.121:/bricks/brick2/testvol0
Brick2: 10.70.47.113:/bricks/brick2/testvol1
Brick3: 10.70.47.114:/bricks/brick2/testvol2
Options Reconfigured:
nfs.disable: on
transport.address-family: inet
performance.quick-read: off
performance.read-ahead: off
performance.io-cache: off
performance.stat-prefetch: off
performance.open-behind: off
performance.readdir-ahead: off
network.remote-dio: enable
cluster.eager-lock: enable
cluster.quorum-type: auto
cluster.data-self-heal-algorithm: full
cluster.locking-scheme: granular
cluster.shd-max-threads: 8
cluster.shd-wait-qlength: 10000
features.shard: on
user.cifs: off
server.allow-insecure: on
cluster.brick-multiplex: disable
cluster.enable-shared-storage: enable
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# gluster-block create testvol/bk2 ha 1 10.70.47.114 1M
IQN: iqn.2016-12.org.gluster-block:0288af11-603a-42f7-896f-d2fc498b900f
PORTAL(S):  10.70.47.114:3260
RESULT: SUCCESS
[root@dhcp47-114 ~]# gluster-block info testvol/bk2
NAME: bk2
VOLUME: testvol
GBID: 0288af11-603a-42f7-896f-d2fc498b900f
SIZE: 1048576
HA: 1
PASSWORD: 
BLOCK CONFIG NODE(S): 10.70.47.114
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# gluster-block create testvol/bk3 ha 2 10.70.47.114,10.70.47.113 1M
IQN: iqn.2016-12.org.gluster-block:54b06dc2-8961-4a1f-b107-d8b68c93db2d
PORTAL(S):  10.70.47.114:3260 10.70.47.113:3260
RESULT: SUCCESS
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# gluster-block info testvol/bk3
NAME: bk3
VOLUME: testvol
GBID: 54b06dc2-8961-4a1f-b107-d8b68c93db2d
SIZE: 1048576
HA: 2
PASSWORD: 
BLOCK CONFIG NODE(S): 10.70.47.113 10.70.47.114
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# gluster-block create testvol/bk4 auth enable ha 2 10.70.47.114,10.70.47.113 1M
failed to configure on ha : Success
RESULT:FAIL
[root@dhcp47-114 ~]# gluster-block create
Inadequate arguments for create:
gluster-block create <volname/blockname> [ha <count>] [auth enable|disable] <HOST1[,HOST2,...]> <size> [--json*]
[root@dhcp47-114 ~]# gluster-block create testvol/bk4  ha 2 auth enable 10.70.47.114,10.70.47.113 1M
^C
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# gluster-block list testvol
testblock
bk1
bk2
bk3
bk4
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# mkdir /mnt/testvol
[root@dhcp47-114 ~]# mount -t glusterfs 10.70.47.114:testvol /mnt/testvol
[root@dhcp47-114 ~]# cd /mnt/testvol
[root@dhcp47-114 testvol]# 
[root@dhcp47-114 testvol]# 
[root@dhcp47-114 testvol]# ls
block-meta  block-store
[root@dhcp47-114 testvol]# cd block-meta/
[root@dhcp47-114 block-meta]# ls
bk1  bk2  bk3  bk4  meta.lock  testblock
[root@dhcp47-114 block-meta]# cat bk3
VOLUME: testvol
GBID: 54b06dc2-8961-4a1f-b107-d8b68c93db2d
SIZE: 1048576
HA: 2
ENTRYCREATE: INPROGRESS
ENTRYCREATE: SUCCESS
10.70.47.113: CONFIGINPROGRESS
10.70.47.114: CONFIGINPROGRESS
10.70.47.114: CONFIGSUCCESS
10.70.47.113: CONFIGSUCCESS
[root@dhcp47-114 block-meta]# bk4
-bash: bk4: command not found
[root@dhcp47-114 block-meta]# cat bk4
VOLUME: testvol
GBID: 626a71ca-2372-420b-b0b4-b449fa2d6f88
SIZE: 1048576
HA: 2
ENTRYCREATE: INPROGRESS
ENTRYCREATE: SUCCESS
PASSWORD: c18af2c1-fc49-4ec5-bdd4-a1b3b358e905
10.70.47.114: CONFIGINPROGRESS
10.70.47.113: CONFIGINPROGRESS
10.70.47.114: CONFIGSUCCESS
10.70.47.114: AUTHENFORCED
10.70.47.113: CONFIGSUCCESS
10.70.47.113: AUTHENFORCED
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# gluster-block info testvol/bk4
NAME: bk4
VOLUME: testvol
GBID: 626a71ca-2372-420b-b0b4-b449fa2d6f88
SIZE: 1048576
HA: 2
PASSWORD: c18af2c1-fc49-4ec5-bdd4-a1b3b358e905
BLOCK CONFIG NODE(S): 10.70.47.114 10.70.47.113
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# gluster-block delete testvol/bk3
^C
[root@dhcp47-114 block-meta]# gluster-block list testvol
testblock
bk1
bk2
bk4
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# ls
bk1  bk2  bk4  meta.lock  testblock
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# gluster-block modify testvol/bk2 auth enable
^C
[root@dhcp47-114 block-meta]# gluster-block list testvol
testblock
bk1
bk2
bk4
[root@dhcp47-114 block-meta]# gluster-block info testvol/bk2
NAME: bk2
VOLUME: testvol
GBID: 0288af11-603a-42f7-896f-d2fc498b900f
SIZE: 1048576
HA: 1
PASSWORD: 8ed48837-f22b-43f1-911c-6e635ca7a711
BLOCK CONFIG NODE(S): 10.70.47.114
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# 
[root@dhcp47-114 block-meta]# ls
bk1  bk2  bk4  meta.lock  testblock
[root@dhcp47-114 block-meta]# cat bk2
VOLUME: testvol
GBID: 0288af11-603a-42f7-896f-d2fc498b900f
SIZE: 1048576
HA: 1
ENTRYCREATE: INPROGRESS
ENTRYCREATE: SUCCESS
10.70.47.114: CONFIGINPROGRESS
10.70.47.114: CONFIGSUCCESS
PASSWORD: 8ed48837-f22b-43f1-911c-6e635ca7a711
10.70.47.114: AUTHENFORCEING
10.70.47.114: AUTHENFORCED
[root@dhcp47-114 block-meta]# 


[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# rpm -qa | grep gluster
glusterfs-cli-3.8.4-33.el7rhgs.x86_64
glusterfs-rdma-3.8.4-33.el7rhgs.x86_64
python-gluster-3.8.4-33.el7rhgs.noarch
vdsm-gluster-4.17.33-1.1.el7rhgs.noarch
glusterfs-client-xlators-3.8.4-33.el7rhgs.x86_64
glusterfs-fuse-3.8.4-33.el7rhgs.x86_64
gluster-nagios-common-0.2.4-1.el7rhgs.noarch
glusterfs-events-3.8.4-33.el7rhgs.x86_64
gluster-block-0.2.1-6.el7rhgs.x86_64
libvirt-daemon-driver-storage-gluster-3.2.0-14.el7.x86_64
gluster-nagios-addons-0.2.9-1.el7rhgs.x86_64
samba-vfs-glusterfs-4.6.3-3.el7rhgs.x86_64
glusterfs-3.8.4-33.el7rhgs.x86_64
glusterfs-debuginfo-3.8.4-26.el7rhgs.x86_64
glusterfs-api-3.8.4-33.el7rhgs.x86_64
glusterfs-geo-replication-3.8.4-33.el7rhgs.x86_64
glusterfs-libs-3.8.4-33.el7rhgs.x86_64
glusterfs-server-3.8.4-33.el7rhgs.x86_64
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# rpm -qa | grep tcmu-runner
tcmu-runner-1.2.0-9.el7rhgs.x86_64
[root@dhcp47-114 ~]# 
[root@dhcp47-114 ~]# gluster pool list
UUID					Hostname                         	State
49610061-1788-4cbc-9205-0e59fe91d842	dhcp47-121.lab.eng.blr.redhat.com	Connected 
a0557927-4e5e-4ff7-8dce-94873f867707	dhcp47-113.lab.eng.blr.redhat.com	Connected 
f828fdfa-e08f-4d12-85d8-2121cafcf9d0	dhcp47-115.lab.eng.blr.redhat.com	Connected 
a96e0244-b5ce-4518-895c-8eb453c71ded	dhcp47-116.lab.eng.blr.redhat.com	Connected 
17eb3cef-17e7-4249-954b-fc19ec608304	dhcp47-117.lab.eng.blr.redhat.com	Connected 
c0dac197-5a4d-4db7-b709-dbf8b8eb0896	localhost                        	Connected 
[root@dhcp47-114 ~]#

Comment 13 errata-xmlrpc 2017-09-21 04:19:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:2773


Note You need to log in before you can comment on or make changes to this bug.