Bug 1463173

Summary: [gluster-block]: Block create command fails on rhel7.4 with python exception
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Sweta Anandpara <sanandpa>
Component: gluster-blockAssignee: Prasanna Kumar Kalever <prasanna.kalever>
Status: CLOSED ERRATA QA Contact: Rahul Hinduja <rhinduja>
Severity: high Docs Contact:
Priority: unspecified    
Version: rhgs-3.3CC: amukherj, rhs-bugs, sanandpa, sbhaloth, storage-qa-internal
Target Milestone: ---   
Target Release: RHGS 3.3.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-09-21 04:19:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1417151    

Description Sweta Anandpara 2017-06-20 10:06:01 UTC 
Description of problem:
========================
Created a 1*3 volume 'testvol' on rhel7.4 and executed the command to create a block. It failed. Messages in logs, and traceback pasted below.

[2017-06-20 09:14:42.215485] ERROR: failed dependency, check if you have targetcli and tcmu-runner installed [No such device] hence create block testblock on host 10.70.47.162 volume (null) failed [at block_svc_routines.c+516 :<glusterBlockCreateRemote>]
[2017-06-20 09:14:42.285179] ERROR: tcmu-runner running, but targetcli doesn't list user:glfs handler [at block_svc_routines.c+1884 :<blockNodeSanityCheck>]
[2017-06-20 09:14:42.285640] ERROR: failed dependency, check if you have targetcli and tcmu-runner installed [No such device] hence create block testblock on host 10.70.47.165 volume (null) failed [at block_svc_routines.c+516 :<glusterBlockCreateRemote>]
[2017-06-20 09:14:42.285828] WARNING: glusterBlockCreateRemoteAsync: return -1 failed in remote async create for block testblock on volume testvol with hosts 10.70.47.165,10.70.47.164,10.70.47.162 [at block_svc_routines.c+1828 :<block_create_cli_1_svc>]


Jun 20 05:03:50 dhcp47-165 python: detected unhandled Python exception in '/usr/bin/targetcli'
Jun 20 05:03:50 dhcp47-165 abrt-server: Package 'targetcli' isn't signed with proper key
Jun 20 05:03:50 dhcp47-165 abrt-server: 'post-create' on '/var/spool/abrt/Python-2017-06-20-05:03:50-2959' exited with 1
Jun 20 05:03:50 dhcp47-165 abrt-server: Deleting problem directory '/var/spool/abrt/Python-2017-06-20-05:03:50-2959'
Jun 20 05:03:50 dhcp47-165 gluster-blockd: Traceback (most recent call last):
Jun 20 05:03:50 dhcp47-165 gluster-blockd: File "/usr/bin/targetcli", line 24, in <module>
Jun 20 05:03:50 dhcp47-165 gluster-blockd: from targetcli import UIRoot
Jun 20 05:03:50 dhcp47-165 gluster-blockd: File "/usr/lib/python2.7/site-packages/targetcli/__init__.py", line 18, in <module>
Jun 20 05:03:50 dhcp47-165 gluster-blockd: from .ui_root import UIRoot
Jun 20 05:03:50 dhcp47-165 gluster-blockd: File "/usr/lib/python2.7/site-packages/targetcli/ui_root.py", line 26, in <module>
Jun 20 05:03:50 dhcp47-165 gluster-blockd: from configshell_fb import ExecutionError
Jun 20 05:03:50 dhcp47-165 gluster-blockd: File "/usr/lib/python2.7/site-packages/configshell_fb/__init__.py", line 27, in <module>
Jun 20 05:03:50 dhcp47-165 gluster-blockd: from .shell import ConfigShell
Jun 20 05:03:50 dhcp47-165 gluster-blockd: File "/usr/lib/python2.7/site-packages/configshell_fb/shell.py", line 53, in <module>
Jun 20 05:03:50 dhcp47-165 gluster-blockd: oldTerm = os.environ['TERM']
Jun 20 05:03:50 dhcp47-165 gluster-blockd: File "/usr/lib64/python2.7/UserDict.py", line 23, in __getitem__
Jun 20 05:03:50 dhcp47-165 gluster-blockd: raise KeyError(key)
Jun 20 05:03:50 dhcp47-165 gluster-blockd: KeyError: 'TERM'


Additional info:
================

[root@dhcp47-165 ~]# 
[root@dhcp47-165 ~]# rpm -qa | grep gluster-block
gluster-block-0.2.1-1.el7rhgs.x86_64
[root@dhcp47-165 ~]# 
[root@dhcp47-165 ~]# rpm -qa | grep glusterfs
glusterfs-fuse-3.8.4-28.el7rhgs.x86_64
glusterfs-client-xlators-3.8.4-28.el7rhgs.x86_64
samba-vfs-glusterfs-4.6.3-1.el7rhgs.x86_64
glusterfs-libs-3.8.4-28.el7rhgs.x86_64
glusterfs-server-3.8.4-28.el7rhgs.x86_64
glusterfs-events-3.8.4-28.el7rhgs.x86_64
glusterfs-api-3.8.4-28.el7rhgs.x86_64
glusterfs-geo-replication-3.8.4-28.el7rhgs.x86_64
glusterfs-rdma-3.8.4-28.el7rhgs.x86_64
glusterfs-3.8.4-28.el7rhgs.x86_64
glusterfs-cli-3.8.4-28.el7rhgs.x86_64
[root@dhcp47-165 ~]# 
[root@dhcp47-165 ~]# gluster pool list
UUID					Hostname                         	State
afa697a0-2cc6-4705-892e-f5ec56a9f9de	dhcp47-164.lab.eng.blr.redhat.com	Connected 
95491d39-d83a-4053-b1d5-682ca7290bd2	dhcp47-162.lab.eng.blr.redhat.com	Connected 
d0955c85-94d0-41ba-aea8-1ffde3575ea5	dhcp47-157.lab.eng.blr.redhat.com	Connected 
834d66eb-fb65-4ea3-949a-e7cb4c198f2b	localhost                        	Connected 
[root@dhcp47-165 ~]# 
[root@dhcp47-165 ~]# cat /etc/redhat-*
cat: /etc/redhat-access-insights: Is a directory
Red Hat Enterprise Linux Server release 7.4 Beta (Maipo)
Red Hat Gluster Storage Server 3.3.0
[root@dhcp47-165 ~]# 
[root@dhcp47-165 ~]# gluster v list testvol
dist_slave
distrep
ozone
testvol
[root@dhcp47-165 ~]# 
[root@dhcp47-165 ~]# 
[root@dhcp47-165 ~]# gluster v info testvol
 
Volume Name: testvol
Type: Replicate
Volume ID: 615ec7dc-4a78-4674-ba86-3dcdbb2d2d01
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.47.165:/bricks/brick0/testvol0
Brick2: 10.70.47.164:/bricks/brick0/testvol1
Brick3: 10.70.47.162:/bricks/brick0/testvol2
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
[root@dhcp47-165 ~]#
Comment 2 Prasanna Kumar Kalever 2017-06-20 12:18:44 UTC 
Digging through the logs, this seems to be more like a python-configshell problem.

Can you please provide "python-configshell" version ?
Comment 3 surabhi 2017-06-20 12:28:07 UTC 
This is the version:
python-configshell-1.1.fb23-1.el7.noarch
Comment 5 Prasanna Kumar Kalever 2017-06-20 12:47:24 UTC 
OKAY - there seems to be bug in python-configshell-1.1.fb23-1.el7, i.e. targetcli ls command will fail with 'TERM' not defined in os.enviorn

can you try upgrading python-configshell to  1.1.fb23-3.el7, rpm source [1] and give a quick try ?

I hope that should solve this for you. more context at bug #1432303

[1] http://download.eng.bos.redhat.com/brewroot/repos/rhel-7.4-build/latest/x86_64/toplink/packages/python-configshell/1.1.fb23/3.el7/noarch/python-configshell-1.1.fb23-3.el7.noarch.rpm
Comment 7 surabhi 2017-06-20 14:03:38 UTC 
I updated the package and tried on my setup, the create works fine and there is no error seen. I have asked Sweta(reporter of the bz)to try on her setup and update the same.
Comment 8 Prasanna Kumar Kalever 2017-06-20 14:30:03 UTC 
Thanks for the quick turn around surabhi.
Comment 13 Sweta Anandpara 2017-06-22 06:55:06 UTC 
Issue was seen in python-configshell >= 1.1.fb23-1, and after upgrading to python-configshell >= 1.1.fb23-3 - all new creates for block worked. The said package is there in the latest rhel7.4 compose. 

Moving this to verified in rhgs-3.3.
Comment 15 errata-xmlrpc 2017-09-21 04:19:33 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:2773