Bug 1278781

Summary: [RHEL-7.2] targetcli crashed with many block backstores
Product: Red Hat Enterprise Linux 7 Reporter: Honggang LI <honli>
Component: python-rtslibAssignee: Andy Grover <agrover>
Status: CLOSED ERRATA QA Contact: Martin Hoyer <mhoyer>
Severity: high Docs Contact:
Priority: high    
Version: 7.3CC: agrover, bgoncalv, jkachuck, knoha, lmiksik, snagar, wchadwic
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-04 08:33:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1213541, 1295577, 1313485, 1364088    

Description Honggang LI 2015-11-06 11:54:45 UTC
Description of problem:

[root@rdma-dev-00 ~]$ grep -i distro /etc/motd
                           DISTRO=RHEL-7.2-20151105.n.0
[root@rdma-dev-00 ~]$ cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.2 (Maipo)
[root@rdma-dev-00 ~]$ 
[root@rdma-dev-00 ~]$ 
[root@rdma-dev-00 ~]$ rpm -q kernel targetcli
kernel-3.10.0-327.el7.x86_64
targetcli-2.1.fb41-3.el7.noarch
[root@rdma-dev-00 ~]$ 
[root@rdma-dev-00 ~]$ 
[root@rdma-dev-00 ~]$ uname -r
3.10.0-327.el7.x86_64
[root@rdma-dev-00 ~]$ 
[root@rdma-dev-00 ~]$ 
[root@rdma-dev-00 ~]$ df -h | grep part | sort -k 5
/dev/sda5       976M  2.6M  907M   1% /parts/p00
/dev/sda6       976M  2.6M  907M   1% /parts/p01
/dev/sda7       976M  2.6M  907M   1% /parts/p02
/dev/sda8       976M  2.6M  907M   1% /parts/p03
/dev/sda9       976M  2.6M  907M   1% /parts/p04
/dev/sda10      976M  2.6M  907M   1% /parts/p05
/dev/sda11      976M  2.6M  907M   1% /parts/p06
/dev/sda12      976M  2.6M  907M   1% /parts/p07
/dev/sda13      976M  2.6M  907M   1% /parts/p08
/dev/sda14      976M  2.6M  907M   1% /parts/p09
[root@rdma-dev-00 ~]$ 
[root@rdma-dev-00 ~]$ 
[root@rdma-dev-00 ~]$ df -h | grep part | sort -k 5 | awk '{print $1}' | xargs -n 1 umount
[root@rdma-dev-00 ~]$ 
[root@rdma-dev-00 ~]$ 
[root@rdma-dev-00 ~]$ targetcli 
targetcli shell version 2.1.fb41
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.

/> ls /
o- / .................................................................... [...]
  o- backstores ......................................................... [...]
  | o- block ............................................. [Storage Objects: 0]
  | o- fileio ............................................ [Storage Objects: 0]
  | o- pscsi ............................................. [Storage Objects: 0]
  | o- ramdisk ........................................... [Storage Objects: 0]
  o- iscsi ....................................................... [Targets: 0]
  o- loopback .................................................... [Targets: 0]
  o- srpt ........................................................ [Targets: 0]
/> 
/> /backstores/block create p00 /dev/sda5
Created block storage object p00 using /dev/sda5.
/> ls /
o- / .................................................................... [...]
  o- backstores ......................................................... [...]
  | o- block ............................................. [Storage Objects: 1]
  | | o- p00 ...................... [/dev/sda5 (1.0GiB) write-thru deactivated]
  | o- fileio ............................................ [Storage Objects: 0]
  | o- pscsi ............................................. [Storage Objects: 0]
  | o- ramdisk ........................................... [Storage Objects: 0]
  o- iscsi ....................................................... [Targets: 0]
  o- loopback .................................................... [Targets: 0]
  o- srpt ........................................................ [Targets: 0]
/> /backstores/block create p01 /dev/sda6
Created block storage object p01 using /dev/sda6.
/> ls /
o- / .................................................................... [...]
  o- backstores ......................................................... [...]
  | o- block ............................................. [Storage Objects: 2]
  | | o- p00 ...................... [/dev/sda5 (1.0GiB) write-thru deactivated]
  | | o- p01 ...................... [/dev/sda6 (1.0GiB) write-thru deactivated]
  | o- fileio ............................................ [Storage Objects: 0]
  | o- pscsi ............................................. [Storage Objects: 0]
  | o- ramdisk ........................................... [Storage Objects: 0]
  o- iscsi ....................................................... [Targets: 0]
  o- loopback .................................................... [Targets: 0]
  o- srpt ........................................................ [Targets: 0]
/> /backstores/block create p02 /dev/sda7
Created block storage object p02 using /dev/sda7.
/> 
/> ls /
o- / .................................................................... [...]
  o- backstores ......................................................... [...]
  | o- block ............................................. [Storage Objects: 3]
  | | o- p00 ...................... [/dev/sda5 (1.0GiB) write-thru deactivated]
  | | o- p01 ...................... [/dev/sda6 (1.0GiB) write-thru deactivated]
  | | o- p02 ...................... [/dev/sda7 (1.0GiB) write-thru deactivated]
  | o- fileio ............................................ [Storage Objects: 0]
  | o- pscsi ............................................. [Storage Objects: 0]
  | o- ramdisk ........................................... [Storage Objects: 0]
  o- iscsi ....................................................... [Targets: 0]
  o- loopback .................................................... [Targets: 0]
  o- srpt ........................................................ [Targets: 0]
/> /backstores/block create p03 /dev/sda8
Created block storage object p03 using /dev/sda8.
/> ls /
o- / .................................................................... [...]
  o- backstores ......................................................... [...]
  | o- block ............................................. [Storage Objects: 4]
  | | o- p00 ...................... [/dev/sda5 (1.0GiB) write-thru deactivated]
  | | o- p01 ...................... [/dev/sda6 (1.0GiB) write-thru deactivated]
  | | o- p02 ...................... [/dev/sda7 (1.0GiB) write-thru deactivated]
  | | o- p03 ...................... [/dev/sda8 (1.0GiB) write-thru deactivated]
  | o- fileio ............................................ [Storage Objects: 0]
  | o- pscsi ............................................. [Storage Objects: 0]
  | o- ramdisk ........................................... [Storage Objects: 0]
  o- iscsi ....................................................... [Targets: 0]
  o- loopback .................................................... [Targets: 0]
  o- srpt ........................................................ [Targets: 0]
/> /backstores/block create p04 /dev/sda9
Created block storage object p04 using /dev/sda9.
/> ls /
o- / .................................................................... [...]
  o- backstores ......................................................... [...]
  | o- block ............................................. [Storage Objects: 5]
  | | o- p00 ...................... [/dev/sda5 (1.0GiB) write-thru deactivated]
  | | o- p01 ...................... [/dev/sda6 (1.0GiB) write-thru deactivated]
  | | o- p02 ...................... [/dev/sda7 (1.0GiB) write-thru deactivated]
  | | o- p03 ...................... [/dev/sda8 (1.0GiB) write-thru deactivated]
  | | o- p04 ...................... [/dev/sda9 (1.0GiB) write-thru deactivated]
  | o- fileio ............................................ [Storage Objects: 0]
  | o- pscsi ............................................. [Storage Objects: 0]
  | o- ramdisk ........................................... [Storage Objects: 0]
  o- iscsi ....................................................... [Targets: 0]
  o- loopback .................................................... [Targets: 0]
  o- srpt ........................................................ [Targets: 0]
/> /backstores/block create p05 /dev/sda10
Created block storage object p05 using /dev/sda10.
/> ls /
Traceback (most recent call last):
  File "/usr/bin/targetcli", line 121, in <module>
    main()
  File "/usr/bin/targetcli", line 111, in main
    shell.run_interactive()
  File "/usr/lib/python2.7/site-packages/configshell_fb/shell.py", line 894, in run_interactive
    self._cli_loop()
  File "/usr/lib/python2.7/site-packages/configshell_fb/shell.py", line 723, in _cli_loop
    self.run_cmdline(cmdline)
  File "/usr/lib/python2.7/site-packages/configshell_fb/shell.py", line 837, in run_cmdline
    self._execute_command(path, command, pparams, kparams)
  File "/usr/lib/python2.7/site-packages/configshell_fb/shell.py", line 812, in _execute_command
    result = target.execute_command(command, pparams, kparams)
  File "/usr/lib/python2.7/site-packages/configshell_fb/node.py", line 1411, in execute_command
    return method(*pparams, **kparams)
  File "/usr/lib/python2.7/site-packages/configshell_fb/node.py", line 710, in ui_command_ls
    tree = self._render_tree(target, depth=depth)
  File "/usr/lib/python2.7/site-packages/configshell_fb/node.py", line 861, in _render_tree
    + self._render_tree(children[i], margin, depth)
  File "/usr/lib/python2.7/site-packages/configshell_fb/node.py", line 861, in _render_tree
    + self._render_tree(children[i], margin, depth)
  File "/usr/lib/python2.7/site-packages/configshell_fb/node.py", line 861, in _render_tree
    + self._render_tree(children[i], margin, depth)
  File "/usr/lib/python2.7/site-packages/configshell_fb/node.py", line 765, in _render_tree
    (description, is_healthy) = root.summary()
  File "/usr/lib/python2.7/site-packages/targetcli/ui_backstore.py", line 540, in summary
    return ("%s (%s) %s%s %s" % (so.udev_path, bytes_to_human(so.size),
  File "/usr/lib/python2.7/site-packages/rtslib_fb/tcm.py", line 703, in _get_size
    return get_size_for_disk_name(self._parse_info('device')) * int(self._parse_info('SectorSize'))
  File "/usr/lib/python2.7/site-packages/rtslib_fb/utils.py", line 162, in get_size_for_disk_name
    return get_size("/sys/block/%s/%s" % (disk, m.group()), True)
  File "/usr/lib/python2.7/site-packages/rtslib_fb/utils.py", line 141, in get_size
    sect_size = int(fread("%s/size" % path))
  File "/usr/lib/python2.7/site-packages/rtslib_fb/utils.py", line 90, in fread
    with open(path, 'r') as file_fd:
IOError: [Errno 2] No such file or directory: '/sys/block/sda1/sda10/size'
[root@rdma-dev-00 ~]$ 


Version-Release number of selected component (if applicable):


How reproducible:
always

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:
I need create about 30 block backstores for SRP multipath test. So, I hope targetcli works with as many block as possible.

Additional info:

Comment 1 Andy Grover 2015-11-06 17:10:50 UTC
Root-caused to incorrect regex. We should be non-greedy when matching the first part of the name, so we get 'sda' and '10' subgroups instead of 'sda1' and '0'.

If you want to verify the fix, change the regex at line 155 of /usr/lib/python2.7/site-packages/rtslib_fb/utils.py to

m = re.search(r'^([a-z0-9_\-!]+?)(\d+)$', name)

Comment 2 Honggang LI 2015-11-07 01:20:26 UTC
Thanks. The patch works for me.


/> ls /
o- / ............................................................................. [...]
  o- backstores .................................................................. [...]
  | o- block ..................................................... [Storage Objects: 10]
  | | o- p00 ................................. [/dev/sda5 (1.0GiB) write-thru activated]
  | | o- p01 ................................. [/dev/sda6 (1.0GiB) write-thru activated]
  | | o- p02 ................................. [/dev/sda7 (1.0GiB) write-thru activated]
  | | o- p03 ................................. [/dev/sda8 (1.0GiB) write-thru activated]
  | | o- p04 ................................. [/dev/sda9 (1.0GiB) write-thru activated]
  | | o- p05 .............................. [/dev/sda10 (1.0GiB) write-thru deactivated]
  | | o- p06 .............................. [/dev/sda11 (1.0GiB) write-thru deactivated]
  | | o- p07 .............................. [/dev/sda12 (1.0GiB) write-thru deactivated]
  | | o- p08 .............................. [/dev/sda13 (1.0GiB) write-thru deactivated]
  | | o- p09 .............................. [/dev/sda14 (1.0GiB) write-thru deactivated]
  | o- fileio ..................................................... [Storage Objects: 0]
  | o- pscsi ...................................................... [Storage Objects: 0]
  | o- ramdisk .................................................... [Storage Objects: 0]
  o- iscsi ................................................................ [Targets: 0]

Comment 4 Andy Grover 2016-03-30 02:23:41 UTC
*** Bug 1312270 has been marked as a duplicate of this bug. ***

Comment 8 Whitney Chadwick 2016-04-12 14:13:31 UTC
Set PM Ack to +

Comment 13 Martin Hoyer 2016-08-05 09:35:02 UTC
Tested with python-rtslib-2.1.fb57-5.el7.noarch.rpm Works well.

Comment 15 errata-xmlrpc 2016-11-04 08:33:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2561.html