1278781 – [RHEL-7.2] targetcli crashed with many block backstores

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1278781 - [RHEL-7.2] targetcli crashed with many block backstores

Summary: [RHEL-7.2] targetcli crashed with many block backstores

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	python-rtslib
Sub Component:
Version:	7.3
Hardware:	x86_64
OS:	Linux
Priority:	high
Severity:	high
Target Milestone:	rc
Target Release:	---
Assignee:	Andy Grover
QA Contact:	Martin Hoyer
Docs Contact:
URL:
Whiteboard:
Duplicates (1):	1312270 (view as bug list)
Depends On:
Blocks:	1213541 1295577 1313485 1364088
TreeView+	depends on / blocked

Reported:	2015-11-06 11:54 UTC by Honggang LI
Modified:	2021-09-03 13:43 UTC (History)
CC List:	7 users (show)
Fixed In Version:
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:	2016-11-04 08:33:54 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2016:2561	0	normal	SHIPPED_LIVE	python-rtslib bug fix update	2016-11-03 14:22:24 UTC

Description Honggang LI 2015-11-06 11:54:45 UTC

Description of problem:

[root@rdma-dev-00 ~]$ grep -i distro /etc/motd
                           DISTRO=RHEL-7.2-20151105.n.0
[root@rdma-dev-00 ~]$ cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.2 (Maipo)
[root@rdma-dev-00 ~]$ 
[root@rdma-dev-00 ~]$ 
[root@rdma-dev-00 ~]$ rpm -q kernel targetcli
kernel-3.10.0-327.el7.x86_64
targetcli-2.1.fb41-3.el7.noarch
[root@rdma-dev-00 ~]$ 
[root@rdma-dev-00 ~]$ 
[root@rdma-dev-00 ~]$ uname -r
3.10.0-327.el7.x86_64
[root@rdma-dev-00 ~]$ 
[root@rdma-dev-00 ~]$ 
[root@rdma-dev-00 ~]$ df -h | grep part | sort -k 5
/dev/sda5       976M  2.6M  907M   1% /parts/p00
/dev/sda6       976M  2.6M  907M   1% /parts/p01
/dev/sda7       976M  2.6M  907M   1% /parts/p02
/dev/sda8       976M  2.6M  907M   1% /parts/p03
/dev/sda9       976M  2.6M  907M   1% /parts/p04
/dev/sda10      976M  2.6M  907M   1% /parts/p05
/dev/sda11      976M  2.6M  907M   1% /parts/p06
/dev/sda12      976M  2.6M  907M   1% /parts/p07
/dev/sda13      976M  2.6M  907M   1% /parts/p08
/dev/sda14      976M  2.6M  907M   1% /parts/p09
[root@rdma-dev-00 ~]$ 
[root@rdma-dev-00 ~]$ 
[root@rdma-dev-00 ~]$ df -h | grep part | sort -k 5 | awk '{print $1}' | xargs -n 1 umount
[root@rdma-dev-00 ~]$ 
[root@rdma-dev-00 ~]$ 
[root@rdma-dev-00 ~]$ targetcli 
targetcli shell version 2.1.fb41
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.

/> ls /
o- / .................................................................... [...]
  o- backstores ......................................................... [...]
  | o- block ............................................. [Storage Objects: 0]
  | o- fileio ............................................ [Storage Objects: 0]
  | o- pscsi ............................................. [Storage Objects: 0]
  | o- ramdisk ........................................... [Storage Objects: 0]
  o- iscsi ....................................................... [Targets: 0]
  o- loopback .................................................... [Targets: 0]
  o- srpt ........................................................ [Targets: 0]
/> 
/> /backstores/block create p00 /dev/sda5
Created block storage object p00 using /dev/sda5.
/> ls /
o- / .................................................................... [...]
  o- backstores ......................................................... [...]
  | o- block ............................................. [Storage Objects: 1]
  | | o- p00 ...................... [/dev/sda5 (1.0GiB) write-thru deactivated]
  | o- fileio ............................................ [Storage Objects: 0]
  | o- pscsi ............................................. [Storage Objects: 0]
  | o- ramdisk ........................................... [Storage Objects: 0]
  o- iscsi ....................................................... [Targets: 0]
  o- loopback .................................................... [Targets: 0]
  o- srpt ........................................................ [Targets: 0]
/> /backstores/block create p01 /dev/sda6
Created block storage object p01 using /dev/sda6.
/> ls /
o- / .................................................................... [...]
  o- backstores ......................................................... [...]
  | o- block ............................................. [Storage Objects: 2]
  | | o- p00 ...................... [/dev/sda5 (1.0GiB) write-thru deactivated]
  | | o- p01 ...................... [/dev/sda6 (1.0GiB) write-thru deactivated]
  | o- fileio ............................................ [Storage Objects: 0]
  | o- pscsi ............................................. [Storage Objects: 0]
  | o- ramdisk ........................................... [Storage Objects: 0]
  o- iscsi ....................................................... [Targets: 0]
  o- loopback .................................................... [Targets: 0]
  o- srpt ........................................................ [Targets: 0]
/> /backstores/block create p02 /dev/sda7
Created block storage object p02 using /dev/sda7.
/> 
/> ls /
o- / .................................................................... [...]
  o- backstores ......................................................... [...]
  | o- block ............................................. [Storage Objects: 3]
  | | o- p00 ...................... [/dev/sda5 (1.0GiB) write-thru deactivated]
  | | o- p01 ...................... [/dev/sda6 (1.0GiB) write-thru deactivated]
  | | o- p02 ...................... [/dev/sda7 (1.0GiB) write-thru deactivated]
  | o- fileio ............................................ [Storage Objects: 0]
  | o- pscsi ............................................. [Storage Objects: 0]
  | o- ramdisk ........................................... [Storage Objects: 0]
  o- iscsi ....................................................... [Targets: 0]
  o- loopback .................................................... [Targets: 0]
  o- srpt ........................................................ [Targets: 0]
/> /backstores/block create p03 /dev/sda8
Created block storage object p03 using /dev/sda8.
/> ls /
o- / .................................................................... [...]
  o- backstores ......................................................... [...]
  | o- block ............................................. [Storage Objects: 4]
  | | o- p00 ...................... [/dev/sda5 (1.0GiB) write-thru deactivated]
  | | o- p01 ...................... [/dev/sda6 (1.0GiB) write-thru deactivated]
  | | o- p02 ...................... [/dev/sda7 (1.0GiB) write-thru deactivated]
  | | o- p03 ...................... [/dev/sda8 (1.0GiB) write-thru deactivated]
  | o- fileio ............................................ [Storage Objects: 0]
  | o- pscsi ............................................. [Storage Objects: 0]
  | o- ramdisk ........................................... [Storage Objects: 0]
  o- iscsi ....................................................... [Targets: 0]
  o- loopback .................................................... [Targets: 0]
  o- srpt ........................................................ [Targets: 0]
/> /backstores/block create p04 /dev/sda9
Created block storage object p04 using /dev/sda9.
/> ls /
o- / .................................................................... [...]
  o- backstores ......................................................... [...]
  | o- block ............................................. [Storage Objects: 5]
  | | o- p00 ...................... [/dev/sda5 (1.0GiB) write-thru deactivated]
  | | o- p01 ...................... [/dev/sda6 (1.0GiB) write-thru deactivated]
  | | o- p02 ...................... [/dev/sda7 (1.0GiB) write-thru deactivated]
  | | o- p03 ...................... [/dev/sda8 (1.0GiB) write-thru deactivated]
  | | o- p04 ...................... [/dev/sda9 (1.0GiB) write-thru deactivated]
  | o- fileio ............................................ [Storage Objects: 0]
  | o- pscsi ............................................. [Storage Objects: 0]
  | o- ramdisk ........................................... [Storage Objects: 0]
  o- iscsi ....................................................... [Targets: 0]
  o- loopback .................................................... [Targets: 0]
  o- srpt ........................................................ [Targets: 0]
/> /backstores/block create p05 /dev/sda10
Created block storage object p05 using /dev/sda10.
/> ls /
Traceback (most recent call last):
  File "/usr/bin/targetcli", line 121, in <module>
    main()
  File "/usr/bin/targetcli", line 111, in main
    shell.run_interactive()
  File "/usr/lib/python2.7/site-packages/configshell_fb/shell.py", line 894, in run_interactive
    self._cli_loop()
  File "/usr/lib/python2.7/site-packages/configshell_fb/shell.py", line 723, in _cli_loop
    self.run_cmdline(cmdline)
  File "/usr/lib/python2.7/site-packages/configshell_fb/shell.py", line 837, in run_cmdline
    self._execute_command(path, command, pparams, kparams)
  File "/usr/lib/python2.7/site-packages/configshell_fb/shell.py", line 812, in _execute_command
    result = target.execute_command(command, pparams, kparams)
  File "/usr/lib/python2.7/site-packages/configshell_fb/node.py", line 1411, in execute_command
    return method(*pparams, **kparams)
  File "/usr/lib/python2.7/site-packages/configshell_fb/node.py", line 710, in ui_command_ls
    tree = self._render_tree(target, depth=depth)
  File "/usr/lib/python2.7/site-packages/configshell_fb/node.py", line 861, in _render_tree
    + self._render_tree(children[i], margin, depth)
  File "/usr/lib/python2.7/site-packages/configshell_fb/node.py", line 861, in _render_tree
    + self._render_tree(children[i], margin, depth)
  File "/usr/lib/python2.7/site-packages/configshell_fb/node.py", line 861, in _render_tree
    + self._render_tree(children[i], margin, depth)
  File "/usr/lib/python2.7/site-packages/configshell_fb/node.py", line 765, in _render_tree
    (description, is_healthy) = root.summary()
  File "/usr/lib/python2.7/site-packages/targetcli/ui_backstore.py", line 540, in summary
    return ("%s (%s) %s%s %s" % (so.udev_path, bytes_to_human(so.size),
  File "/usr/lib/python2.7/site-packages/rtslib_fb/tcm.py", line 703, in _get_size
    return get_size_for_disk_name(self._parse_info('device')) * int(self._parse_info('SectorSize'))
  File "/usr/lib/python2.7/site-packages/rtslib_fb/utils.py", line 162, in get_size_for_disk_name
    return get_size("/sys/block/%s/%s" % (disk, m.group()), True)
  File "/usr/lib/python2.7/site-packages/rtslib_fb/utils.py", line 141, in get_size
    sect_size = int(fread("%s/size" % path))
  File "/usr/lib/python2.7/site-packages/rtslib_fb/utils.py", line 90, in fread
    with open(path, 'r') as file_fd:
IOError: [Errno 2] No such file or directory: '/sys/block/sda1/sda10/size'
[root@rdma-dev-00 ~]$ 


Version-Release number of selected component (if applicable):


How reproducible:
always

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:
I need create about 30 block backstores for SRP multipath test. So, I hope targetcli works with as many block as possible.

Additional info:

Comment 1 Andy Grover 2015-11-06 17:10:50 UTC

Root-caused to incorrect regex. We should be non-greedy when matching the first part of the name, so we get 'sda' and '10' subgroups instead of 'sda1' and '0'.

If you want to verify the fix, change the regex at line 155 of /usr/lib/python2.7/site-packages/rtslib_fb/utils.py to

m = re.search(r'^([a-z0-9_\-!]+?)(\d+)$', name)

Comment 2 Honggang LI 2015-11-07 01:20:26 UTC

Thanks. The patch works for me.


/> ls /
o- / ............................................................................. [...]
  o- backstores .................................................................. [...]
  | o- block ..................................................... [Storage Objects: 10]
  | | o- p00 ................................. [/dev/sda5 (1.0GiB) write-thru activated]
  | | o- p01 ................................. [/dev/sda6 (1.0GiB) write-thru activated]
  | | o- p02 ................................. [/dev/sda7 (1.0GiB) write-thru activated]
  | | o- p03 ................................. [/dev/sda8 (1.0GiB) write-thru activated]
  | | o- p04 ................................. [/dev/sda9 (1.0GiB) write-thru activated]
  | | o- p05 .............................. [/dev/sda10 (1.0GiB) write-thru deactivated]
  | | o- p06 .............................. [/dev/sda11 (1.0GiB) write-thru deactivated]
  | | o- p07 .............................. [/dev/sda12 (1.0GiB) write-thru deactivated]
  | | o- p08 .............................. [/dev/sda13 (1.0GiB) write-thru deactivated]
  | | o- p09 .............................. [/dev/sda14 (1.0GiB) write-thru deactivated]
  | o- fileio ..................................................... [Storage Objects: 0]
  | o- pscsi ...................................................... [Storage Objects: 0]
  | o- ramdisk .................................................... [Storage Objects: 0]
  o- iscsi ................................................................ [Targets: 0]

Comment 4 Andy Grover 2016-03-30 02:23:41 UTC

*** Bug 1312270 has been marked as a duplicate of this bug. ***

Comment 8 Whitney Chadwick 2016-04-12 14:13:31 UTC

Set PM Ack to +

Comment 13 Martin Hoyer 2016-08-05 09:35:02 UTC

Tested with python-rtslib-2.1.fb57-5.el7.noarch.rpm Works well.

Comment 15 errata-xmlrpc 2016-11-04 08:33:54 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2561.html

Note You need to log in before you can comment on or make changes to this bug.