1186903 – 'Internal error: Performing unsafe table load' when splitting images from thinpool stacked on raid1

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1186903 - 'Internal error: Performing unsafe table load' when splitting images from thinpool stacked on raid1

Summary: 'Internal error: Performing unsafe table load' when splitting images from thi...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	lvm2
Sub Component:
Version:	7.1
Hardware:	x86_64
OS:	Linux
Priority:	unspecified
Severity:	low
Target Milestone:	rc
Target Release:	---
Assignee:	Zdenek Kabelac
QA Contact:	cluster-qe@redhat.com
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2015-01-28 20:27 UTC by Corey Marthaler
Modified:	2021-09-03 12:51 UTC (History)
CC List:	7 users (show)
Fixed In Version:	lvm2-2.02.125-1.el7
Doc Type:	Bug Fix
Doc Text:	Code for splitting raid and mirror legs tried to also rename thin pool volume and failed. Fixed code takes handles pool properties and allows splitting operation to pass correctly.
Clone Of:
Environment:
Last Closed:	2015-11-19 12:46:02 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)
-vvvv of the lvconvert --splitmirrors (153.57 KB, text/plain) 2015-01-28 20:29 UTC, Corey Marthaler	no flags	Details
View All

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2015:2147	0	normal	SHIPPED_LIVE	lvm2 bug fix and enhancement update	2015-11-19 11:11:07 UTC

Description Corey Marthaler 2015-01-28 20:27:34 UTC

Description of problem:
SCENARIO - [sequentially_split_off_all_stacked_thinpool_raid1_pvs]
Create raid1 volume with many legs, convert to thin pool, and then sequentially split off each one of the PVs
host-116.virt.lab.msp.redhat.com: lvcreate  --type raid1 -m 4 -n split_thin -L 500M split_image
Waiting until all mirror|raid volumes become fully syncd...
   0/1 mirror(s) are fully synced: ( 93.53% )
   1/1 mirror(s) are fully synced: ( 100.00% )

lvcreate  -n meta -L 200M split_image
lvconvert --yes --thinpool split_image/split_thin --poolmetadata meta
  WARNING: Converting logical volume split_image/split_thin and split_image/meta to pool's data and metadata volumes.
  THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.)
lvcreate  --virtualsize 100M --thinpool split_image/split_thin -n virt


splitting off legs:

/dev/sdf1:  lvconvert --splitmirrors 1 --name new0 split_image/split_thin_tdata /dev/sdf1
/dev/sdc1:  lvconvert --splitmirrors 1 --name new1 split_image/split_thin_tdata /dev/sdc1
/dev/sda1:  lvconvert --splitmirrors 1 --name new2 split_image/split_thin_tdata /dev/sda1
/dev/sdd1:  lvconvert --splitmirrors 1 --name new3 split_image/split_thin_tdata /dev/sdd1

Although the raid split passed, errors were found in it's output
  Internal error: Performing unsafe table load while 5 device(s) are known to be suspended:  (253:12) 


#activate/dev_manager.c:2590         Checking kernel supports error segment type for split_thin_tdata_rimage_0
#activate/activate.c:530       Getting target version for error
#ioctl/libdm-iface.c:1795         dm versions   OF   [16384] (*1)
#activate/activate.c:567       Found error target v1.2.0.
#metadata/metadata.c:2379         Calculated readahead of LV split_thin_tdata_rimage_0 is 0
#libdm-deptree.c:2646     Loading split_image-split_thin_tdata_rimage_1 table (253:12)
#libdm-deptree.c:2590         Adding target to (253:12): 0 1024000 error
#ioctl/libdm-iface.c:1795         dm table   (253:12) OF   [16384] (*1)
#ioctl/libdm-iface.c:1911   Internal error: Performing unsafe table load while 5 device(s) are known to be suspended:  (253:12)
#ioctl/libdm-iface.c:1795         dm reload   (253:12) NF   [16384] (*1)
#libdm-deptree.c:1287     Renaming split_image-split_thin_tdata_rimage_1 (253:12) to split_image-split_thin_tdata_rimage_0
#libdm-common.c:2196         Udev cookie 0xd4db4ac (semid 25231361) incremented to 8
#libdm-common.c:2446         Udev cookie 0xd4db4ac (semid 25231361) assigned to RENAME task(8) with flags DISABLE_DISK_RULES DISABLE_OTHER_RULES DISABLE_LIBRARY_FALLBACK         (0x2c)
#ioctl/libdm-iface.c:1795         dm rename split_image-split_thin_tdata_rimage_1  split_image-split_thin_tdata_rimage_0 NF   [16384] (*1)
#libdm-common.c:1399         split_image-split_thin_tdata_rimage_1: Stacking NODE_RENAME to split_image-split_thin_tdata_rimage_0 [trust_udev]
#libdm-deptree.c:1325     Resuming split_image-split_thin_tdata_rimage_0 (253:12)
#libdm-common.c:2196         Udev cookie 0xd4db4ac (semid 25231361) incremented to 9
#libdm-common.c:2446         Udev cookie 0xd4db4ac (semid 25231361) assigned to RESUME task(5) with flags DISABLE_DISK_RULES DISABLE_OTHER_RULES DISABLE_LIBRARY_FALLBACK         (0x2c)
#ioctl/libdm-iface.c:1795         dm resume   (253:12) NF   [16384] (*1)
#libdm-common.c:1393         split_image-split_thin_tdata_rimage_0: Stacking NODE_ADD (253,12) 0:6 0660 [trust_udev]
#libdm-common.c:1403         split_image-split_thin_tdata_rimage_0: Stacking NODE_READ_AHEAD 0 (flags=1)
#libdm-common.c:230         Suspended device counter reduced to 4





Version-Release number of selected component (if applicable):
3.10.0-225.el7.x86_64
lvm2-2.02.115-3.el7    BUILT: Wed Jan 28 09:59:01 CST 2015
lvm2-libs-2.02.115-3.el7    BUILT: Wed Jan 28 09:59:01 CST 2015
lvm2-cluster-2.02.115-3.el7    BUILT: Wed Jan 28 09:59:01 CST 2015
device-mapper-1.02.93-3.el7    BUILT: Wed Jan 28 09:59:01 CST 2015
device-mapper-libs-1.02.93-3.el7    BUILT: Wed Jan 28 09:59:01 CST 2015
device-mapper-event-1.02.93-3.el7    BUILT: Wed Jan 28 09:59:01 CST 2015
device-mapper-event-libs-1.02.93-3.el7    BUILT: Wed Jan 28 09:59:01 CST 2015
device-mapper-persistent-data-0.4.1-2.el7    BUILT: Wed Nov 12 12:39:46 CST 2014
cmirror-2.02.115-3.el7    BUILT: Wed Jan 28 09:59:01 CST 2015


How reproducible:
Often

Comment 1 Corey Marthaler 2015-01-28 20:29:42 UTC

Created attachment 985335 [details]
-vvvv of the lvconvert --splitmirrors

Comment 2 Zdenek Kabelac 2015-01-29 12:08:36 UTC

Yep - it's the same problem as with cache pool where 'for_each_sub_lv()' internal lvm2 function needs at least 2 operation levels. However for plain
cache pool there is currently a small workaround.

There are different requirements when doing i.e. 'rename' and 'split' when in some case for thin/cache LV we want to proceed all sub volume while in other we want to stop at pool border.

We need to do a bit of thinking how to do it in the best way so the device stacking operates properly in all cases.

Comment 3 Zdenek Kabelac 2015-01-30 12:13:57 UTC

So should be addressed by todays for_each_sub_lv_except_pools() patchset:

https://www.redhat.com/archives/lvm-devel/2015-January/msg00087.html

Comment 5 Corey Marthaler 2015-08-05 22:21:38 UTC

Marking verified in the latest rpms.


3.10.0-302.el7.x86_64
lvm2-2.02.126-1.el7    BUILT: Tue Jul 28 11:32:33 CDT 2015
lvm2-libs-2.02.126-1.el7    BUILT: Tue Jul 28 11:32:33 CDT 2015
lvm2-cluster-2.02.126-1.el7    BUILT: Tue Jul 28 11:32:33 CDT 2015
device-mapper-1.02.103-1.el7    BUILT: Tue Jul 28 11:32:33 CDT 2015
device-mapper-libs-1.02.103-1.el7    BUILT: Tue Jul 28 11:32:33 CDT 2015
device-mapper-event-1.02.103-1.el7    BUILT: Tue Jul 28 11:32:33 CDT 2015
device-mapper-event-libs-1.02.103-1.el7    BUILT: Tue Jul 28 11:32:33 CDT 2015
device-mapper-persistent-data-0.5.4-1.el7    BUILT: Fri Jul 17 08:56:22 CDT 2015
cmirror-2.02.126-1.el7    BUILT: Tue Jul 28 11:32:33 CDT 2015
sanlock-3.2.4-1.el7    BUILT: Fri Jun 19 12:48:49 CDT 2015
sanlock-lib-3.2.4-1.el7    BUILT: Fri Jun 19 12:48:49 CDT 2015
lvm2-lockd-2.02.126-1.el7    BUILT: Tue Jul 28 11:32:33 CDT 2015



[root@harding-03 ~]# lvconvert --splitmirrors 1 --name new0 split_image/split_thin_tdata /dev/mapper/mpatha1
[root@harding-03 ~]# lvs -a -o +devices
  LV                           Attr       LSize   Pool       Data%  Meta% Cpy%Sync Devices
  [lvol0_pmspare]              ewi------- 200.00m                                  /dev/mapper/mpatha1(176)
  new0                         -wi-a----- 500.00m                                  /dev/mapper/mpatha1(1)
  split_thin                   twi---tz-- 500.00m            0.00   0.03           split_thin_tdata(0)
  [split_thin_tdata]           rwi-aor--- 500.00m                         100.00   split_thin_tdata_rimage_0(0),split_thin_tdata_rimage_1(0),split_thin_tdata_rimage_2(0),split_thin_tdata_rimage_3(0)
  [split_thin_tdata_rimage_0]  iwi-aor--- 500.00m                                  /dev/mapper/mpatha2(1)
  [split_thin_tdata_rimage_1]  iwi-aor--- 500.00m                                  /dev/mapper/mpathb1(1)
  [split_thin_tdata_rimage_2]  iwi-aor--- 500.00m                                  /dev/mapper/mpathb2(1)
  [split_thin_tdata_rimage_3]  iwi-aor--- 500.00m                                  /dev/mapper/mpathc1(1)
  [split_thin_tdata_rmeta_0]   ewi-aor---   4.00m                                  /dev/mapper/mpatha2(0)
  [split_thin_tdata_rmeta_1]   ewi-aor---   4.00m                                  /dev/mapper/mpathb1(0)
  [split_thin_tdata_rmeta_2]   ewi-aor---   4.00m                                  /dev/mapper/mpathb2(0)
  [split_thin_tdata_rmeta_3]   ewi-aor---   4.00m                                  /dev/mapper/mpathc1(0)
  [split_thin_tmeta]           ewi-ao---- 200.00m                                  /dev/mapper/mpatha1(126)
  virt                         Vwi-a-tz-- 100.00m split_thin 0.00

Comment 6 errata-xmlrpc 2015-11-19 12:46:02 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2147.html

Note You need to log in before you can comment on or make changes to this bug.