Red Hat Bugzilla – Bug 1021878
LVM should display a more educated (or helpful) error when thin LV is in Failed "mode"
Last modified: 2016-05-10 21:15:01 EDT
Description of problem: If for some reason thin pool ends up in a failed state, lvs throws errors which do not help the user very much to understand what is going on. This could probably be improved. Version-Release number of selected component (if applicable): lvm2-2.02.100-6.el6.x86_64 device-mapper-1.02.79-6.el6.x86_64 How reproducible: 100% Have underlying devices vanish (iscsi time out for example) while VG with thin pool is active. Actual results after the devices are back: [root@virt-008 ~]# lvs -a Failed to parse thin pool params: Fail. Failed to parse thin pool params: Fail. dm_report_object: report function failed for field data_percent LV VG Attr LSize Pool Origin Data% Move Log Cpy%Sync Convert POOL normal twi-a-tz-- 3.00g [POOL_tdata] normal rwi-aor--- 3.00g 100.00 [POOL_tdata_rimage_0] normal iwi-aor--- 3.00g [POOL_tdata_rimage_1] normal iwi-aor--- 3.00g [POOL_tdata_rmeta_0] normal ewi-aor--- 4.00m [POOL_tdata_rmeta_1] normal ewi-aor--- 4.00m [POOL_tmeta] normal ewi-ao---- 4.00m cut from dmsetup status: normal-POOL-tpool: 0 6291456 thin-pool Fail Expected results: LVM could be aware that thin pool entered (or is) in the failed state and issue a message to the user saying that thin pool is currently in failed mode, thin_check is needed or some other suggested action to help rectify the issue.
Yep - thin pool lvm2 support needs to handle 'Failed' state better here. It's also partially related to bug #905028.
Should be essentially solved through bug #908792. Lvs now shows 'F', 'D', 'M' health status attribute for thin-pool.
Forget to mention 'F' for failed thin volume.
Marking verified in the latest rpms. The "F" state is now present when the pool device is in a failed state. Errors are also printed to the console. That said, anytime the device is in a failed state, one risks hitting kernel panics like mentioned in bug 1305983 and bug 1310661 [root@host-116 ~]# lvs -a -o +devices LV VG Attr LSize Pool Origin Data% Meta% Devices POOL snapper_thinp twi-aot--- 5.00g 7.01 4.79 POOL_tdata(0) [POOL_tdata] snapper_thinp Twi-ao---- 5.00g /dev/sdb1(1) [POOL_tmeta] snapper_thinp ewi-ao---- 4.00m /dev/sdf1(0) [lvol0_pmspare] snapper_thinp ewi------- 4.00m /dev/sdb1(0) origin snapper_thinp Vwi-a-t--- 1.00g POOL 33.47 other1 snapper_thinp Vwi-a-t--- 1.00g POOL 0.01 other2 snapper_thinp Vwi-a-t--- 1.00g POOL 0.01 other3 snapper_thinp Vwi-a-t--- 1.00g POOL 0.01 other4 snapper_thinp Vwi-a-t--- 1.00g POOL 0.01 other5 snapper_thinp Vwi-a-t--- 1.00g POOL 0.01 snap1 snapper_thinp Vwi-a-t--- 1.00g POOL origin 16.70 [root@host-116 ~]# dmsetup suspend snapper_thinp-POOL_tmeta [root@host-116 ~]# dmsetup load snapper_thinp-POOL_tmeta --table " 0 8192 error 8:81 2048" [root@host-116 ~]# dmsetup resume snapper_thinp-POOL_tmeta [root@host-116 ~]# lvs -a -o +devices Failed to get thin-pool major:minor for thin device 253:6. Failed to get thin-pool major:minor for thin device 253:7. Failed to get thin-pool major:minor for thin device 253:8. Failed to get thin-pool major:minor for thin device 253:9. Failed to get thin-pool major:minor for thin device 253:10. Failed to get thin-pool major:minor for thin device 253:11. Failed to get thin-pool major:minor for thin device 253:12. LV VG Attr LSize Pool Origin Data% Meta% Devices POOL snapper_thinp twi-aot-F- 5.00g POOL_tdata(0) [POOL_tdata] snapper_thinp Twi-ao---- 5.00g /dev/sdb1(1) [POOL_tmeta] snapper_thinp ewi-ao---- 4.00m /dev/sdf1(0) [lvol0_pmspare] snapper_thinp ewi------- 4.00m /dev/sdb1(0) origin snapper_thinp Vwi-a-t-F- 1.00g POOL other1 snapper_thinp Vwi-a-t-F- 1.00g POOL other2 snapper_thinp Vwi-a-t-F- 1.00g POOL other3 snapper_thinp Vwi-a-t-F- 1.00g POOL other4 snapper_thinp Vwi-a-t-F- 1.00g POOL other5 snapper_thinp Vwi-a-t-F- 1.00g POOL snap1 snapper_thinp Vwi-a-t-F- 1.00g POOL origin Mar 9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1008 Mar 9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1008 Mar 9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1022 Mar 9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1022 Mar 9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 0 Mar 9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 0 Mar 9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1 Mar 9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1023 Mar 9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1023 Mar 9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1023 Mar 9 10:51:16 host-116 xinetd[1945]: EXIT: qarsh status=0 pid=11830 duration=0(sec) Mar 9 10:51:17 host-116 xinetd[1945]: START: qarsh pid=11836 from=::ffff:10.15.80.224 Mar 9 10:51:17 host-116 qarshd[11836]: Talking to peer 10.15.80.224:34644 (IPv4) Mar 9 10:51:17 host-116 qarshd[11836]: Running cmdline: vgchange -an snapper_thinp Mar 9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: metadata operation 'dm_pool_commit_metadata' failed: error = -5 Mar 9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: aborting current metadata transaction Mar 9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: failed to abort metadata transaction Mar 9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: switching pool to failure mode Mar 9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: metadata operation 'dm_pool_commit_metadata' failed: error = -22 Mar 9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: aborting current metadata transaction Mar 9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: failed to abort metadata transaction Mar 9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: switching pool to failure mode Mar 9 10:51:18 host-116 kernel: device-mapper: thin metadata: couldn't read superblock Mar 9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: failed to set 'needs_check' flag in metadata Mar 9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: dm_pool_get_metadata_transaction_id returned -22 Mar 9 10:51:18 host-116 kernel: device-mapper: thin metadata: couldn't read superblock Mar 9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: failed to set 'needs_check' flag in metadata Mar 9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: dm_pool_get_metadata_transaction_id returned -22 Mar 9 10:51:18 host-116 lvm[7440]: Failed to get thin-pool major:minor for thin device 253:6. Mar 9 10:51:18 host-116 lvm[7440]: Failed to get thin-pool major:minor for thin device 253:7. Mar 9 10:51:18 host-116 lvm[7440]: Failed to get thin-pool major:minor for thin device 253:8. Mar 9 10:51:18 host-116 lvm[7440]: Failed to get thin-pool major:minor for thin device 253:9. Mar 9 10:51:18 host-116 lvm[7440]: Failed to get thin-pool major:minor for thin device 253:10. Mar 9 10:51:18 host-116 lvm[7440]: Failed to get thin-pool major:minor for thin device 253:11.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-0964.html