Bug 1021878

Summary: LVM should display a more educated (or helpful) error when thin LV is in Failed "mode"
Product: Red Hat Enterprise Linux 6 Reporter: Nenad Peric <nperic>
Component: lvm2Assignee: Zdenek Kabelac <zkabelac>
Status: CLOSED ERRATA QA Contact: cluster-qe <cluster-qe>
Severity: low Docs Contact:
Priority: unspecified    
Version: 6.6CC: agk, cmarthal, cvv30693, dwysocha, heinzm, jbrassow, msnitzer, prajnoha, prockai, thornber, zkabelac
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: lvm2-2.02.143-1.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-05-11 01:15:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1112642    
Bug Blocks: 1268411    

Description Nenad Peric 2013-10-22 09:07:55 UTC 
Description of problem:

If for some reason thin pool ends up in a failed state, lvs throws errors which do not help the user very much to understand what is going on. This could probably be improved.

Version-Release number of selected component (if applicable):


lvm2-2.02.100-6.el6.x86_64
device-mapper-1.02.79-6.el6.x86_64


How reproducible:
100% 

Have underlying devices vanish (iscsi time out for example) while VG with thin pool is active. 

Actual results after the devices are back:

[root@virt-008 ~]# lvs -a
  Failed to parse thin pool params: Fail.
  Failed to parse thin pool params: Fail.
  dm_report_object: report function failed for field data_percent
  LV                    VG         Attr       LSize   Pool Origin Data%  Move Log Cpy%Sync Convert
  POOL                  normal     twi-a-tz--   3.00g            
  [POOL_tdata]          normal     rwi-aor---   3.00g                               100.00        
  [POOL_tdata_rimage_0] normal     iwi-aor---   3.00g                                             
  [POOL_tdata_rimage_1] normal     iwi-aor---   3.00g                                             
  [POOL_tdata_rmeta_0]  normal     ewi-aor---   4.00m                                             
  [POOL_tdata_rmeta_1]  normal     ewi-aor---   4.00m                                             
  [POOL_tmeta]          normal     ewi-ao----   4.00m                                             

cut from dmsetup status:

normal-POOL-tpool: 0 6291456 thin-pool Fail


Expected results:

LVM could be aware that thin pool entered (or is) in the failed state and issue a message to the user saying that thin pool is currently in failed mode, thin_check is needed or some other suggested action to help rectify the issue.
Comment 1 Zdenek Kabelac 2013-10-22 09:12:29 UTC 
Yep - thin pool lvm2 support needs to handle  'Failed' state  better here.
It's also partially related to bug #905028.
Comment 7 Zdenek Kabelac 2016-02-19 14:02:56 UTC 
Should be essentially solved through bug #908792.

Lvs  now shows  'F', 'D', 'M' health status attribute for thin-pool.
Comment 8 Zdenek Kabelac 2016-02-19 14:03:45 UTC 
Forget to mention 'F' for failed thin volume.
Comment 10 Corey Marthaler 2016-03-09 17:05:17 UTC 
Marking verified in the latest rpms. The "F" state is now present when the pool device is in a failed state. Errors are also printed to the console. 

That said, anytime the device is in a failed state, one risks hitting kernel panics like mentioned in bug 1305983 and bug 1310661



[root@host-116 ~]# lvs -a -o +devices
  LV              VG            Attr       LSize Pool Origin Data%  Meta% Devices
  POOL            snapper_thinp twi-aot--- 5.00g             7.01   4.79  POOL_tdata(0)
  [POOL_tdata]    snapper_thinp Twi-ao---- 5.00g                          /dev/sdb1(1)
  [POOL_tmeta]    snapper_thinp ewi-ao---- 4.00m                          /dev/sdf1(0)
  [lvol0_pmspare] snapper_thinp ewi------- 4.00m                          /dev/sdb1(0)
  origin          snapper_thinp Vwi-a-t--- 1.00g POOL        33.47
  other1          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01
  other2          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01
  other3          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01
  other4          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01
  other5          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01
  snap1           snapper_thinp Vwi-a-t--- 1.00g POOL origin 16.70

[root@host-116 ~]# dmsetup suspend snapper_thinp-POOL_tmeta
[root@host-116 ~]# dmsetup load snapper_thinp-POOL_tmeta --table " 0 8192 error 8:81 2048"
[root@host-116 ~]# dmsetup resume snapper_thinp-POOL_tmeta

[root@host-116 ~]# lvs -a -o +devices
  Failed to get thin-pool major:minor for thin device 253:6.
  Failed to get thin-pool major:minor for thin device 253:7.
  Failed to get thin-pool major:minor for thin device 253:8.
  Failed to get thin-pool major:minor for thin device 253:9.
  Failed to get thin-pool major:minor for thin device 253:10.
  Failed to get thin-pool major:minor for thin device 253:11.
  Failed to get thin-pool major:minor for thin device 253:12.
  LV              VG            Attr       LSize Pool Origin Data%  Meta% Devices
  POOL            snapper_thinp twi-aot-F- 5.00g                          POOL_tdata(0)
  [POOL_tdata]    snapper_thinp Twi-ao---- 5.00g                          /dev/sdb1(1)
  [POOL_tmeta]    snapper_thinp ewi-ao---- 4.00m                          /dev/sdf1(0)
  [lvol0_pmspare] snapper_thinp ewi------- 4.00m                          /dev/sdb1(0)
  origin          snapper_thinp Vwi-a-t-F- 1.00g POOL
  other1          snapper_thinp Vwi-a-t-F- 1.00g POOL
  other2          snapper_thinp Vwi-a-t-F- 1.00g POOL
  other3          snapper_thinp Vwi-a-t-F- 1.00g POOL
  other4          snapper_thinp Vwi-a-t-F- 1.00g POOL
  other5          snapper_thinp Vwi-a-t-F- 1.00g POOL
  snap1           snapper_thinp Vwi-a-t-F- 1.00g POOL origin


Mar  9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1008
Mar  9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1008
Mar  9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1022
Mar  9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1022
Mar  9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 0
Mar  9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 0
Mar  9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1
Mar  9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1023
Mar  9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1023
Mar  9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1023
Mar  9 10:51:16 host-116 xinetd[1945]: EXIT: qarsh status=0 pid=11830 duration=0(sec)
Mar  9 10:51:17 host-116 xinetd[1945]: START: qarsh pid=11836 from=::ffff:10.15.80.224
Mar  9 10:51:17 host-116 qarshd[11836]: Talking to peer 10.15.80.224:34644 (IPv4)
Mar  9 10:51:17 host-116 qarshd[11836]: Running cmdline: vgchange -an snapper_thinp
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: metadata operation 'dm_pool_commit_metadata' failed: error = -5
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: aborting current metadata transaction
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: failed to abort metadata transaction
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: switching pool to failure mode
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: metadata operation 'dm_pool_commit_metadata' failed: error = -22
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: aborting current metadata transaction
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: failed to abort metadata transaction
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: switching pool to failure mode
Mar  9 10:51:18 host-116 kernel: device-mapper: thin metadata: couldn't read superblock
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: failed to set 'needs_check' flag in metadata
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: dm_pool_get_metadata_transaction_id returned -22
Mar  9 10:51:18 host-116 kernel: device-mapper: thin metadata: couldn't read superblock
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: failed to set 'needs_check' flag in metadata
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: dm_pool_get_metadata_transaction_id returned -22
Mar  9 10:51:18 host-116 lvm[7440]: Failed to get thin-pool major:minor for thin device 253:6.
Mar  9 10:51:18 host-116 lvm[7440]: Failed to get thin-pool major:minor for thin device 253:7.
Mar  9 10:51:18 host-116 lvm[7440]: Failed to get thin-pool major:minor for thin device 253:8.
Mar  9 10:51:18 host-116 lvm[7440]: Failed to get thin-pool major:minor for thin device 253:9.
Mar  9 10:51:18 host-116 lvm[7440]: Failed to get thin-pool major:minor for thin device 253:10.
Mar  9 10:51:18 host-116 lvm[7440]: Failed to get thin-pool major:minor for thin device 253:11.
Comment 12 errata-xmlrpc 2016-05-11 01:15:01 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0964.html
Comment 13 Armstrong 2022-11-25 13:29:36 UTC Comment hidden (spam)