Bug 1021878 - LVM should display a more educated (or helpful) error when thin LV is in Failed "mode"
LVM should display a more educated (or helpful) error when thin LV is in Fail...
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: lvm2 (Show other bugs)
6.6
Unspecified Unspecified
unspecified Severity low
: rc
: ---
Assigned To: Zdenek Kabelac
cluster-qe@redhat.com
:
Depends On: 1112642
Blocks: 1268411
  Show dependency treegraph
 
Reported: 2013-10-22 05:07 EDT by Nenad Peric
Modified: 2016-05-10 21:15 EDT (History)
10 users (show)

See Also:
Fixed In Version: lvm2-2.02.143-1.el6
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-05-10 21:15:01 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Nenad Peric 2013-10-22 05:07:55 EDT
Description of problem:

If for some reason thin pool ends up in a failed state, lvs throws errors which do not help the user very much to understand what is going on. This could probably be improved.

Version-Release number of selected component (if applicable):


lvm2-2.02.100-6.el6.x86_64
device-mapper-1.02.79-6.el6.x86_64


How reproducible:
100% 

Have underlying devices vanish (iscsi time out for example) while VG with thin pool is active. 

Actual results after the devices are back:

[root@virt-008 ~]# lvs -a
  Failed to parse thin pool params: Fail.
  Failed to parse thin pool params: Fail.
  dm_report_object: report function failed for field data_percent
  LV                    VG         Attr       LSize   Pool Origin Data%  Move Log Cpy%Sync Convert
  POOL                  normal     twi-a-tz--   3.00g            
  [POOL_tdata]          normal     rwi-aor---   3.00g                               100.00        
  [POOL_tdata_rimage_0] normal     iwi-aor---   3.00g                                             
  [POOL_tdata_rimage_1] normal     iwi-aor---   3.00g                                             
  [POOL_tdata_rmeta_0]  normal     ewi-aor---   4.00m                                             
  [POOL_tdata_rmeta_1]  normal     ewi-aor---   4.00m                                             
  [POOL_tmeta]          normal     ewi-ao----   4.00m                                             

cut from dmsetup status:

normal-POOL-tpool: 0 6291456 thin-pool Fail


Expected results:

LVM could be aware that thin pool entered (or is) in the failed state and issue a message to the user saying that thin pool is currently in failed mode, thin_check is needed or some other suggested action to help rectify the issue.
Comment 1 Zdenek Kabelac 2013-10-22 05:12:29 EDT
Yep - thin pool lvm2 support needs to handle  'Failed' state  better here.
It's also partially related to bug #905028.
Comment 7 Zdenek Kabelac 2016-02-19 09:02:56 EST
Should be essentially solved through bug #908792.

Lvs  now shows  'F', 'D', 'M' health status attribute for thin-pool.
Comment 8 Zdenek Kabelac 2016-02-19 09:03:45 EST
Forget to mention 'F' for failed thin volume.
Comment 10 Corey Marthaler 2016-03-09 12:05:17 EST
Marking verified in the latest rpms. The "F" state is now present when the pool device is in a failed state. Errors are also printed to the console. 

That said, anytime the device is in a failed state, one risks hitting kernel panics like mentioned in bug 1305983 and bug 1310661



[root@host-116 ~]# lvs -a -o +devices
  LV              VG            Attr       LSize Pool Origin Data%  Meta% Devices
  POOL            snapper_thinp twi-aot--- 5.00g             7.01   4.79  POOL_tdata(0)
  [POOL_tdata]    snapper_thinp Twi-ao---- 5.00g                          /dev/sdb1(1)
  [POOL_tmeta]    snapper_thinp ewi-ao---- 4.00m                          /dev/sdf1(0)
  [lvol0_pmspare] snapper_thinp ewi------- 4.00m                          /dev/sdb1(0)
  origin          snapper_thinp Vwi-a-t--- 1.00g POOL        33.47
  other1          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01
  other2          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01
  other3          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01
  other4          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01
  other5          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01
  snap1           snapper_thinp Vwi-a-t--- 1.00g POOL origin 16.70

[root@host-116 ~]# dmsetup suspend snapper_thinp-POOL_tmeta
[root@host-116 ~]# dmsetup load snapper_thinp-POOL_tmeta --table " 0 8192 error 8:81 2048"
[root@host-116 ~]# dmsetup resume snapper_thinp-POOL_tmeta

[root@host-116 ~]# lvs -a -o +devices
  Failed to get thin-pool major:minor for thin device 253:6.
  Failed to get thin-pool major:minor for thin device 253:7.
  Failed to get thin-pool major:minor for thin device 253:8.
  Failed to get thin-pool major:minor for thin device 253:9.
  Failed to get thin-pool major:minor for thin device 253:10.
  Failed to get thin-pool major:minor for thin device 253:11.
  Failed to get thin-pool major:minor for thin device 253:12.
  LV              VG            Attr       LSize Pool Origin Data%  Meta% Devices
  POOL            snapper_thinp twi-aot-F- 5.00g                          POOL_tdata(0)
  [POOL_tdata]    snapper_thinp Twi-ao---- 5.00g                          /dev/sdb1(1)
  [POOL_tmeta]    snapper_thinp ewi-ao---- 4.00m                          /dev/sdf1(0)
  [lvol0_pmspare] snapper_thinp ewi------- 4.00m                          /dev/sdb1(0)
  origin          snapper_thinp Vwi-a-t-F- 1.00g POOL
  other1          snapper_thinp Vwi-a-t-F- 1.00g POOL
  other2          snapper_thinp Vwi-a-t-F- 1.00g POOL
  other3          snapper_thinp Vwi-a-t-F- 1.00g POOL
  other4          snapper_thinp Vwi-a-t-F- 1.00g POOL
  other5          snapper_thinp Vwi-a-t-F- 1.00g POOL
  snap1           snapper_thinp Vwi-a-t-F- 1.00g POOL origin


Mar  9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1008
Mar  9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1008
Mar  9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1022
Mar  9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1022
Mar  9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 0
Mar  9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 0
Mar  9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1
Mar  9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1023
Mar  9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1023
Mar  9 10:51:16 host-116 kernel: Buffer I/O error on device dm-2, logical block 1023
Mar  9 10:51:16 host-116 xinetd[1945]: EXIT: qarsh status=0 pid=11830 duration=0(sec)
Mar  9 10:51:17 host-116 xinetd[1945]: START: qarsh pid=11836 from=::ffff:10.15.80.224
Mar  9 10:51:17 host-116 qarshd[11836]: Talking to peer 10.15.80.224:34644 (IPv4)
Mar  9 10:51:17 host-116 qarshd[11836]: Running cmdline: vgchange -an snapper_thinp
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: metadata operation 'dm_pool_commit_metadata' failed: error = -5
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: aborting current metadata transaction
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: failed to abort metadata transaction
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: switching pool to failure mode
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: metadata operation 'dm_pool_commit_metadata' failed: error = -22
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: aborting current metadata transaction
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: failed to abort metadata transaction
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: switching pool to failure mode
Mar  9 10:51:18 host-116 kernel: device-mapper: thin metadata: couldn't read superblock
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: failed to set 'needs_check' flag in metadata
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: dm_pool_get_metadata_transaction_id returned -22
Mar  9 10:51:18 host-116 kernel: device-mapper: thin metadata: couldn't read superblock
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: failed to set 'needs_check' flag in metadata
Mar  9 10:51:18 host-116 kernel: device-mapper: thin: 253:4: dm_pool_get_metadata_transaction_id returned -22
Mar  9 10:51:18 host-116 lvm[7440]: Failed to get thin-pool major:minor for thin device 253:6.
Mar  9 10:51:18 host-116 lvm[7440]: Failed to get thin-pool major:minor for thin device 253:7.
Mar  9 10:51:18 host-116 lvm[7440]: Failed to get thin-pool major:minor for thin device 253:8.
Mar  9 10:51:18 host-116 lvm[7440]: Failed to get thin-pool major:minor for thin device 253:9.
Mar  9 10:51:18 host-116 lvm[7440]: Failed to get thin-pool major:minor for thin device 253:10.
Mar  9 10:51:18 host-116 lvm[7440]: Failed to get thin-pool major:minor for thin device 253:11.
Comment 12 errata-xmlrpc 2016-05-10 21:15:01 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0964.html

Note You need to log in before you can comment on or make changes to this bug.