Bug 1438013

Summary: RAID TAKEOVER: needs to be able to handle insufficient PVs to complete request
Product: Red Hat Enterprise Linux 7 Reporter: Corey Marthaler <cmarthal>
Component: lvm2Assignee: Heinz Mauelshagen <heinzm>
lvm2 sub component: Mirroring and RAID QA Contact: cluster-qe <cluster-qe>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: unspecified CC: agk, heinzm, jbrassow, mcsontos, msnitzer, prajnoha, prockai, zkabelac
Version: 7.4   
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: lvm2-2.02.171-1.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-01 21:52:19 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Corey Marthaler 2017-03-31 15:53:08 UTC
Description of problem:

# Striped -> Raid6 (WITH enough PVs to satisfy takeover)                                                                                                                                                                                                
[root@host-073 ~]# lvcreate  --type striped -i 5 -n takeover -L 500M centipede2                                                                                                                                                                        
  Using default stripesize 64.00 KiB.                                                                                                                                                                                                                  
  Logical volume "takeover" created.                                                                                                                                                                                                                  
[root@host-073 ~]# lvconvert --yes --type raid6 centipede2/takeover                                                                                                                                                                                
  Using default stripesize 64.00 KiB.                                                                                                                                                                                                              
  Replaced LV type raid6 with possible type raid6_n_6.                                                                                                                                                                                             
  Logical volume centipede2/takeover successfully converted.                                                                                                                                                                                    
[root@host-073 ~]# lvs -a -o +devices,segtype                                                                                                                                                                                                   
  LV                  VG          Attr       LSize   Cpy%Sync Devices                                                                                                                                            Type                           
  takeover            centipede2  rwi-a-r--- 500.00m 100.00   takeover_rimage_0(0),takeover_rimage_1(0),takeover_rimage_2(0),takeover_rimage_3(0),takeover_rimage_4(0),takeover_rimage_5(0),takeover_rimage_6(0) raid6_n_6                    
  [takeover_rimage_0] centipede2  iwi-aor--- 100.00m          /dev/sde1(0)                                                                                                                                       linear                       
  [takeover_rimage_1] centipede2  iwi-aor--- 100.00m          /dev/sdd1(0)                                                                                                                                       linear                       
  [takeover_rimage_2] centipede2  iwi-aor--- 100.00m          /dev/sdg1(0)                                                                                                                                       linear                     
  [takeover_rimage_3] centipede2  iwi-aor--- 100.00m          /dev/sda1(0)                                                                                                                                       linear                     
  [takeover_rimage_4] centipede2  iwi-aor--- 100.00m          /dev/sdc1(0)                                                                                                                                       linear                    
  [takeover_rimage_5] centipede2  iwi-aor--- 100.00m          /dev/sdf1(1)                                                                                                                                       linear                   
  [takeover_rimage_6] centipede2  iwi-aor--- 100.00m          /dev/sdh1(1)                                                                                                                                       linear                   
  [takeover_rmeta_0]  centipede2  ewi-aor---   4.00m          /dev/sde1(25)                                                                                                                                      linear                   
  [takeover_rmeta_1]  centipede2  ewi-aor---   4.00m          /dev/sdd1(25)                                                                                                                                      linear                    
  [takeover_rmeta_2]  centipede2  ewi-aor---   4.00m          /dev/sdg1(25)                                                                                                                                      linear                       
  [takeover_rmeta_3]  centipede2  ewi-aor---   4.00m          /dev/sda1(25)                                                                                                                                      linear                             
  [takeover_rmeta_4]  centipede2  ewi-aor---   4.00m          /dev/sdc1(25)                                                                                                                                      linear                                    
  [takeover_rmeta_5]  centipede2  ewi-aor---   4.00m          /dev/sdf1(0)                                                                                                                                       linear                                            
  [takeover_rmeta_6]  centipede2  ewi-aor---   4.00m          /dev/sdh1(0)                                                                                                                                       linear                                                    
                                                                                                                                                                                                                                                                           
                                                                                                                                                                                                                                                                           
                                                                                                                                                                                                                                                                              
# Striped -> Raid6 (WITHOUT enough PVs to satisfy takeover)                                                                                                                                                                                                                   
[root@host-073 ~]# lvcreate  --type striped -i 6 -n takeover -L 500M centipede2
  Using default stripesize 64.00 KiB.
  Rounding size 500.00 MiB (125 extents) up to stripe boundary size 504.00 MiB(126 extents).
  Logical volume "takeover" created.
[root@host-073 ~]# lvconvert --yes --type raid6 centipede2/takeover
  Using default stripesize 64.00 KiB.
  Replaced LV type raid6 with possible type raid6_n_6.
  Insufficient suitable allocatable extents for logical volume : 44 more required
[root@host-073 ~]# echo $?
5
[root@host-073 ~]# lvs -a -o +devices,segtype
  Internal error: WARNING: Segment type striped found does not match expected type raid0 for centipede2/takeover.
  Expected raid segment type but got striped instead.
  LV                  VG          Attr       LSize    Devices                                                                                                                       Type  
  takeover            centipede2  rwi-XXr-r- 504.00m  takeover_rimage_0(0),takeover_rimage_1(0),takeover_rimage_2(0),takeover_rimage_3(0),takeover_rimage_4(0),takeover_rimage_5(0) raid0 
  [takeover_rimage_0] centipede2  Iwi---r---  84.00m  /dev/sde1(0)                                                                                                                  linear
  [takeover_rimage_1] centipede2  Iwi---r---  84.00m  /dev/sdd1(0)                                                                                                                  linear
  [takeover_rimage_2] centipede2  Iwi---r---  84.00m  /dev/sdg1(0)                                                                                                                  linear
  [takeover_rimage_3] centipede2  Iwi---r---  84.00m  /dev/sda1(0)                                                                                                                  linear
  [takeover_rimage_4] centipede2  Iwi---r---  84.00m  /dev/sdc1(0)                                                                                                                  linear
  [takeover_rimage_5] centipede2  Iwi---r---  84.00m  /dev/sdf1(0)                                                                                                                  linear
  takeover_rmeta_0    centipede2  -wi-------   4.00m  /dev/sde1(21)                                                                                                                 linear
  takeover_rmeta_1    centipede2  -wi-------   4.00m  /dev/sdd1(21)                                                                                                                 linear
  takeover_rmeta_2    centipede2  -wi-------   4.00m  /dev/sdg1(21)                                                                                                                 linear
  takeover_rmeta_3    centipede2  -wi-------   4.00m  /dev/sda1(21)                                                                                                                 linear
  takeover_rmeta_4    centipede2  -wi-------   4.00m  /dev/sdc1(21)                                                                                                                 linear
  takeover_rmeta_5    centipede2  -wi-------   4.00m  /dev/sdf1(21)                                                                                                                 linear



# Striped -> Raid5 (WITHOUT enough PVs to satisfy takeover)
[root@host-073 ~]# lvcreate  --type striped -i 7 -n takeover -L 500M centipede2
  Using default stripesize 64.00 KiB.
  Rounding size 500.00 MiB (125 extents) up to stripe boundary size 504.00 MiB(126 extents).
  Logical volume "takeover" created.
[root@host-073 ~]# lvconvert --yes --type raid4 centipede2/takeover
  Using default stripesize 64.00 KiB.
  Insufficient suitable allocatable extents for logical volume : 19 more required
[root@host-073 ~]# lvs -a -o +devices,segtype
  LV                         VG          Attr       LSize    Devices                                                                                                                                            Type  
  takeover                   centipede2  rwi-a-r--- 504.00m  takeover_rimage_0(0),takeover_rimage_1(0),takeover_rimage_2(0),takeover_rimage_3(0),takeover_rimage_4(0),takeover_rimage_5(0),takeover_rimage_6(0) raid0 
  [takeover_rimage_0]        centipede2  iwi-aor---  72.00m  /dev/sde1(0)                                                                                                                                       linear
  [takeover_rimage_1]        centipede2  iwi-aor---  72.00m  /dev/sdd1(0)                                                                                                                                       linear
  [takeover_rimage_2]        centipede2  iwi-aor---  72.00m  /dev/sdg1(0)                                                                                                                                       linear
  [takeover_rimage_3]        centipede2  iwi-aor---  72.00m  /dev/sda1(0)                                                                                                                                       linear
  [takeover_rimage_4]        centipede2  iwi-aor---  72.00m  /dev/sdc1(0)                                                                                                                                       linear
  [takeover_rimage_5]        centipede2  iwi-aor---  72.00m  /dev/sdf1(0)                                                                                                                                       linear
  [takeover_rimage_6]        centipede2  iwi-aor---  72.00m  /dev/sdh1(0)                                                                                                                                       linear
  takeover_rmeta_0_extracted centipede2  -wi-------   4.00m  /dev/sde1(18)                                                                                                                                      linear
  takeover_rmeta_1_extracted centipede2  -wi-------   4.00m  /dev/sdd1(18)                                                                                                                                      linear
  takeover_rmeta_2_extracted centipede2  -wi-------   4.00m  /dev/sdg1(18)                                                                                                                                      linear
  takeover_rmeta_3_extracted centipede2  -wi-------   4.00m  /dev/sda1(18)                                                                                                                                      linear
  takeover_rmeta_4_extracted centipede2  -wi-------   4.00m  /dev/sdc1(18)                                                                                                                                      linear
  takeover_rmeta_5_extracted centipede2  -wi-------   4.00m  /dev/sdf1(18)                                                                                                                                      linear
  takeover_rmeta_6_extracted centipede2  -wi-------   4.00m  /dev/sdh1(18)                                                                                                                                      linear


Version-Release number of selected component (if applicable):
3.10.0-635.el7.x86_64

lvm2-2.02.169-3.el7    BUILT: Wed Mar 29 09:17:46 CDT 2017
lvm2-libs-2.02.169-3.el7    BUILT: Wed Mar 29 09:17:46 CDT 2017
lvm2-cluster-2.02.169-3.el7    BUILT: Wed Mar 29 09:17:46 CDT 2017
device-mapper-1.02.138-3.el7    BUILT: Wed Mar 29 09:17:46 CDT 2017
device-mapper-libs-1.02.138-3.el7    BUILT: Wed Mar 29 09:17:46 CDT 2017
device-mapper-event-1.02.138-3.el7    BUILT: Wed Mar 29 09:17:46 CDT 2017
device-mapper-event-libs-1.02.138-3.el7    BUILT: Wed Mar 29 09:17:46 CDT 2017
device-mapper-persistent-data-0.7.0-0.1.rc6.el7    BUILT: Mon Mar 27 10:15:46 CDT 2017


How reproducible:
Everytime

Comment 4 Corey Marthaler 2017-04-17 15:38:50 UTC
The raid4 scenario is still not fixed yet.

3.10.0-651.el7.x86_64

lvm2-2.02.170-2.el7    BUILT: Thu Apr 13 14:37:43 CDT 2017
lvm2-libs-2.02.170-2.el7    BUILT: Thu Apr 13 14:37:43 CDT 2017
lvm2-cluster-2.02.170-2.el7    BUILT: Thu Apr 13 14:37:43 CDT 2017
device-mapper-1.02.139-2.el7    BUILT: Thu Apr 13 14:37:43 CDT 2017
device-mapper-libs-1.02.139-2.el7    BUILT: Thu Apr 13 14:37:43 CDT 2017
device-mapper-event-1.02.139-2.el7    BUILT: Thu Apr 13 14:37:43 CDT 2017
device-mapper-event-libs-1.02.139-2.el7    BUILT: Thu Apr 13 14:37:43 CDT 2017
device-mapper-persistent-data-0.7.0-0.1.rc6.el7    BUILT: Mon Mar 27 10:15:46 CDT 2017



[root@host-128 ~]# pvscan
  PV /dev/sda1   VG VG              lvm2 [<21.00 GiB / <21.00 GiB free]
  PV /dev/sdb1   VG VG              lvm2 [<21.00 GiB / <21.00 GiB free]
  PV /dev/sdc1   VG VG              lvm2 [<21.00 GiB / <21.00 GiB free]
  PV /dev/sdd1   VG VG              lvm2 [<21.00 GiB / <21.00 GiB free]
  PV /dev/sde1   VG VG              lvm2 [<21.00 GiB / <21.00 GiB free]
  PV /dev/sdf1   VG VG              lvm2 [<21.00 GiB / <21.00 GiB free]

[root@host-128 ~]#  lvcreate  --type striped -i 6 -n takeover -L 500M VG
  Using default stripesize 64.00 KiB.
  Rounding size 500.00 MiB (125 extents) up to stripe boundary size 504.00 MiB(126 extents).
  Logical volume "takeover" created.

[root@host-128 ~]# lvs -a -o +devices,segtype
  LV       VG Attr       LSize   Cpy%Sync Devices                                                                       Type   
  takeover VG -wi-a----- 504.00m          /dev/sda1(0),/dev/sdb1(0),/dev/sdc1(0),/dev/sdd1(0),/dev/sde1(0),/dev/sdf1(0) striped


# stripe -> raid6 (now properly fails)
[root@host-128 ~]# lvconvert --yes --type raid6 VG/takeover
  Using default stripesize 64.00 KiB.
  Replaced LV type raid6 with possible type raid6_n_6.
  Insufficient suitable allocatable extents for logical volume : 44 more required
[root@host-128 ~]# lvs -a -o +devices,segtype
  LV       VG Attr       LSize   Cpy%Sync Devices                                                                       Type   
  takeover VG -wi-a----- 504.00m          /dev/sda1(0),/dev/sdb1(0),/dev/sdc1(0),/dev/sdd1(0),/dev/sde1(0),/dev/sdf1(0) striped


# stripe -> raid5 (also now properly fails)
[root@host-128 ~]# lvconvert --yes --type raid5 VG/takeover
  Using default stripesize 64.00 KiB.
  Replaced LV type raid5 with possible type raid5_n.
  Insufficient suitable allocatable extents for logical volume : 22 more required
[root@host-128 ~]# lvs -a -o +devices,segtype
  LV       VG Attr       LSize   Cpy%Sync Devices                                                                       Type   
  takeover VG -wi-a----- 504.00m          /dev/sda1(0),/dev/sdb1(0),/dev/sdc1(0),/dev/sdd1(0),/dev/sde1(0),/dev/sdf1(0) striped


# stripe -> raid4 (*DOES NOT* properly fail yet)
[root@host-128 ~]# lvconvert --yes --type raid4 VG/takeover
  Using default stripesize 64.00 KiB.
  Insufficient suitable allocatable extents for logical volume : 22 more required
  Device VG-takeover_rimage_0 (253:3) is used by another device.
[root@host-128 ~]# lvs -a -o +devices,segtype
  LV                         VG Attr       LSize   Cpy%Sync Devices                                                                                                                       Type  
  takeover                   VG rwi-a-r--- 504.00m          takeover_rimage_0(0),takeover_rimage_1(0),takeover_rimage_2(0),takeover_rimage_3(0),takeover_rimage_4(0),takeover_rimage_5(0) raid0 
  [takeover_rimage_0]        VG iwi-aor---  84.00m          /dev/sda1(0)                                                                                                                  linear
  [takeover_rimage_1]        VG iwi-aor---  84.00m          /dev/sdb1(0)                                                                                                                  linear
  [takeover_rimage_2]        VG iwi-aor---  84.00m          /dev/sdc1(0)                                                                                                                  linear
  [takeover_rimage_3]        VG iwi-aor---  84.00m          /dev/sdd1(0)                                                                                                                  linear
  [takeover_rimage_4]        VG iwi-aor---  84.00m          /dev/sde1(0)                                                                                                                  linear
  [takeover_rimage_5]        VG iwi-aor---  84.00m          /dev/sdf1(0)                                                                                                                  linear
  takeover_rmeta_0_extracted VG -wi-------   4.00m          /dev/sda1(21)                                                                                                                 linear
  takeover_rmeta_1_extracted VG -wi-------   4.00m          /dev/sdb1(21)                                                                                                                 linear
  takeover_rmeta_2_extracted VG -wi-------   4.00m          /dev/sdc1(21)                                                                                                                 linear
  takeover_rmeta_3_extracted VG -wi-------   4.00m          /dev/sdd1(21)                                                                                                                 linear
  takeover_rmeta_4_extracted VG -wi-------   4.00m          /dev/sde1(21)                                                                                                                 linear
  takeover_rmeta_5_extracted VG -wi-------   4.00m          /dev/sdf1(21)                                                                                                                 linear

Comment 5 Heinz Mauelshagen 2017-04-24 14:25:54 UTC
Upstream commit 8f305f025ed862689a8b4e34557e6a469201cba0

Comment 6 Corey Marthaler 2017-06-01 19:03:36 UTC
Marking verified in the latest rpms. All stripe -> raid cases now properly fail.

lvm2-2.02.171-3.el7    BUILT: Wed May 31 08:36:29 CDT 2017
lvm2-libs-2.02.171-3.el7    BUILT: Wed May 31 08:36:29 CDT 2017
lvm2-cluster-2.02.171-3.el7    BUILT: Wed May 31 08:36:29 CDT 2017
device-mapper-1.02.140-3.el7    BUILT: Wed May 31 08:36:29 CDT 2017
device-mapper-libs-1.02.140-3.el7    BUILT: Wed May 31 08:36:29 CDT 2017
device-mapper-event-1.02.140-3.el7    BUILT: Wed May 31 08:36:29 CDT 2017
device-mapper-event-libs-1.02.140-3.el7    BUILT: Wed May 31 08:36:29 CDT 2017
device-mapper-persistent-data-0.7.0-0.1.rc6.el7    BUILT: Mon Mar 27 10:15:46 CDT 2017



[root@host-128 ~]# lvcreate  --type striped -i 6 -n takeover -L 500M VG
  Using default stripesize 64.00 KiB.
  Rounding size 500.00 MiB (125 extents) up to stripe boundary size 504.00 MiB(126 extents).
  Logical volume "takeover" created.
[root@host-128 ~]# lvs -a -o +devices,segtype
  LV       VG  Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices                                                                       Type      
  takeover VG  -wi-a----- 504.00m                                                     /dev/sda1(0),/dev/sdb1(0),/dev/sdc1(0),/dev/sdd1(0),/dev/sde1(0),/dev/sdf1(0) striped

[root@host-128 ~]# lvconvert --yes --type raid6 VG/takeover
  Using default stripesize 64.00 KiB.
  Replaced LV type raid6 (same as raid6_zr) with possible type raid6_n_6.
  Repeat this command to convert to raid6 after an interim conversion has finished.
  Insufficient suitable allocatable extents for logical volume : 44 more required

[root@host-128 ~]# lvconvert --yes --type raid5 VG/takeover
  Using default stripesize 64.00 KiB.
  Replaced LV type raid5 (same as raid5_ls) with possible type raid5_n.
  Repeat this command to convert to raid5 after an interim conversion has finished.
  Insufficient suitable allocatable extents for logical volume : 22 more required

[root@host-128 ~]# lvconvert --yes --type raid4 VG/takeover
  Using default stripesize 64.00 KiB.
  Insufficient suitable allocatable extents for logical volume : 22 more required

[root@host-128 ~]# lvconvert --yes --type raid10 VG/takeover
  Using default stripesize 64.00 KiB.
  Insufficient suitable allocatable extents for logical volume : 132 more required

Comment 7 errata-xmlrpc 2017-08-01 21:52:19 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2222