Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
DescriptionCorey Marthaler
2020-05-11 19:34:54 UTC
Cloning this bug so that it can be dealt with in rhel8 (if desired) and the
issue 1504044 can be closed in rhel7 as it's too late to fix there.
kernel-4.18.0-193.2.1.el8_2 BUILT: Mon May 4 09:57:10 CDT 2020
lvm2-2.03.08-3.el8 BUILT: Wed Mar 18 08:37:57 CDT 2020
lvm2-libs-2.03.08-3.el8 BUILT: Wed Mar 18 08:37:57 CDT 2020
lvm2-dbusd-2.03.08-3.el8 BUILT: Wed Mar 18 08:40:15 CDT 2020
lvm2-lockd-2.03.08-3.el8 BUILT: Wed Mar 18 08:37:57 CDT 2020
boom-boot-1.0-1.el8 BUILT: Fri Nov 29 05:18:30 CST 2019
device-mapper-1.02.169-3.el8 BUILT: Wed Mar 18 08:37:57 CDT 2020
device-mapper-libs-1.02.169-3.el8 BUILT: Wed Mar 18 08:37:57 CDT 2020
device-mapper-event-1.02.169-3.el8 BUILT: Wed Mar 18 08:37:57 CDT 2020
device-mapper-event-libs-1.02.169-3.el8 BUILT: Wed Mar 18 08:37:57 CDT 2020
device-mapper-persistent-data-0.8.5-3.el8 BUILT: Wed Nov 27 07:05:21 CST 2019
[root@hayes-03 ~]# lvcreate --type raid1 -L 2G -n Thin -m 1 vg
Logical volume "Thin" created.
[root@hayes-03 ~]# lvcreate --type raid1 -L 100M -n boot -m 1 vg
Logical volume "boot" created.
[root@hayes-03 ~]# lvcreate --type raid1 -L 100M -n thinmeta -m 1 vg
Logical volume "thinmeta" created.
[root@hayes-03 ~]# lvs -a -o +devices,segtype
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices Type
Thin vg rwi-a-r--- 2.00g 100.00 Thin_rimage_0(0),Thin_rimage_1(0) raid1
[Thin_rimage_0] vg iwi-aor--- 2.00g /dev/sdb1(1) linear
[Thin_rimage_1] vg iwi-aor--- 2.00g /dev/sdc1(1) linear
[Thin_rmeta_0] vg ewi-aor--- 4.00m /dev/sdb1(0) linear
[Thin_rmeta_1] vg ewi-aor--- 4.00m /dev/sdc1(0) linear
boot vg rwi-a-r--- 100.00m 100.00 boot_rimage_0(0),boot_rimage_1(0) raid1
[boot_rimage_0] vg iwi-aor--- 100.00m /dev/sdb1(514) linear
[boot_rimage_1] vg iwi-aor--- 100.00m /dev/sdc1(514) linear
[boot_rmeta_0] vg ewi-aor--- 4.00m /dev/sdb1(513) linear
[boot_rmeta_1] vg ewi-aor--- 4.00m /dev/sdc1(513) linear
thinmeta vg rwi-a-r--- 100.00m 100.00 thinmeta_rimage_0(0),thinmeta_rimage_1(0) raid1
[thinmeta_rimage_0] vg iwi-aor--- 100.00m /dev/sdb1(540) linear
[thinmeta_rimage_1] vg iwi-aor--- 100.00m /dev/sdc1(540) linear
[thinmeta_rmeta_0] vg ewi-aor--- 4.00m /dev/sdb1(539) linear
[thinmeta_rmeta_1] vg ewi-aor--- 4.00m /dev/sdc1(539) linear
[root@hayes-03 ~]# lvconvert -y --type thin-pool --poolmetadata vg/thinmeta vg/Thin
Thin pool volume with chunk size 64.00 KiB can address at most 15.81 TiB of data.
WARNING: Converting vg/Thin and vg/thinmeta to thin pool's data and metadata volumes with metadata wiping.
THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.)
Converted vg/Thin and vg/thinmeta to thin pool.
[root@hayes-03 ~]# vgchange -an vg
0 logical volume(s) in volume group "vg" now active
[root@hayes-03 ~]# lvs -a -o +devices
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices
Thin vg twi---tz-- 2.00g Thin_tdata(0)
[Thin_tdata] vg rwi---r--- 2.00g Thin_tdata_rimage_0(0),Thin_tdata_rimage_1(0)
[Thin_tdata_rimage_0] vg Iwi---r--- 2.00g /dev/sdb1(1)
[Thin_tdata_rimage_1] vg Iwi---r--- 2.00g /dev/sdc1(1)
[Thin_tdata_rmeta_0] vg ewi---r--- 4.00m /dev/sdb1(0)
[Thin_tdata_rmeta_1] vg ewi---r--- 4.00m /dev/sdc1(0)
[Thin_tmeta] vg ewi---r--- 100.00m Thin_tmeta_rimage_0(0),Thin_tmeta_rimage_1(0)
[Thin_tmeta_rimage_0] vg Iwi---r--- 100.00m /dev/sdb1(540)
[Thin_tmeta_rimage_1] vg Iwi---r--- 100.00m /dev/sdc1(540)
[Thin_tmeta_rmeta_0] vg ewi---r--- 4.00m /dev/sdb1(539)
[Thin_tmeta_rmeta_1] vg ewi---r--- 4.00m /dev/sdc1(539)
boot vg rwi---r--- 100.00m boot_rimage_0(0),boot_rimage_1(0)
[boot_rimage_0] vg Iwi---r--- 100.00m /dev/sdb1(514)
[boot_rimage_1] vg Iwi---r--- 100.00m /dev/sdc1(514)
[boot_rmeta_0] vg ewi---r--- 4.00m /dev/sdb1(513)
[boot_rmeta_1] vg ewi---r--- 4.00m /dev/sdc1(513)
[lvol0_pmspare] vg ewi------- 100.00m /dev/sdb1(565)
# Set filter to:
filter = [ "r|^/dev/sdc1$|" ]
[root@hayes-03 ~]# lvs -a -o lv_name,lv_attr,devices
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to /dev/sdc1).
LV Attr Devices
Thin twi---tzp- Thin_tdata(0)
[Thin_tdata] rwi---r-p- Thin_tdata_rimage_0(0),Thin_tdata_rimage_1(0)
[Thin_tdata_rimage_0] Iwi---r--- /dev/sdb1(1)
[Thin_tdata_rimage_1] Iwi---r-p- [unknown](1)
[Thin_tdata_rmeta_0] ewi---r--- /dev/sdb1(0)
[Thin_tdata_rmeta_1] ewi---r-p- [unknown](0)
[Thin_tmeta] ewi---r-p- Thin_tmeta_rimage_0(0),Thin_tmeta_rimage_1(0)
[Thin_tmeta_rimage_0] Iwi---r--- /dev/sdb1(540)
[Thin_tmeta_rimage_1] Iwi---r-p- [unknown](540)
[Thin_tmeta_rmeta_0] ewi---r--- /dev/sdb1(539)
[Thin_tmeta_rmeta_1] ewi---r-p- [unknown](539)
boot rwi---r-p- boot_rimage_0(0),boot_rimage_1(0)
[boot_rimage_0] Iwi---r--- /dev/sdb1(514)
[boot_rimage_1] Iwi---r-p- [unknown](514)
[boot_rmeta_0] ewi---r--- /dev/sdb1(513)
[boot_rmeta_1] ewi---r-p- [unknown](513)
[lvol0_pmspare] ewi------- /dev/sdb1(565)
### degraded activation mode fails for thinpool
[root@hayes-03 ~]# vgchange -a y vg --activationmode degraded
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to /dev/sdc1).
Refusing activation of partial LV vg/Thin. Use '--activationmode partial' to override.
1 logical volume(s) in volume group "vg" now active
[root@hayes-03 ~]# lvs -a -o lv_name,lv_attr,devices
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to /dev/sdc1).
LV Attr Devices
Thin twi---tzp- Thin_tdata(0)
[Thin_tdata] rwi---r-p- Thin_tdata_rimage_0(0),Thin_tdata_rimage_1(0)
[Thin_tdata_rimage_0] Iwi---r--- /dev/sdb1(1)
[Thin_tdata_rimage_1] Iwi---r-p- [unknown](1)
[Thin_tdata_rmeta_0] ewi---r--- /dev/sdb1(0)
[Thin_tdata_rmeta_1] ewi---r-p- [unknown](0)
[Thin_tmeta] ewi---r-p- Thin_tmeta_rimage_0(0),Thin_tmeta_rimage_1(0)
[Thin_tmeta_rimage_0] Iwi---r--- /dev/sdb1(540)
[Thin_tmeta_rimage_1] Iwi---r-p- [unknown](540)
[Thin_tmeta_rmeta_0] ewi---r--- /dev/sdb1(539)
[Thin_tmeta_rmeta_1] ewi---r-p- [unknown](539)
boot rwi-a-r-p- boot_rimage_0(0),boot_rimage_1(0)
[boot_rimage_0] iwi-aor--- /dev/sdb1(514)
[boot_rimage_1] Iwi-aor-p- [unknown](514)
[boot_rmeta_0] ewi-aor--- /dev/sdb1(513)
[boot_rmeta_1] ewi-aor-p- [unknown](513)
[lvol0_pmspare] ewi------- /dev/sdb1(565)
### partial mode works for thinpool
[root@hayes-03 ~]# vgchange -a y vg --activationmode partial
PARTIAL MODE. Incomplete logical volumes will be processed.
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to /dev/sdc1).
2 logical volume(s) in volume group "vg" now active
[root@hayes-03 ~]# lvs -a -o lv_name,lv_attr,devices
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to /dev/sdc1).
LV Attr Devices
Thin twi-a-tzp- Thin_tdata(0)
[Thin_tdata] rwi-aor-p- Thin_tdata_rimage_0(0),Thin_tdata_rimage_1(0)
[Thin_tdata_rimage_0] iwi-aor--- /dev/sdb1(1)
[Thin_tdata_rimage_1] Iwi-aor-p- [unknown](1)
[Thin_tdata_rmeta_0] ewi-aor--- /dev/sdb1(0)
[Thin_tdata_rmeta_1] ewi-aor-p- [unknown](0)
[Thin_tmeta] ewi-aor-p- Thin_tmeta_rimage_0(0),Thin_tmeta_rimage_1(0)
[Thin_tmeta_rimage_0] iwi-aor--- /dev/sdb1(540)
[Thin_tmeta_rimage_1] Iwi-aor-p- [unknown](540)
[Thin_tmeta_rmeta_0] ewi-aor--- /dev/sdb1(539)
[Thin_tmeta_rmeta_1] ewi-aor-p- [unknown](539)
boot rwi-a-r-p- boot_rimage_0(0),boot_rimage_1(0)
[boot_rimage_0] iwi-aor--- /dev/sdb1(514)
[boot_rimage_1] Iwi-aor-p- [unknown](514)
[boot_rmeta_0] ewi-aor--- /dev/sdb1(513)
[boot_rmeta_1] ewi-aor-p- [unknown](513)
[lvol0_pmspare] ewi------- /dev/sdb1(565)
###The repair attempt on non-thin worked for me when I skipped the vgreduce --removemissing part.
[root@hayes-03 ~]# vgextend vg /dev/sdd1
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to /dev/sdc1).
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to /dev/sdc1).
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
Volume group "vg" successfully extended
[root@hayes-03 ~]# lvconvert --repair /dev/vg/boot /dev/sdd1
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to [unknown]).
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
Attempt to replace failed RAID images (requires full device resync)? [y/n]: y
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
Faulty devices in vg/boot successfully replaced.
### However, trying to repair thinpool directly does not work (unsupported?)
[root@hayes-03 ~]# lvchange -an vg/Thin
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to [unknown]).
[root@hayes-03 ~]# lvconvert --repair /dev/vg/Thin /dev/sdd1
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to [unknown]).
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: LV vg/Thin_meta0 holds a backup of the unrepaired metadata. Use lvremove when no longer required.
WARNING: New metadata LV vg/Thin_tmeta might use different PVs. Move it with pvmove if required.
[root@hayes-03 ~]# lvs -a -o lv_name,lv_attr,devices
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to [unknown]).
LV Attr Devices
Thin twi---tzp- Thin_tdata(0)
Thin_meta0 rwi---r-p- Thin_meta0_rimage_0(0),Thin_meta0_rimage_1(0)
[Thin_meta0_rimage_0] Iwi---r--- /dev/sdb1(540)
[Thin_meta0_rimage_1] Iwi---r-p- [unknown](540)
[Thin_meta0_rmeta_0] ewi---r--- /dev/sdb1(539)
[Thin_meta0_rmeta_1] ewi---r-p- [unknown](539)
[Thin_tdata] rwi---r-p- Thin_tdata_rimage_0(0),Thin_tdata_rimage_1(0)
[Thin_tdata_rimage_0] Iwi---r--- /dev/sdb1(1)
[Thin_tdata_rimage_1] Iwi---r-p- [unknown](1)
[Thin_tdata_rmeta_0] ewi---r--- /dev/sdb1(0)
[Thin_tdata_rmeta_1] ewi---r-p- [unknown](0)
[Thin_tmeta] ewi------- /dev/sdb1(565)
boot rwi-a-r--- boot_rimage_0(0),boot_rimage_1(0)
[boot_rimage_0] iwi-aor--- /dev/sdb1(514)
[boot_rimage_1] iwi-aor--- /dev/sdd1(1)
[boot_rmeta_0] ewi-aor--- /dev/sdb1(513)
[boot_rmeta_1] ewi-aor--- /dev/sdd1(0)
[lvol1_pmspare] ewi------- /dev/sdd1(26)
### Repairing tmeta and tdata volumes separately as Jon suggested in Comment 4 should do the trick:
[root@hayes-03 ~]# lvchange -ay --activationmode partial vg/Thin
PARTIAL MODE. Incomplete logical volumes will be processed.
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to [unknown]).
[root@hayes-03 ~]# lvchange -ay --activationmode partial vg/Thin_meta0
PARTIAL MODE. Incomplete logical volumes will be processed.
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to [unknown]).
[root@hayes-03 ~]# lvconvert --repair vg/Thin_tdata /dev/sdd1
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to [unknown]).
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
Attempt to replace failed RAID images (requires full device resync)? [y/n]: y
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
Faulty devices in vg/Thin_tdata successfully replaced.
[root@hayes-03 ~]# lvconvert --repair vg/Thin_meta /dev/sdd1
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to [unknown]).
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
Failed to find logical volume "vg/Thin_meta"
Attempt to replace failed RAID images (requires full device resync)? [y/n]: y
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN.
Faulty devices in vg/Thin_meta0 successfully replaced.
Comment 4RHEL Program Management
2021-11-11 07:27:09 UTC
After evaluating this issue, there are no plans to address it further or fix it in an upcoming release. Therefore, it is being closed. If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.
Cloning this bug so that it can be dealt with in rhel8 (if desired) and the issue 1504044 can be closed in rhel7 as it's too late to fix there. kernel-4.18.0-193.2.1.el8_2 BUILT: Mon May 4 09:57:10 CDT 2020 lvm2-2.03.08-3.el8 BUILT: Wed Mar 18 08:37:57 CDT 2020 lvm2-libs-2.03.08-3.el8 BUILT: Wed Mar 18 08:37:57 CDT 2020 lvm2-dbusd-2.03.08-3.el8 BUILT: Wed Mar 18 08:40:15 CDT 2020 lvm2-lockd-2.03.08-3.el8 BUILT: Wed Mar 18 08:37:57 CDT 2020 boom-boot-1.0-1.el8 BUILT: Fri Nov 29 05:18:30 CST 2019 device-mapper-1.02.169-3.el8 BUILT: Wed Mar 18 08:37:57 CDT 2020 device-mapper-libs-1.02.169-3.el8 BUILT: Wed Mar 18 08:37:57 CDT 2020 device-mapper-event-1.02.169-3.el8 BUILT: Wed Mar 18 08:37:57 CDT 2020 device-mapper-event-libs-1.02.169-3.el8 BUILT: Wed Mar 18 08:37:57 CDT 2020 device-mapper-persistent-data-0.8.5-3.el8 BUILT: Wed Nov 27 07:05:21 CST 2019 [root@hayes-03 ~]# lvcreate --type raid1 -L 2G -n Thin -m 1 vg Logical volume "Thin" created. [root@hayes-03 ~]# lvcreate --type raid1 -L 100M -n boot -m 1 vg Logical volume "boot" created. [root@hayes-03 ~]# lvcreate --type raid1 -L 100M -n thinmeta -m 1 vg Logical volume "thinmeta" created. [root@hayes-03 ~]# lvs -a -o +devices,segtype LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices Type Thin vg rwi-a-r--- 2.00g 100.00 Thin_rimage_0(0),Thin_rimage_1(0) raid1 [Thin_rimage_0] vg iwi-aor--- 2.00g /dev/sdb1(1) linear [Thin_rimage_1] vg iwi-aor--- 2.00g /dev/sdc1(1) linear [Thin_rmeta_0] vg ewi-aor--- 4.00m /dev/sdb1(0) linear [Thin_rmeta_1] vg ewi-aor--- 4.00m /dev/sdc1(0) linear boot vg rwi-a-r--- 100.00m 100.00 boot_rimage_0(0),boot_rimage_1(0) raid1 [boot_rimage_0] vg iwi-aor--- 100.00m /dev/sdb1(514) linear [boot_rimage_1] vg iwi-aor--- 100.00m /dev/sdc1(514) linear [boot_rmeta_0] vg ewi-aor--- 4.00m /dev/sdb1(513) linear [boot_rmeta_1] vg ewi-aor--- 4.00m /dev/sdc1(513) linear thinmeta vg rwi-a-r--- 100.00m 100.00 thinmeta_rimage_0(0),thinmeta_rimage_1(0) raid1 [thinmeta_rimage_0] vg iwi-aor--- 100.00m /dev/sdb1(540) linear [thinmeta_rimage_1] vg iwi-aor--- 100.00m /dev/sdc1(540) linear [thinmeta_rmeta_0] vg ewi-aor--- 4.00m /dev/sdb1(539) linear [thinmeta_rmeta_1] vg ewi-aor--- 4.00m /dev/sdc1(539) linear [root@hayes-03 ~]# lvconvert -y --type thin-pool --poolmetadata vg/thinmeta vg/Thin Thin pool volume with chunk size 64.00 KiB can address at most 15.81 TiB of data. WARNING: Converting vg/Thin and vg/thinmeta to thin pool's data and metadata volumes with metadata wiping. THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.) Converted vg/Thin and vg/thinmeta to thin pool. [root@hayes-03 ~]# vgchange -an vg 0 logical volume(s) in volume group "vg" now active [root@hayes-03 ~]# lvs -a -o +devices LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices Thin vg twi---tz-- 2.00g Thin_tdata(0) [Thin_tdata] vg rwi---r--- 2.00g Thin_tdata_rimage_0(0),Thin_tdata_rimage_1(0) [Thin_tdata_rimage_0] vg Iwi---r--- 2.00g /dev/sdb1(1) [Thin_tdata_rimage_1] vg Iwi---r--- 2.00g /dev/sdc1(1) [Thin_tdata_rmeta_0] vg ewi---r--- 4.00m /dev/sdb1(0) [Thin_tdata_rmeta_1] vg ewi---r--- 4.00m /dev/sdc1(0) [Thin_tmeta] vg ewi---r--- 100.00m Thin_tmeta_rimage_0(0),Thin_tmeta_rimage_1(0) [Thin_tmeta_rimage_0] vg Iwi---r--- 100.00m /dev/sdb1(540) [Thin_tmeta_rimage_1] vg Iwi---r--- 100.00m /dev/sdc1(540) [Thin_tmeta_rmeta_0] vg ewi---r--- 4.00m /dev/sdb1(539) [Thin_tmeta_rmeta_1] vg ewi---r--- 4.00m /dev/sdc1(539) boot vg rwi---r--- 100.00m boot_rimage_0(0),boot_rimage_1(0) [boot_rimage_0] vg Iwi---r--- 100.00m /dev/sdb1(514) [boot_rimage_1] vg Iwi---r--- 100.00m /dev/sdc1(514) [boot_rmeta_0] vg ewi---r--- 4.00m /dev/sdb1(513) [boot_rmeta_1] vg ewi---r--- 4.00m /dev/sdc1(513) [lvol0_pmspare] vg ewi------- 100.00m /dev/sdb1(565) # Set filter to: filter = [ "r|^/dev/sdc1$|" ] [root@hayes-03 ~]# lvs -a -o lv_name,lv_attr,devices WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to /dev/sdc1). LV Attr Devices Thin twi---tzp- Thin_tdata(0) [Thin_tdata] rwi---r-p- Thin_tdata_rimage_0(0),Thin_tdata_rimage_1(0) [Thin_tdata_rimage_0] Iwi---r--- /dev/sdb1(1) [Thin_tdata_rimage_1] Iwi---r-p- [unknown](1) [Thin_tdata_rmeta_0] ewi---r--- /dev/sdb1(0) [Thin_tdata_rmeta_1] ewi---r-p- [unknown](0) [Thin_tmeta] ewi---r-p- Thin_tmeta_rimage_0(0),Thin_tmeta_rimage_1(0) [Thin_tmeta_rimage_0] Iwi---r--- /dev/sdb1(540) [Thin_tmeta_rimage_1] Iwi---r-p- [unknown](540) [Thin_tmeta_rmeta_0] ewi---r--- /dev/sdb1(539) [Thin_tmeta_rmeta_1] ewi---r-p- [unknown](539) boot rwi---r-p- boot_rimage_0(0),boot_rimage_1(0) [boot_rimage_0] Iwi---r--- /dev/sdb1(514) [boot_rimage_1] Iwi---r-p- [unknown](514) [boot_rmeta_0] ewi---r--- /dev/sdb1(513) [boot_rmeta_1] ewi---r-p- [unknown](513) [lvol0_pmspare] ewi------- /dev/sdb1(565) ### degraded activation mode fails for thinpool [root@hayes-03 ~]# vgchange -a y vg --activationmode degraded WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to /dev/sdc1). Refusing activation of partial LV vg/Thin. Use '--activationmode partial' to override. 1 logical volume(s) in volume group "vg" now active [root@hayes-03 ~]# lvs -a -o lv_name,lv_attr,devices WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to /dev/sdc1). LV Attr Devices Thin twi---tzp- Thin_tdata(0) [Thin_tdata] rwi---r-p- Thin_tdata_rimage_0(0),Thin_tdata_rimage_1(0) [Thin_tdata_rimage_0] Iwi---r--- /dev/sdb1(1) [Thin_tdata_rimage_1] Iwi---r-p- [unknown](1) [Thin_tdata_rmeta_0] ewi---r--- /dev/sdb1(0) [Thin_tdata_rmeta_1] ewi---r-p- [unknown](0) [Thin_tmeta] ewi---r-p- Thin_tmeta_rimage_0(0),Thin_tmeta_rimage_1(0) [Thin_tmeta_rimage_0] Iwi---r--- /dev/sdb1(540) [Thin_tmeta_rimage_1] Iwi---r-p- [unknown](540) [Thin_tmeta_rmeta_0] ewi---r--- /dev/sdb1(539) [Thin_tmeta_rmeta_1] ewi---r-p- [unknown](539) boot rwi-a-r-p- boot_rimage_0(0),boot_rimage_1(0) [boot_rimage_0] iwi-aor--- /dev/sdb1(514) [boot_rimage_1] Iwi-aor-p- [unknown](514) [boot_rmeta_0] ewi-aor--- /dev/sdb1(513) [boot_rmeta_1] ewi-aor-p- [unknown](513) [lvol0_pmspare] ewi------- /dev/sdb1(565) ### partial mode works for thinpool [root@hayes-03 ~]# vgchange -a y vg --activationmode partial PARTIAL MODE. Incomplete logical volumes will be processed. WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to /dev/sdc1). 2 logical volume(s) in volume group "vg" now active [root@hayes-03 ~]# lvs -a -o lv_name,lv_attr,devices WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to /dev/sdc1). LV Attr Devices Thin twi-a-tzp- Thin_tdata(0) [Thin_tdata] rwi-aor-p- Thin_tdata_rimage_0(0),Thin_tdata_rimage_1(0) [Thin_tdata_rimage_0] iwi-aor--- /dev/sdb1(1) [Thin_tdata_rimage_1] Iwi-aor-p- [unknown](1) [Thin_tdata_rmeta_0] ewi-aor--- /dev/sdb1(0) [Thin_tdata_rmeta_1] ewi-aor-p- [unknown](0) [Thin_tmeta] ewi-aor-p- Thin_tmeta_rimage_0(0),Thin_tmeta_rimage_1(0) [Thin_tmeta_rimage_0] iwi-aor--- /dev/sdb1(540) [Thin_tmeta_rimage_1] Iwi-aor-p- [unknown](540) [Thin_tmeta_rmeta_0] ewi-aor--- /dev/sdb1(539) [Thin_tmeta_rmeta_1] ewi-aor-p- [unknown](539) boot rwi-a-r-p- boot_rimage_0(0),boot_rimage_1(0) [boot_rimage_0] iwi-aor--- /dev/sdb1(514) [boot_rimage_1] Iwi-aor-p- [unknown](514) [boot_rmeta_0] ewi-aor--- /dev/sdb1(513) [boot_rmeta_1] ewi-aor-p- [unknown](513) [lvol0_pmspare] ewi------- /dev/sdb1(565) ###The repair attempt on non-thin worked for me when I skipped the vgreduce --removemissing part. [root@hayes-03 ~]# vgextend vg /dev/sdd1 WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to /dev/sdc1). WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to /dev/sdc1). WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. Volume group "vg" successfully extended [root@hayes-03 ~]# lvconvert --repair /dev/vg/boot /dev/sdd1 WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to [unknown]). WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. Attempt to replace failed RAID images (requires full device resync)? [y/n]: y WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. Faulty devices in vg/boot successfully replaced. ### However, trying to repair thinpool directly does not work (unsupported?) [root@hayes-03 ~]# lvchange -an vg/Thin WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to [unknown]). [root@hayes-03 ~]# lvconvert --repair /dev/vg/Thin /dev/sdd1 WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to [unknown]). WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: LV vg/Thin_meta0 holds a backup of the unrepaired metadata. Use lvremove when no longer required. WARNING: New metadata LV vg/Thin_tmeta might use different PVs. Move it with pvmove if required. [root@hayes-03 ~]# lvs -a -o lv_name,lv_attr,devices WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to [unknown]). LV Attr Devices Thin twi---tzp- Thin_tdata(0) Thin_meta0 rwi---r-p- Thin_meta0_rimage_0(0),Thin_meta0_rimage_1(0) [Thin_meta0_rimage_0] Iwi---r--- /dev/sdb1(540) [Thin_meta0_rimage_1] Iwi---r-p- [unknown](540) [Thin_meta0_rmeta_0] ewi---r--- /dev/sdb1(539) [Thin_meta0_rmeta_1] ewi---r-p- [unknown](539) [Thin_tdata] rwi---r-p- Thin_tdata_rimage_0(0),Thin_tdata_rimage_1(0) [Thin_tdata_rimage_0] Iwi---r--- /dev/sdb1(1) [Thin_tdata_rimage_1] Iwi---r-p- [unknown](1) [Thin_tdata_rmeta_0] ewi---r--- /dev/sdb1(0) [Thin_tdata_rmeta_1] ewi---r-p- [unknown](0) [Thin_tmeta] ewi------- /dev/sdb1(565) boot rwi-a-r--- boot_rimage_0(0),boot_rimage_1(0) [boot_rimage_0] iwi-aor--- /dev/sdb1(514) [boot_rimage_1] iwi-aor--- /dev/sdd1(1) [boot_rmeta_0] ewi-aor--- /dev/sdb1(513) [boot_rmeta_1] ewi-aor--- /dev/sdd1(0) [lvol1_pmspare] ewi------- /dev/sdd1(26) ### Repairing tmeta and tdata volumes separately as Jon suggested in Comment 4 should do the trick: [root@hayes-03 ~]# lvchange -ay --activationmode partial vg/Thin PARTIAL MODE. Incomplete logical volumes will be processed. WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to [unknown]). [root@hayes-03 ~]# lvchange -ay --activationmode partial vg/Thin_meta0 PARTIAL MODE. Incomplete logical volumes will be processed. WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to [unknown]). [root@hayes-03 ~]# lvconvert --repair vg/Thin_tdata /dev/sdd1 WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to [unknown]). WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. Attempt to replace failed RAID images (requires full device resync)? [y/n]: y WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. Faulty devices in vg/Thin_tdata successfully replaced. [root@hayes-03 ~]# lvconvert --repair vg/Thin_meta /dev/sdd1 WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: VG vg is missing PV 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN (last written to [unknown]). WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. Failed to find logical volume "vg/Thin_meta" Attempt to replace failed RAID images (requires full device resync)? [y/n]: y WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. WARNING: Couldn't find device with uuid 3YlSWL-FfwV-UpYu-efm4-Mjte-S8Yy-BtLWaN. Faulty devices in vg/Thin_meta0 successfully replaced.