Description of problem: If you have an exported mirrored LVM DG that is missing a mirror it is impossible to import the DG. Tested in 5.8 and 6.3... See following reproducer: [root@ll-chsystest03 logs]# dd if=/dev/zero of=loop0 bs=1M count=1000 1000+0 records in 1000+0 records out 1048576000 bytes (1.0 GB) copied, 0.583542 s, 1.8 GB/s [root@ll-chsystest03 logs]# dd if=/dev/zero of=loop1 bs=1M count=1000 1000+0 records in 1000+0 records out 1048576000 bytes (1.0 GB) copied, 0.593067 s, 1.8 GB/s [root@ll-chsystest03 logs]# losetup /dev/loop0 /logs/loop0 [root@ll-chsystest03 logs]# losetup /dev/loop1 /logs/loop1 [root@ll-chsystest03 logs]# pvcreate /dev/loop0 [root@ll-chsystest03 logs]# pvcreate /dev/loop1 [root@ll-chsystest03 logs]# vgcreate TRASHME /dev/loop0 /dev/loop1 [root@ll-chsystest03 logs]# lvcreate --contiguous n --alloc normal -m 1 -n trashme -L500m TRASHME /dev/loop0 /dev/loop1 Logical volume "trashme" created [root@ll-chsystest03 logs]# lvs LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert trashme TRASHME mwn-aom- 500.00m trashme_mlog 100.00 logs vg0 -wi-ao-- 520.69g root vg0 -wi-ao-- 14.00g tmp vg0 -wi-ao-- 2.00g var vg0 -wi-ao-- 6.00g [root@ll-chsystest03 logs]# vgs VG #PV #LV #SN Attr VSize VFree TRASHME 2 1 0 wz--n- 1.95g 988.00m vg0 1 4 0 wz--n- 542.69g 0 [root@ll-chsystest03 logs]# vgchange -an TRASHME 0 logical volume(s) in volume group "TRASHME" now active [root@ll-chsystest03 logs]# vgexport TRASHME Volume group "TRASHME" successfully exported [root@ll-chsystest03 logs]# losetup -d /dev/loop1 [root@ll-chsystest03 logs]# vgs Couldn't find device with uuid utPEIg-JC7U-4xP0-BXsu-oRl9-IjHS-4oyEGQ. VG #PV #LV #SN Attr VSize VFree TRASHME 2 1 0 wzxpn- 1.95g 988.00m vg0 1 4 0 wz--n- 542.69g 0 [root@ll-chsystest03 logs]# vgimport TRASHME Couldn't find device with uuid utPEIg-JC7U-4xP0-BXsu-oRl9-IjHS-4oyEGQ. Cannot change VG TRASHME while PVs are missing. Consider vgreduce --removemissing. [root@ll-chsystest03 logs]# vgreduce TRASHME --removemissing --force Couldn't find device with uuid utPEIg-JC7U-4xP0-BXsu-oRl9-IjHS-4oyEGQ. Unable to determine mirror sync status of TRASHME/trashme. Volume group "TRASHME" is exported Failed to lock trashme The "Failed to lock" problem is where the issue comes in. "vgreduce" cannot fix the problem because it can't get a lock on an exported diskgroup, and vgimport can't import the diskgroup since its "broken". Catch-22. Version-Release number of selected component (if applicable): RHEL5.8 and RHEL6.3 How reproducible: See above Additional info: Workaround #1 #this will give me the pv uuid vgdisplay --partial --verbose cd /etc/lvm/backup There should be a file there called TRASHME get a pv same size as what was there before, your are simulating losetup /dev/loop1 /logs/loop1 now restore the pv with same uuid and meta data from our /etc/lvm/backup/TRASHME file pvcreate --restorefile TRASHME --uuid mBR3gH-V90u-yOOk-V4BL-Llg0-UFH1-MfKbib /dev/loop1 -ff [root@localhost backup]# vgimport TRASHME Volume group "TRASHME" successfully imported Workaround #2 You could hand edit the backup file of the LVM metadata (generally found in /etc/lvm/backup/<VG NAME>). You would simply need to remove the 'EXPORTED' flag wherever it is found and then run vgcfgrestore
This bug is the same as bug 913644. It was fixed with the following upstream commit: https://bugzilla.redhat.com/show_bug.cgi?id=913644#c2 commit 3ab46449f4dbcc45fe878149838a8439f5ac8b34 Author: Jonathan Brassow <jbrassow> Date: Wed Feb 20 16:28:26 2013 -0600 vgimport: Allow '--force' to import VGs with missing PVs. When there are missing PVs in a volume group, most operations that alter the LVM metadata are disallowed. It turns out that 'vgimport' is one of those disallowed operations. This is bad because it creates a circular dependency. 'vgimport' will complain that the VG is inconsistent and that 'vgreduce --removemissing' must be run. However, 'vgreduce' cannot be run because it has not been imported. Therefore, 'vgimport' must be one of the operations allowed to change the metadata when PVs are missing. The '--force' option is the way to make 'vgimport' happen in spite of the missing PVs.
Verified that import can succeeed with suppliting --force with lvm2-2.02.88-11.el5 (04:59:32) [root@r5-node02:~]$ vgimport trashed Couldn't find device with uuid 6QXgTN-nbEw-kTvO-2IR7-s0tS-RwlG-GvGV9f. Cannot change VG trashed while PVs are missing. Consider vgreduce --removemissing. (04:59:42) [root@r5-node02:~]$ vgimport trashed --force '--force' supplied. Volume groups with missing PVs will be imported. Couldn't find device with uuid 6QXgTN-nbEw-kTvO-2IR7-s0tS-RwlG-GvGV9f. Volume group "trashed" successfully imported (04:59:55) [root@r5-node02:~]$ vgs Couldn't find device with uuid 6QXgTN-nbEw-kTvO-2IR7-s0tS-RwlG-GvGV9f. VG #PV #LV #SN Attr VSize VFree VolGroup00 1 2 0 wz--n- 8.88G 0 test 7 1 0 wz--n- 69.97G 69.48G trashed 2 1 0 wz-pn- 1.95G 988.00M
Marking verified with lvm2-2.02.88-11.el5 (08:12:18) [root@r5-node02:~]$ losetup -d /dev/loop1 (08:12:39) [root@r5-node02:~]$ vgs Couldn't find device with uuid oEuL7I-FJfA-kg0P-gbRm-VOUq-FAyE-Mc17Kn. VG #PV #LV #SN Attr VSize VFree VolGroup00 1 2 0 wz--n- 8.88G 0 test 4 3 0 wz--n- 39.98G 39.39G trashme 2 1 0 wzxpn- 1.95G 988.00M (08:12:43) [root@r5-node02:~]$ vgimport trashme Couldn't find device with uuid oEuL7I-FJfA-kg0P-gbRm-VOUq-FAyE-Mc17Kn. Cannot change VG trashme while PVs are missing. Consider vgreduce --removemissing. (08:12:53) [root@r5-node02:~]$ vgimport trashme --force '--force' supplied. Volume groups with missing PVs will be imported. Couldn't find device with uuid oEuL7I-FJfA-kg0P-gbRm-VOUq-FAyE-Mc17Kn. Volume group "trashme" successfully imported (08:12:56) [root@r5-node02:~]$ vgs Couldn't find device with uuid oEuL7I-FJfA-kg0P-gbRm-VOUq-FAyE-Mc17Kn. VG #PV #LV #SN Attr VSize VFree VolGroup00 1 2 0 wz--n- 8.88G 0 test 4 3 0 wz--n- 39.98G 39.39G trashme 2 1 0 wz-pn- 1.95G 988.00M
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2013-1352.html