Bug 636991

Finally, I checked in some code.  If it can't complete the log allocation, the code now attempts to place the logs on already-used devices.  There are still some hopefully-obscure cases it doesn't handle, but I hope what it does now is adequate. There's an lvm.conf setting to (largely) disable the new behaviour.

http://www.redhat.com/archives/lvm-devel/2011-February/msg00143.html

Change to allocation policy need to be tested in upstream code, because there was not yet any upstream release including it, changes in 5.7 are too risky.

Moving this to 5.8 timeframe (it will appear in rebased lvm2 code in 5.8).

This is a nasty one that bit us as we upgraded from 5.3 to 5.6...

I found a workaround:  create the mirrored volumes with --corelog, then convert the log to disk with the --alloc anywhere option.

Alternatively I think this methodology still works

lvcreate --alloc anywhere -m1 -l 254 -n redo1 vgeflxpdb1adm /dev/mapper/cx1430_lun34:0-253 /dev/mapper/cx1431_lun34:0-253 /dev/mapper/cx1430_lun34:254

FWIW - this affects device failure repair/conversion. Before the failure, this mirror was set up with each leg having a different device, after the failure and after reconfiguring the failed device, a convert was unable to place the leg on the proper device.

 [root@taft-01 ~]# lvs -a -o +devices
 [syncd_secondary_log_2devices_1_mimage_0] iwi-ao 600.00M  /dev/sde1(0)
 [syncd_secondary_log_2devices_1_mimage_1] iwi-ao 600.00M  /dev/sdd1(0)
 [syncd_secondary_log_2devices_1_mlog]     lwa-ao   4.00M  /dev/sdd1(150)

Test Output:

Enabling device sdd on taft-01
Enabling device sdd on taft-02
Enabling device sdd on taft-03
Enabling device sdd on taft-04

Recreating PVs /dev/sdd1
  WARNING: Volume group helter_skelter is not consistent
Extending the recreated PVs back into VG helter_skelter

Attempting to reconvert volume(s) back to their original mirrored state on taft-02...
(it's possible this may be a redundant operation in some allocate scenarios)
taft-02: lvconvert --alloc anywhere --mirrorlog disk -m 1 -b helter_skelter/syncd_secondary_log_2devices_1 /dev/sde1:0-1000 /dev/sdd1:0-1000
Verify that each of the mirror repairs finished successfully

Verifying the up conversions by checking if all original leg devices are back in the mirror(s)
Verifying device /dev/sde1 *IS* one of the legs in the mirror(s)
Verifying device /dev/sdd1 *IS* one of the legs in the mirror(s)
Device /dev/sdd1 should be in mirror on taft-02


[root@taft-01 ~]# lvs -a -o +devices
 [syncd_secondary_log_2devices_1_mimage_0] iwi-ao 600.00M  /dev/sde1(0)
 [syncd_secondary_log_2devices_1_mimage_1] iwi-ao 600.00M  /dev/sde1(150)
 [syncd_secondary_log_2devices_1_mlog]     lwa-ao   4.00M  /dev/sdd1(0)

Fixed in lvm2-2.02.88-1.el5.

Fix verified in the latest rpms.

2.6.18-274.el5
lvm2-2.02.88-4.el5    BUILT: Wed Nov 16 09:40:55 CST 2011
lvm2-cluster-2.02.88-4.el5    BUILT: Wed Nov 16 09:46:51 CST 2011
device-mapper-1.02.67-2.el5    BUILT: Mon Oct 17 08:31:56 CDT 2011
device-mapper-event-1.02.67-2.el5    BUILT: Mon Oct 17 08:31:56 CDT 2011
cmirror-1.1.39-10.el5    BUILT: Wed Sep  8 16:32:05 CDT 2010
kmod-cmirror-0.1.22-3.el5    BUILT: Tue Dec 22 13:39:47 CST 2009


SCENARIO - [alloc_anywhere]
Create a mirror using only 2 physical devices
lvcreate -m 1 -n alloc_anywhere --alloc anywhere -L 50M mirror_sanity /dev/sdc6:0-1500 /dev/sdb1:0-1500
image1 should be on /dev/sdc6
image2 should be on /dev/sdb1
log should be on either /dev/sdc6 or /dev/sdb1
Deactivating mirror alloc_anywhere... and removing

    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Generally, placing mirror legs on different physical devices improves data availability. The command <command>lvcreate --alloc anywhere</command> does not guarantee placement of data on different physical devices. Consequently, the use of this option is not recommended. If this option is used, the location of the data placement must be manually verified.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2012-0161.html