Bug 594525 - two device alloc anywhere leg placement has regressed
two device alloc anywhere leg placement has regressed
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: lvm2 (Show other bugs)
6.0
All Linux
high Severity high
: rc
: ---
Assigned To: Alasdair Kergon
Corey Marthaler
: Regression, TestBlocker
: 500530 606492 (view as bug list)
Depends On:
Blocks: 636991
  Show dependency treegraph
 
Reported: 2010-05-20 18:48 EDT by Corey Marthaler
Modified: 2011-12-06 11:52 EST (History)
12 users (show)

See Also:
Fixed In Version: lvm2-2.02.86-1.el6
Doc Type: Bug Fix
Doc Text:
Generally, placing mirror legs on different physical devices improves data availability. The command <command>lvcreate --alloc anywhere</command> does not guarantee placement of data on different physical devices. Consequently, the use of this option is not recommended. If this option is used, the location of the data placement must be manually verified.
Story Points: ---
Clone Of:
: 636991 (view as bug list)
Environment:
Last Closed: 2011-12-06 11:52:30 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Corey Marthaler 2010-05-20 18:48:47 EDT
Description of problem:
In RHEL5, if you only had 2 devices with which to create mirrors, the alloc anywhere logic new to place the two legs on the two devices, and then double up the log on one of the legs. That appears to have regressed.

SCENARIO - [alloc_anywhere]
Create a mirror using only 2 physical devices
lvcreate -m 1 -n alloc_anywhere --alloc anywhere -L 50M mirror_sanity /dev/sdd1:0-1500 /dev/sdf1:0-1500
image1 should be on /dev/sdd1
image2 should be on /dev/sdf1
mirror leg 2 is not on proper device: /dev/sdd1 should be /dev/sdf1

alloc_anywhere            mirror_sanity mwa-a- 52.00m   alloc_anywhere_mlog 100.00  alloc_anywhere_mimage_0(0),alloc_anywhere_mimage_1(0)
[alloc_anywhere_mimage_0] mirror_sanity iwi-ao 52.00m                               /dev/sdd1(0)
[alloc_anywhere_mimage_1] mirror_sanity iwi-ao 52.00m                               /dev/sdd1(13)
[alloc_anywhere_mlog]     mirror_sanity lwa-ao  4.00m                               /dev/sdf1(0)


Version-Release number of selected component (if applicable):
2.6.32-25.el6.x86_64

lvm2-2.02.65-1.el6    BUILT: Tue May 18 04:46:06 CDT 2010
lvm2-libs-2.02.65-1.el6    BUILT: Tue May 18 04:46:06 CDT 2010
lvm2-cluster-2.02.65-1.el6    BUILT: Tue May 18 04:46:06 CDT 2010
device-mapper-1.02.48-1.el6    BUILT: Tue May 18 04:46:06 CDT 2010
device-mapper-libs-1.02.48-1.el6    BUILT: Tue May 18 04:46:06 CDT 2010
device-mapper-event-1.02.48-1.el6    BUILT: Tue May 18 04:46:06 CDT 2010
device-mapper-event-libs-1.02.48-1.el6    BUILT: Tue May 18 04:46:06 CDT 2010
cmirror-2.02.65-1.el6    BUILT: Wed May 19 11:19:57 CDT 2010
Comment 1 Alasdair Kergon 2010-05-20 19:20:16 EDT
--alloc anywhere never guaranteed that.
It just usually happened to work.

As discussed elsewhere, a new alloc policy is required to guarantee the desired layout.
Comment 2 Corey Marthaler 2010-06-21 15:53:33 EDT
*** Bug 606492 has been marked as a duplicate of this bug. ***
Comment 3 Tom Coughlan 2010-07-28 15:51:08 EDT
So, it looks like we need a release note for 6.0, describing the current behavior, and this BZ should move to 6.1. Agreed?
Comment 6 Tom Coughlan 2010-07-29 12:09:28 EDT
Technical note added. If any revisions are required, please edit the "Technical Notes" field
accordingly. All revisions will be proofread by the Engineering Content Services team.

New Contents:
Proposed 6.0 release note:

As a general rule, you will want to place mirror legs on different physical devices, for improved data availability. The command "lvcreate --alloc anywhere" does not guarantee this placement of the data. Thus, the use of this command option is not recommended, or if it is used you are advised to check the resulting data placement. 

Alasdair, please review.
Comment 7 Corey Marthaler 2010-07-29 12:23:24 EDT
Whoa, hold on here, this bug is not about the command not guaranteeing the placement of the data when used with 2 devices. This bug is that it flat out no longer works! It used to in RHEL5, and no longer does now. 

If all we're going to do is add a Tech note, then it needs to say "We no longer support 2 device mirrors in RHEL6, there's no way to create them."
Comment 9 Jonathan Earl Brassow 2010-08-02 16:04:19 EDT
I think you could create a 2 device mirror by using a combination of 'lvcreate' and 'lvconvert'.

~> lvcreate -l 1 -n lv vg /dev/sdb1
~> lvconvert -m1 vg/lv /dev/sdc1

That's probably what the release note would say, right?
Comment 10 Corey Marthaler 2010-08-02 16:59:45 EDT
[root@taft-02 ~]# lvcreate -L 100M -n mirror taft /dev/sdb1
  Logical volume "mirror" created

[root@taft-02 ~]# lvconvert -m1 taft/mirror /dev/sdc1
  Not enough PVs with free space available for parallel allocation.
  Consider --alloc anywhere if desperate.
  Unable to allocate extents for mirror(s).

[root@taft-02 ~]# lvconvert -m1 --alloc anywhere taft/mirror /dev/sdc1
  taft/mirror: Converted: 4.0%
  taft/mirror: Converted: 100.0%

[root@taft-02 ~]# lvs -a -o +devices
  LV                VG        Attr   LSize   Log         Copy%  Devices
  mirror            taft      mwi-a- 100.00m mirror_mlog 100.00 mirror_mimage_0(0),mirror_mimage_1(0)
  [mirror_mimage_0] taft      iwi-ao 100.00m                    /dev/sdb1(0)
  [mirror_mimage_1] taft      iwi-ao 100.00m                    /dev/sdc1(0)
  [mirror_mlog]     taft      lwa-ao   4.00m                    /dev/sdc1(25)
Comment 15 Ryan Lerch 2010-10-18 00:00:11 EDT
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1,5 +1 @@
-Proposed 6.0 release note:
+Generally, placing mirror legs on different physical devices improves data availability. The command <command>lvcreate --alloc anywhere</command> does not guarantee placement of data on different physical devices. Consequently, the use of this option is not recommended. If this option is used, the location of the data placement must be manually verified.-
-As a general rule, you will want to place mirror legs on different physical devices, for improved data availability. The command "lvcreate --alloc anywhere" does not guarantee this placement of the data. Thus, the use of this command option is not recommended, or if it is used you are advised to check the resulting data placement. 
-
-Alasdair, please review.
Comment 16 Alasdair Kergon 2010-11-03 17:16:55 EDT
That's at least accurate.
Comment 17 Corey Marthaler 2010-11-03 17:30:44 EDT
Ryan, manually verifying the "lvcreate --alloc anywhere" command won't matter because it won't work, we know that. I thought the tech note would have something about the hack needed for it *too* work. The hack in comment #10.
Comment 18 Alasdair Kergon 2010-11-03 22:30:53 EDT
So, this is not just for mirrors, but any future segment type that depends on other invisible segments that get created internally (e.g. thin provisioning metadata).  Propose we add this new policy between the existing 'normal' and 'anywhere' policies and then rename normal to "separated" (or some better name if we can think of one) and make the new policy the default 'normal'.
Comment 19 Alasdair Kergon 2010-11-30 12:45:33 EST
*** Bug 500530 has been marked as a duplicate of this bug. ***
Comment 20 Alasdair Kergon 2011-02-26 20:49:00 EST
Finally, I checked in some code.  If it can't complete the log allocation, the code now attempts to place the logs on already-used devices.  There are still some hopefully-obscure cases it doesn't handle, but I hope what it does now is adequate. There's an lvm.conf setting to (largely) disable the new behaviour.

http://www.redhat.com/archives/lvm-devel/2011-February/msg00143.html
Comment 23 Corey Marthaler 2011-06-02 10:47:01 EDT
Adding QA ack for 6.2. 

Devel will need to provide unit testing results however before this bug can be ultimately verified by QA.
Comment 25 Corey Marthaler 2011-08-18 15:51:52 EDT
Fix verified in the latest rpms.

SCENARIO - [alloc_anywhere]
Create a mirror using only 2 physical devices
lvcreate -m 1 -n alloc_anywhere --alloc anywhere -L 50M mirror_sanity /dev/sdh1:0-1500 /dev/sdg1:0-1500
image1 should be on /dev/sdh1
image2 should be on /dev/sdg1
log should be on either /dev/sdh1 or /dev/sdg1
Deactivating mirror alloc_anywhere... and removing



2.6.32-188.el6.x86_64

lvm2-2.02.87-1.el6    BUILT: Fri Aug 12 06:11:57 CDT 2011
lvm2-libs-2.02.87-1.el6    BUILT: Fri Aug 12 06:11:57 CDT 2011
lvm2-cluster-2.02.87-1.el6    BUILT: Fri Aug 12 06:11:57 CDT 2011
udev-147-2.37.el6    BUILT: Wed Aug 10 07:48:15 CDT 2011
device-mapper-1.02.66-1.el6    BUILT: Fri Aug 12 06:11:57 CDT 2011
device-mapper-libs-1.02.66-1.el6    BUILT: Fri Aug 12 06:11:57 CDT 2011
device-mapper-event-1.02.66-1.el6    BUILT: Fri Aug 12 06:11:57 CDT 2011
device-mapper-event-libs-1.02.66-1.el6    BUILT: Fri Aug 12 06:11:57 CDT 2011
cmirror-2.02.87-1.el6    BUILT: Fri Aug 12 06:11:57 CDT 2011
Comment 26 errata-xmlrpc 2011-12-06 11:52:30 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1522.html

Note You need to log in before you can comment on or make changes to this bug.