Bug 625135

Summary: lvm mirror: mirror log disappears with policies (leg=allocate, log=remove) after error recovery
Product: Red Hat Enterprise Linux 6 Reporter: Takahiro Yasui <tyasui>
Component: lvm2Assignee: Jonathan Earl Brassow <jbrassow>
Status: CLOSED ERRATA QA Contact: Corey Marthaler <cmarthal>
Severity: medium Docs Contact:
Priority: low    
Version: 6.0CC: agk, coughlan, dwysocha, heinzm, jbrassow, lwang, masaki.kimura.kz, mbroz, noboru.obata.ar, prajnoha, prockai, saguchi, takahiro.yasui.mp
Target Milestone: rcKeywords: RHELNAK
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: lvm2-2.02.82-1.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-05-19 14:26:28 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Takahiro Yasui 2010-08-18 16:57:16 UTC
Description of problem:

Mirror log disappears with policies (leg=allocate, log=remove) after error recovery.

Version-Release number of selected component (if applicable):
  lvm2-2.02.72-7.el6

How reproducible:
  Always

Steps to Reproduce:
1. Set the policy in lvm.conf
  mirror_log_fault_policy = "remove"
  mirror_image_fault_policy = "allocate"

2. Create a mirror volume with 2 legs and 1 leg

  # vgcreate vg00 /dev/sd[b-e]
    Volume group "vg00" successfully created
  # lvcreate -m1 -L12m -nlv00 vg00
    Logical volume "lv00" created
  # lvs
    LV   VG   Attr   LSize  Origin Snap%  Move Log       Copy%  Convert
    lv00 vg00 mwi-a- 12.00m                    lv00_mlog 100.00

2. Trigger error recovery by disabling one of mirror legs

  # echo offline > /sys/block/sdc/device/state
  # dd if=/dev/zero of=/dev/vg00/lv00 bs=4096 count=1
  1+0 records in
  1+0 records out
  4096 bytes (4.1 kB) copied, 0.306967 s, 13.3 kB/s

3. Check structure of the lvm mirror

  # lvs
    /dev/sdc: open failed: No such device or address
    LV   VG   Attr   LSize  Origin Snap%  Move Log Copy%  Convert
    lv00 vg00 mwi-a- 12.00m                        100.00

  *** log disappeared!! ***
  
Actual results:
  lvm mirror removes a mirror log.

Expected results:
  lvm mirror keeps a mirror log.

Additional info:
  Already fixed in upstream.
  - post
    [PATCH] Fix log disk disappear with mirror allocate policy
    https://www.redhat.com/archives/lvm-devel/2010-July/msg00206.html

  - commit:
    LVM2 ./WHATS_NEW tools/lvconvert.c
    https://www.redhat.com/archives/lvm-devel/2010-July/msg00215.html

Comment 2 RHEL Program Management 2010-08-18 17:18:07 UTC
This issue has been proposed when we are only considering blocker
issues in the current Red Hat Enterprise Linux release.

** If you would still like this issue considered for the current
release, ask your support representative to file as a blocker on
your behalf. Otherwise ask that it be considered for the next
Red Hat Enterprise Linux release. **

Comment 3 RHEL Program Management 2010-08-18 21:22:38 UTC
Thank you for your bug report. This issue was evaluated for inclusion
in the current release of Red Hat Enterprise Linux. Unfortunately, we
are unable to address this request in the current release. Because we
are in the final stage of Red Hat Enterprise Linux 6 development, only
significant, release-blocking issues involving serious regressions and
data corruption can be considered.

If you believe this issue meets the release blocking criteria as
defined and communicated to you by your Red Hat Support representative,
please ask your representative to file this issue as a blocker for the
current release. Otherwise, ask that it be evaluated for inclusion in
the next minor release of Red Hat Enterprise Linux.

Comment 4 Jonathan Earl Brassow 2011-01-19 18:52:20 UTC
Seems to be fixed upstream:

[root@bp-04 test]# devices
  LV            Copy%  Devices
  lv            100.00 lv_mimage_0(0),lv_mimage_1(0)
  [lv_mimage_0]        /dev/sdb1(0)
  [lv_mimage_1]        /dev/sdc1(0)
  [lv_mlog]            /dev/sdi1(0)
[root@bp-04 test]# off.sh sdc
Turning off sdc
[root@bp-04 test]# lvconvert --repair vg/lv --use-policies --config 'activation { mirror_image_fault_policy = "replace" mirror_log_fault_policy = "remove" }'
  /dev/sdc1: read failed after 0 of 2048 at 250994294784: Input/output error
  /dev/sdc1: read failed after 0 of 2048 at 250994376704: Input/output error
  /dev/sdc1: read failed after 0 of 2048 at 0: Input/output error
  /dev/sdc1: read failed after 0 of 2048 at 4096: Input/output error
  Couldn't find device with uuid 812uYW-ypVE-iApg-Vpuz-3VFq-fc6w-WJK9HZ.
  Mirror status: 1 of 2 images failed.
  Trying to up-convert to 2 images, 1 logs.
  1 missing and now unallocated Physical Volumes removed from VG.
  vg/lv: Converted: 0.0%
  vg/lv: Converted: 100.0%
[root@bp-04 test]# devices
  /dev/sdc1: open failed: No such device or address
  LV            Copy%  Devices
  lv            100.00 lv_mimage_0(0),lv_mimage_1(0)
  [lv_mimage_0]        /dev/sdb1(0)
  [lv_mimage_1]        /dev/sdd1(0)
  [lv_mlog]            /dev/sdi1(0)

Upstream version tested: 2.02.82

Comment 7 Corey Marthaler 2011-03-16 22:43:36 UTC
Verified that the log device is no longer missing after a failure, however, the log *is* still removed and then replaced with a different device. I'll open an RFE for lvm to attempt to use the same log device that was originally used/picked at creation.

# Fail the SECONDARY device

# before the failure
[root@taft-01 ~]# lvs -a -o +devices
  LV                                 Attr   Log                          Copy%  Devices
  syncd_secondary_2legs_1            mwi-ao syncd_secondary_2legs_1_mlog 100.00 syncd_secondary_2legs_1_mimage_0(0),syncd_secondary_2legs_1_mimage_1(0)
  [syncd_secondary_2legs_1_mimage_0] iwi-ao                                     /dev/sdc1(0)
  [syncd_secondary_2legs_1_mimage_1] iwi-ao                                     /dev/sdf1(0)
  [syncd_secondary_2legs_1_mlog]     lwi-ao                                     /dev/sdg1(0)

# after the failure
[root@taft-01 ~]# lvs -a -o +devices
  LV                                 Attr   Log                          Copy%  Devices
  syncd_secondary_2legs_1            mwi-ao syncd_secondary_2legs_1_mlog 100.00 syncd_secondary_2legs_1_mimage_0(0),syncd_secondary_2legs_1_mimage_1(0)
  [syncd_secondary_2legs_1_mimage_0] iwi-ao                                     /dev/sdc1(0)
  [syncd_secondary_2legs_1_mimage_1] iwi-ao                                     /dev/sdh1(0)
  [syncd_secondary_2legs_1_mlog]     lwi-ao                                     /dev/sdb1(0)


# Fail the PRIMARY device
 
# before the failure
[root@taft-01 ~]# lvs -a -o +devices
  LV                               Attr   Log                        Copy%  Devices
  syncd_primary_2legs_1            mwi-a- syncd_primary_2legs_1_mlog 74.67  syncd_primary_2legs_1_mimage_0(0),syncd_primary_2legs_1_mimage_1(0)
  [syncd_primary_2legs_1_mimage_0] Iwi-ao                                   /dev/sdf1(0)
  [syncd_primary_2legs_1_mimage_1] Iwi-ao                                   /dev/sdh1(0)
  [syncd_primary_2legs_1_mlog]     lwi-ao                                   /dev/sde1(0)

# after the failure
[root@taft-01 ~]# lvs -a -o +devices
  LV                               Attr   Log                        Copy%  Devices
  syncd_primary_2legs_1            mwi-ao syncd_primary_2legs_1_mlog  88.67 syncd_primary_2legs_1_mimage_0(0),syncd_primary_2legs_1_mimage_1(0)
  [syncd_primary_2legs_1_mimage_0] Iwi-ao                                   /dev/sdh1(0)
  [syncd_primary_2legs_1_mimage_1] Iwi-ao                                   /dev/sdg1(0)
  [syncd_primary_2legs_1_mlog]     lwi-ao                                   /dev/sdb1(0)


2.6.32-94.el6.x86_64

lvm2-2.02.83-2.el6    BUILT: Tue Feb  8 10:10:57 CST 2011
lvm2-libs-2.02.83-2.el6    BUILT: Tue Feb  8 10:10:57 CST 2011
lvm2-cluster-2.02.83-2.el6    BUILT: Tue Feb  8 10:10:57 CST 2011
udev-147-2.31.el6    BUILT: Wed Jan 26 05:39:15 CST 2011
device-mapper-1.02.62-2.el6    BUILT: Tue Feb  8 10:10:57 CST 2011
device-mapper-libs-1.02.62-2.el6    BUILT: Tue Feb  8 10:10:57 CST 2011
device-mapper-event-1.02.62-2.el6    BUILT: Tue Feb  8 10:10:57 CST 2011
device-mapper-event-libs-1.02.62-2.el6    BUILT: Tue Feb  8 10:10:57 CST 2011
cmirror-2.02.83-2.el6    BUILT: Tue Feb  8 10:10:57 CST 2011

Comment 8 errata-xmlrpc 2011-05-19 14:26:28 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2011-0772.html