Bug 200176 - mirror volumes appear to be corrupted after leg failure for a period of time
mirror volumes appear to be corrupted after leg failure for a period of time
Status: CLOSED DUPLICATE of bug 199724
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: lvm2 (Show other bugs)
4.0
All Linux
high Severity medium
: ---
: ---
Assigned To: Jonathan Earl Brassow
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2006-07-25 17:47 EDT by Corey Marthaler
Modified: 2007-11-30 17:07 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2006-12-05 13:05:28 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Corey Marthaler 2006-07-25 17:47:53 EDT
Description of problem:
After failing a mirror leg, the mirrored volume appears corrupted for about 5
minutes after one of it's legs are failed.

Right after the failure:
[root@taft-04 ~]# lvscan
  /dev/sdd1: read failed after 0 of 2048 at 0: Input/output error
  /dev/sdd2: read failed after 0 of 2048 at 0: Input/output error
  /dev/dm-3: read failed after 0 of 4096 at 1073676288: Input/output error
  /dev/dm-3: read failed after 0 of 4096 at 0: Input/output error
  /dev/sdd: read failed after 0 of 4096 at 0: Input/output error
  /dev/sdd: read failed after 0 of 4096 at 1999073378304: Input/output error
  /dev/sdd: read failed after 0 of 4096 at 0: Input/output error
  /dev/sdd1: read failed after 0 of 2048 at 1011548160: Input/output error
  /dev/sdd1: read failed after 0 of 2048 at 0: Input/output error
  /dev/sdd2: read failed after 0 of 512 at 1998060257280: Input/output error
  /dev/sdd2: read failed after 0 of 2048 at 0: Input/output error
  Couldn't find device with uuid 'Pd41er-YXpT-tP4G-dXWo-LaUx-SKXH-ZptJIm'.
  Couldn't find all physical volumes for volume group vg.
  /dev/dm-3: read failed after 0 of 4096 at 0: Input/output error
  /dev/sdd: read failed after 0 of 4096 at 0: Input/output error
  /dev/sdd1: read failed after 0 of 2048 at 0: Input/output error
  /dev/sdd2: read failed after 0 of 2048 at 0: Input/output error
  Couldn't find device with uuid 'Pd41er-YXpT-tP4G-dXWo-LaUx-SKXH-ZptJIm'.
  Couldn't find all physical volumes for volume group vg.
  /dev/dm-3: read failed after 0 of 4096 at 0: Input/output error
  /dev/sdd: read failed after 0 of 4096 at 0: Input/output error
  /dev/sdd1: read failed after 0 of 2048 at 0: Input/output error
  /dev/sdd2: read failed after 0 of 2048 at 0: Input/output error
  /dev/dm-3: read failed after 0 of 4096 at 0: Input/output error
  /dev/sdd: read failed after 0 of 4096 at 0: Input/output error
  /dev/sdd1: read failed after 0 of 2048 at 0: Input/output error
  /dev/sdd2: read failed after 0 of 2048 at 0: Input/output error
  Couldn't find device with uuid 'Pd41er-YXpT-tP4G-dXWo-LaUx-SKXH-ZptJIm'.
  Couldn't find all physical volumes for volume group vg.
  /dev/dm-3: read failed after 0 of 4096 at 0: Input/output error
  /dev/sdd: read failed after 0 of 4096 at 0: Input/output error
  /dev/sdd1: read failed after 0 of 2048 at 0: Input/output error
  /dev/sdd2: read failed after 0 of 2048 at 0: Input/output error
  Couldn't find device with uuid 'Pd41er-YXpT-tP4G-dXWo-LaUx-SKXH-ZptJIm'.
  Couldn't find all physical volumes for volume group vg.
  Volume group "vg" not found
  ACTIVE            '/dev/VolGroup00/LogVol00' [19.53 GB] inherit
  ACTIVE            '/dev/VolGroup00/LogVol01' [1.94 GB] inherit


Five or so minutes after the failure:
[root@taft-04 ~]# lvscan
  /dev/sdd: read failed after 0 of 4096 at 0: Input/output error
  /dev/sdd1: read failed after 0 of 2048 at 0: Input/output error
  /dev/sdd2: read failed after 0 of 2048 at 0: Input/output error
  ACTIVE            '/dev/vg/mirror' [1.00 GB] inherit
  ACTIVE            '/dev/VolGroup00/LogVol00' [19.53 GB] inherit
  ACTIVE            '/dev/VolGroup00/LogVol01' [1.94 GB] inherit


[root@taft-04 ~]# dmsetup ls
vg-mirror       (253, 5)
VolGroup00-LogVol01     (253, 1)
VolGroup00-LogVol00     (253, 0)


Version-Release number of selected component (if applicable):
[root@taft-04 ~]# rpm -q lvm2
lvm2-2.02.06-6.0.RHEL4
[root@taft-04 ~]# rpm -q device-mapper
device-mapper-1.02.07-4.0.RHEL4


How reproducible:
every so often
Comment 1 Corey Marthaler 2006-07-25 18:03:59 EDT
During another attempt of this same test case, lvm was deadlocked during whole
"five minute time period" in an flock. After that stuck period, the volumes
appeared fine. 

[root@taft-04 lvm]# lvscan
  /dev/sdd1: read failed after 0 of 2048 at 0: Input/output error
  /dev/sdd2: read failed after 0 of 2048 at 0: Input/output error
  ACTIVE            '/dev/vg/mirror' [1.00 GB] inherit
  ACTIVE            '/dev/VolGroup00/LogVol00' [19.53 GB] inherit
  ACTIVE            '/dev/VolGroup00/LogVol01' [1.94 GB] inherit
Comment 2 Corey Marthaler 2006-08-16 11:07:33 EDT
I believe that we already know this fact, but after the leg failure, the lvm2
mirror will stay "corrupted" until a write to it is attempted. I reproduced this
by not having I/O going to the mirror and failing the leg. The volume was
corrupt for 12 hours or so over night until I attempted to write to it in the
morning.
Comment 3 Jonathan Earl Brassow 2006-12-05 13:05:28 EST
Unless you can show me otherwise, I don't think the volume is corrupted.  It is
simply incomplete.  IOW, dmeventd has not yet reconfigured the mirror yet.

Read failures will not trigger a reconfiguration (because reads can go to other
devices and it is possible for the drive to remap the block - S.M.A.R.T.).

Yes, while the reconfiguration is happening, LVM commands will be stuck.

This is a duplicate of bug 199724, AFAICT

*** This bug has been marked as a duplicate of 199724 ***

Note You need to log in before you can comment on or make changes to this bug.