Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
For bugs related to Red Hat Enterprise Linux 5 product line. The current stable release is 5.10. For Red Hat Enterprise Linux 6 and above, please visit Red Hat JIRA https://issues.redhat.com/secure/CreateIssue!default.jspa?pid=12332745 to report new issues.

Bug 643365

Summary: online resize of LV/ext4 corrupts data
Product: Red Hat Enterprise Linux 5 Reporter: Zdenek Kabelac <zkabelac>
Component: lvm2Assignee: Zdenek Kabelac <zkabelac>
Status: CLOSED ERRATA QA Contact: Corey Marthaler <cmarthal>
Severity: high Docs Contact:
Priority: urgent    
Version: 5.6CC: agk, antillon.maurizio, dwysocha, heinzm, jbrassow, joe.thornber, mbroz, prajnoha, prockai
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 638052 Environment:
Last Closed: 2011-01-13 22:42:24 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Zdenek Kabelac 2010-10-15 12:09:04 UTC 
+++ This bug was initially created as a clone of Bug #638052 +++

This bugzilla is clone for RHEL 5.6

Description of problem:
Growing a 100% full ext4 by issuing the following ended up losing data in it (corruption):
  [oracle@ora14 ~]$ df -h
  Filesystem            Size  Used Avail Use% Mounted on
  ...
  /dev/mapper/vg1-oce    40G  4.1G   34G  11% /ora/db/oce
  /dev/mapper/vg2-work   40G   38G     0 100% /ora/db/oce/work
  [root@ora14 ~]# pvscan
    PV /dev/cciss/c0d1p2   VG vg2   lvm2 [86.69 GiB / 46.69 GiB free]
  [root@ora14 ~]# lvscan
    ACTIVE            '/dev/vg2/work' [40.00 GiB] inherit
  [root@ora14 ~]# lvextend -L +20G  /dev/vg2/work
    Extending logical volume work to 60.00 GiB
    Logical volume work successfully resized
  [root@ora14 ~]# lvextend -L +10G -r  /dev/vg2/work
  fsck from util-linux-ng 2.17.2
  e2fsck 1.41.12 (17-May-2010)
  /dev/mapper/vg2-work is mounted.  

  WARNING!!!  The filesystem is mounted.   If you continue you ***WILL*** cause ***SEVERE*** filesystem damage.

  Do you really want to continue (y/n)? yes

  /dev/mapper/vg2-work: recovering journal
  fsck.ext4: Bad magic number in super-block while trying to re-open /dev/mapper/vg2-work
  e2fsck: io manager magic bad!
    fsadm failed: 8
  [root@ora14 ~]# df -h
  Filesystem            Size  Used Avail Use% Mounted on
  /dev/mapper/vg2-work   64Z   64Z  2.0G 100% /ora/db/oce/work
  [root@ora14 ~]# ls /ora/db/oce/work/
  [root@ora14 ~]#
  [root@ora14 ~]# lvscan
  ACTIVE            '/dev/vg2/work' [60.00 GiB] inherit
  [root@ora14 ~]# uname -a
  Linux ora14.test.redhat.com 2.6.32-71.el6.i686 #1 SMP Wed Sep 1 01:26:34 EDT 2010 i686 i686 i386 GNU/Linux

Version-Release number of selected component (if applicable):
lvm2-2.02.73-2.el5

How reproducible:
It happened once for me

Steps to Reproduce:
1. Create an ext4 filesystem on top of lvm2 logical volume
2. Fill up the volume to 100%
3. Try to grow the volume & filesystem using something like "lvextend -L +10G -r  /dev/vg2/work" as shown below.
4. See above for detail
  
Actual results:
Data corruption

Expected results:
To be able to grow volume without data lose or the tool shouldn't allow one to perform such a task if it causes a corruption.

Additional info:
Comment 2 Milan Broz 2010-10-15 15:10:26 UTC 
Fix in lvm2-2.02.74-1.el5.
Comment 4 Zdenek Kabelac 2010-11-01 14:38:14 UTC 
New upstream patches available:

http://www.redhat.com/archives/lvm-devel/2010-November/msg00004.html
http://www.redhat.com/archives/lvm-devel/2010-November/msg00002.html
Comment 5 Corey Marthaler 2010-11-01 23:08:16 UTC 
This doesn't appear fixed in the latest rpms if an fsck is still being attempted. Also, ext4 isn't available in 5.6 correct? I used ext3 in this example.


[root@grant-01 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/mirror_sanity-open_fsadm_resize
                      4.0G  137M  3.7G   4% /mnt/open_fsadm_resize

[root@grant-01 ~]# lvextend -L +10G -r /dev/mirror_sanity/open_fsadm_resize
  Extending 2 mirror images.
fsck 1.39 (29-May-2006)
/dev/mapper/mirror_sanity-open_fsadm_resize is mounted.

WARNING!!!  Running e2fsck on a mounted filesystem may cause
SEVERE filesystem damage.

Do you really want to continue (y/n)? yes

/dev/mapper/mirror_sanity-open_fsadm_resize: recovering journal
/dev/mapper/mirror_sanity-open_fsadm_resize: clean, 11/524288 files, 51312/1048576 blocks
  Extending logical volume open_fsadm_resize to 14.00 GB

  Logical volume open_fsadm_resize successfully resized
resize2fs 1.39 (29-May-2006)
Filesystem at /dev/mapper/mirror_sanity-open_fsadm_resize is mounted on /mnt/open_fsadm_resize; on-line resizing required
Performing an on-line resize of /dev/mapper/mirror_sanity-open_fsadm_resize to 3670016 (4k) blocks.
The filesystem on /dev/mapper/mirror_sanity-open_fsadm_resize is now 3670016 blocks long.

[root@grant-01 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                       63G  1.6G   58G   3% /
/dev/sda1              99M   17M   78M  18% /boot
tmpfs                 4.0G     0  4.0G   0% /dev/shm
/dev/mapper/mirror_sanity-open_fsadm_resize
                       14G  139M   13G   2% /mnt/open_fsadm_resize
Comment 6 Corey Marthaler 2010-11-02 17:23:58 UTC 
How was this bug moved from ASSIGNED without a comment about what release it's now fixed in, or a respin of the errata packages?
Comment 7 Milan Broz 2010-11-02 17:32:57 UTC 
(In reply to comment #6)
> How was this bug moved from ASSIGNED without a comment about what release it's
> now fixed in, or a respin of the errata packages?

Because it is not yet fixed neither in errata nor 5.6 build (I mean the additional fixes, not the first version of patch).
It is in POST what means it is waiting for build but upstream has fix already.
Comment 8 Milan Broz 2010-11-09 13:57:44 UTC 
Following fix in lvm2-2.02.74-2.el5.
Comment 10 Corey Marthaler 2010-11-09 22:06:44 UTC 
Still doesn't look like anything has changed here. Putting back into ASSIGNED yet again.

[root@taft-01 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/mirror_sanity-open_fsadm_resize
                      4.0G  137M  3.7G   4% /mnt/open_fsadm_resize

[root@taft-01 ~]# lvextend -L +10G -r /dev/mirror_sanity/open_fsadm_resize
  Extending 2 mirror images.
fsck 1.39 (29-May-2006)
/dev/mapper/mirror_sanity-open_fsadm_resize is mounted.

WARNING!!!  Running e2fsck on a mounted filesystem may cause
SEVERE filesystem damage.

Do you really want to continue (y/n)? yes

/dev/mapper/mirror_sanity-open_fsadm_resize: recovering journal
/dev/mapper/mirror_sanity-open_fsadm_resize: clean, 11/524288 files, 51312/1048576 blocks

  Extending logical volume open_fsadm_resize to 14.00 GB
  Logical volume open_fsadm_resize successfully resized
resize2fs 1.39 (29-May-2006)
Filesystem at /dev/mapper/mirror_sanity-open_fsadm_resize is mounted on /mnt/open_fsadm_resize; on-line resizing required
Performing an on-line resize of /dev/mapper/mirror_sanity-open_fsadm_resize to 3670016 (4k) blocks.
The filesystem on /dev/mapper/mirror_sanity-open_fsadm_resize is now 3670016 blocks long.



lvm2-2.02.74-2.el5    BUILT: Tue Nov  9 08:03:06 CST 2010
lvm2-cluster-2.02.74-3.el5    BUILT: Tue Nov  9 08:01:59 CST 2010
device-mapper-1.02.55-2.el5    BUILT: Tue Nov  9 06:41:00 CST 2010
cmirror-1.1.39-10.el5    BUILT: Wed Sep  8 16:32:05 CDT 2010
kmod-cmirror-0.1.22-3.el5    BUILT: Tue Dec 22 13:39:47 CST 2009
Comment 11 Alasdair Kergon 2010-11-10 00:24:50 UTC 
Sigh.  Yes, that is still wrong.

It should not have prompted with that question and it should not have run the fsck.
Comment 12 Zdenek Kabelac 2010-11-10 10:08:57 UTC 
Looks like regression in fsadm detection of mounted filesystem - where the combination with util-linux gives yet another variant of how to store dm name in /proc/mounts.

Fixed by this patch:

https://www.redhat.com/archives/lvm-devel/2010-November/msg00034.html
Comment 17 Milan Broz 2010-11-11 09:04:34 UTC 
Fix in lvm2-2.02.74-3.el5.
Comment 22 Corey Marthaler 2010-11-11 17:49:12 UTC 
Fix verified in the latest rpm (lvm2-2.02.74-3.el5).

[root@taft-01 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/mirror_sanity-open_fsadm_resize
                      4.0G  137M  3.7G   4% /mnt/open_fsadm_resize

[root@taft-01 ~]# lvextend -L +10G -r /dev/mirror_sanity/open_fsadm_resize
  Extending 2 mirror images.
  Extending logical volume open_fsadm_resize to 14.00 GB
  Logical volume open_fsadm_resize successfully resized
resize2fs 1.39 (29-May-2006)
Filesystem at /dev/mapper/mirror_sanity-open_fsadm_resize is mounted on /mnt/open_fsadm_resize; on-line resizing required
Performing an on-line resize of /dev/mapper/mirror_sanity-open_fsadm_resize to 3670016 (4k) blocks.
The filesystem on /dev/mapper/mirror_sanity-open_fsadm_resize is now 3670016 blocks long.

[root@taft-01 ~]# echo $?
0
Comment 26 errata-xmlrpc 2011-01-13 22:42:24 UTC 
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2011-0052.html