Bug 1056452

Summary: lvm2 should clean any stale signatures and metadata found on newly created LV and PV
Product: Red Hat Enterprise Linux 7 Reporter: Peter Rajnoha <prajnoha>
Component: lvm2Assignee: LVM and device-mapper development team <lvm-team>
lvm2 sub component: Default / Unclassified QA Contact: cluster-qe <cluster-qe>
Status: CLOSED CURRENTRELEASE Docs Contact:
Severity: medium    
Priority: medium CC: agk, dwysocha, heinzm, jbrassow, mpatocka, msnitzer, nperic, prajnoha, prockai, thornber, todoleza, zkabelac
Version: 7.0Keywords: Triaged
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: lvm2-2.02.105-1.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 997223 Environment:
Last Closed: 2014-06-13 09:50:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Peter Rajnoha 2014-01-22 09:09:48 UTC
Let's include the blkid wiping in RHEL7 as it improves the way we initialize the newly created LV - any stale signatures on the newly created LV are wiped (the functionality is the same as what wipefs provides - we reuse the libblkid in lvm2 to achieve this). In addition, lvm2 tries to do that in a race-free way and directs udev with proper flags to not try scanning the new LV before the any stale signatures are completely wiped.

Otherwise, we could get into a situation as described below where the user is not aware of the fact that the stale signatures/metadata caused some other subsystem to be activated as soon as the new LV is created...

+++ This bug was initially created as a clone of Bug #997223 +++

Description of problem:
lvm2 doesn't clean md superblocks when creating a new logical volume.

Version-Release number of selected component (if applicable):
LVM 2.02.98(2)-RHEL6

How reproducible:
always

Steps to Reproduce:
1. download files from http://people.redhat.com/mpatocka/testcases/insufficient-cleaning/
2. change the "vg" variable in "insufficient-cleaning-bug" to some your real vg
3. run the script "bash ./insufficient-cleaning-bug"

Actual results:
The script created a logical volume /dev/vg1/clean-test . This logical volume is unusable. It is open and thus we can't create a filesystem there and mount it. It can't be deleted. There is no indication of what keeps the logical volume open.

Expected results:
The logical volume create with lvcreate should not be open and it should be useable.

Additional info:
The bug is caused by the fact that lvm didn't clear the logical volume. lvm clears only the first 8 sectors, however there is md superblock at sector 8 left from previous use. Some script in RHEL6 detects that the newly created logical volume has a md superblock and assigns it to md subsystem (you can see it if you do cat /proc/mdstats).

However, there is no indication to the sysadmin that the newly created logical volume is assigned to md. To the sysadmin, the bug looks quite serious - the newly create volume is unusable and undeletable and it couldn't be easily find what keeps it open.

LVM should use something like wipefs on new logical volumes to prevent this case.

--- Additional comment from Peter Rajnoha on 2013-08-15 14:16:38 CEST ---

Just for the record: even if wiping, there's still a possibility of a race with udev - bug #796200. So yes, we need to handle this somehow better...

--- Additional comment from RHEL Product and Program Management on 2013-08-19 14:41:55 CEST ---

Since this bug report was entered in bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.

--- Additional comment from Zdenek Kabelac on 2013-08-19 16:29:53 CEST ---

I guess every new created LV must be first activated with those udev invisible flags and just after clearing the flags could be changed to visible if needed.
(DM_UDEV_DISABLE_DISK/OTHER_RULES_FLAG)

Question is if there is chance to make this without  deactivate()/activate().

Other way around is to implement direct PV clearing - which seems to be the best - but hard...

--- Additional comment from Peter Rajnoha on 2013-11-27 16:03:37 CET ---

I've committed the patchset that reuses existing signature wiping known from pvcreate and uses it for lvcreate as well. Also, I've added an optional/configurable blkid wiping (that has richer support for more signatures). I'll post some examples/tests soon...

The patchset:

https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=03c941a4caf411f21045670e205ccdd975be27c7

https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=5b7e543cae5be52e1dcd79c7f6876acc89138cf1

https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=169b4c15864330e3fe47efee7b0def06581b8f08

https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=b6dab4e0598df7b6a44a32749fdb846c03aa692d

https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=9bfc0be493192958f5dbffea4eb7dda968062261

https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=ab2f858af7a72448b6c015a5984c73f5cdaf3664

https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=eaa23d32732c9bc3dd4f948781b5764cf21d84ba

https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=5968f07fd5ccb2b7d4e407cd8fec86b0b1f80f3a

Comment 2 Peter Rajnoha 2014-01-22 09:29:47 UTC
To QA:

To test this, create an LV and create some foreign signature on the LV - whatever you find in "blkid -k". E.g. the MD (the original bug report):

[0] raw/~ # pvcreate /dev/sda
  Physical volume "/dev/sda" successfully created

[0] raw/~ # vgcreate vg /dev/sda
  Volume group "vg" successfully created

[0] raw/~ # lvcreate -L32m vg
  Logical volume "lvol0" created

[0] raw/~ # lvcreate -L32m vg
  Logical volume "lvol1" created

[0] raw/~ # mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/vg/lvol0 /dev/vg/lvol1
mdadm: Note: this array has metadata at the start and
    may not be suitable as a boot device.  If you plan to
    store '/boot' on this device please ensure that
    your boot-loader understands md/v1.x metadata, or use
    --metadata=0.90
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.

[0] raw/~ # lsblk
NAME            MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sda               8:0    0  128M  0 disk  
|-vg-lvol0      253:2    0   32M  0 lvm   
| `-md0           9:0    0   32M  0 raid1 
`-vg-lvol1      253:3    0   32M  0 lvm   
  `-md0           9:0    0   32M  0 raid1 

[0] raw/~ # mdadm -S /dev/md0
mdadm: stopped /dev/md0

[0] raw/~ # lvremove -ff vg
  Logical volume "lvol0" successfully removed
  Logical volume "lvol1" successfully removed

[0] raw/~ # lvcreate -L32m vg
WARNING: linux_raid_member signature detected on /dev/vg/lvol0 at offset 4096. Wipe it? [y/n] y
  Wiping linux_raid_member signature on /dev/vg/lvol0.
  Logical volume "lvol0" created

[0] raw/~ # lvcreate -L32m vg
WARNING: linux_raid_member signature detected on /dev/vg/lvol1 at offset 4096. Wipe it? [y/n] y
  Wiping linux_raid_member signature on /dev/vg/lvol1.
  Logical volume "lvol1" created

[0] raw/~ # lsblk
NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda               8:0    0  128M  0 disk 
|-vg-lvol0      253:2    0   32M  0 lvm  
`-vg-lvol1      253:3    0   32M  0 lvm  


^^^ the important thing here is that the "linux_raid_member" signature got removed and also the MD array has not been autostarted from udev (as MD uses autoactivation with the help of udev).


The same would apply for FS signatures, e.g.:

[0] raw/~ # pvcreate /dev/sda
  Physical volume "/dev/sda" successfully created

[0] raw/~ # vgcreate vg /dev/sda
  Volume group "vg" successfully created

[0] raw/~ # lvcreate -L32m vg
  Logical volume "lvol0" created

[0] raw/~ # mkfs /dev/vg/lvol0
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
Stride=0 blocks, Stripe width=0 blocks
8192 inodes, 32768 blocks
1638 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=33554432
4 block groups
8192 blocks per group, 8192 fragments per group
2048 inodes per group
Superblock backups stored on blocks: 
	8193, 24577

Allocating group tables: done                            
Writing inode tables: done                            
Writing superblocks and filesystem accounting information: done

[0] raw/~ # lvremove -ff vg
  Logical volume "lvol0" successfully removed

[0] raw/~ # lvcreate -L32m vg
WARNING: ext2 signature detected on /dev/vg/lvol0 at offset 1080. Wipe it? [y/n] y
  Wiping ext2 signature on /dev/vg/lvol0.
  Logical volume "lvol0" created


...and similarly for any other signature that is recognized by libblkid (see the "blkid -k" output for the list of recognized signatures).


===

The same applies for pvcreate:

[0] raw/~ # mkfs /dev/sda
mke2fs 1.42.9 (28-Dec-2013)
/dev/sda is entire device, not just one partition!
Proceed anyway? (y,n) y
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
Stride=0 blocks, Stripe width=0 blocks
32768 inodes, 131072 blocks
6553 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67371008
16 block groups
8192 blocks per group, 8192 fragments per group
2048 inodes per group
Superblock backups stored on blocks: 
	8193, 24577, 40961, 57345, 73729

Allocating group tables: done                            
Writing inode tables: done                            
Writing superblocks and filesystem accounting information: done 

[0] raw/~ # pvcreate /dev/sda
WARNING: ext2 signature detected on /dev/sda at offset 1080. Wipe it? [y/n] y
  Wiping ext2 signature on /dev/sda.
  Physical volume "/dev/sda" successfully created

Comment 4 Nenad Peric 2014-04-01 12:22:13 UTC
Current behavior VERIFIED with

lvm2-2.02.105-14.el7    BUILT: Wed Mar 26 14:29:41 CET 2014
lvm2-libs-2.02.105-14.el7    BUILT: Wed Mar 26 14:29:41 CET 2014
lvm2-cluster-2.02.105-14.el7    BUILT: Wed Mar 26 14:29:41 CET 2014
device-mapper-1.02.84-14.el7    BUILT: Wed Mar 26 14:29:41 CET 2014
device-mapper-libs-1.02.84-14.el7    BUILT: Wed Mar 26 14:29:41 CET 2014
device-mapper-event-1.02.84-14.el7    BUILT: Wed Mar 26 14:29:41 CET 2014
device-mapper-event-libs-1.02.84-14.el7    BUILT: Wed Mar 26 14:29:41 CET 2014
device-mapper-persistent-data-0.2.8-5.el7    BUILT: Sat Mar  1 02:15:56 CET 2014
cmirror-2.02.105-14.el7    BUILT: Wed Mar 26 14:29:41 CET 2014


A part of test:

[root@virt-062 ~]# lsblk
NAME                    MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda                       8:0    0    12G  0 disk  
└─sda1                    8:1    0    12G  0 part  
  ├─vg-lvol0            253:2    0   500M  0 lvm   
  │ └─md127               9:127  0 499.7M  0 raid1 
  └─vg-lvol1            253:3    0   500M  0 lvm   
    └─md127               9:127  0 499.7M  0 raid1 

[root@virt-062 ~]# mdadm -S /dev/md127
mdadm: stopped /dev/md127
[root@virt-062 ~]# lvremove vg
Do you really want to remove active logical volume lvol0? [y/n]: y
  Logical volume "lvol0" successfully removed
Do you really want to remove active logical volume lvol1? [y/n]: y
  Logical volume "lvol1" successfully removed
[root@virt-062 ~]# lvcreate -L1G vg
WARNING: linux_raid_member signature detected on /dev/vg/lvol0 at offset 4096. Wipe it? [y/n] y
  Wiping linux_raid_member signature on /dev/vg/lvol0.
  Logical volume "lvol0" created

[root@virt-062 ~]# lsblk
NAME                    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda                       8:0    0   12G  0 disk 
└─sda1                    8:1    0   12G  0 part 
  ├─test-lvol0          253:2    0  500M  0 lvm  
  │ └─layer-lvol0       253:4    0  252M  0 lvm  
  └─test-lvol1          253:3    0    1G  0 lvm  
[root@virt-062 ~]# vgremove -ff layer
  Logical volume "lvol0" successfully removed
  Volume group "layer" successfully removed
[root@virt-062 ~]# vgremove -ff test
  Logical volume "lvol0" successfully removed
  Logical volume "lvol1" successfully removed
  Volume group "test" successfully removed

[root@virt-062 ~]# vgcreate vg /dev/sda1 /dev/sdb1 /dev/sdc1
  Volume group "vg" successfully created
[root@virt-062 ~]# lvcreate -L 500M vg
WARNING: LVM2_member signature detected on /dev/vg/lvol0 at offset 536. Wipe it? [y/n] y
  Wiping LVM2_member signature on /dev/vg/lvol0.
  Logical volume "lvol0" created


LVM successfully recognized signatures on devices.

Comment 5 Ludek Smid 2014-06-13 09:50:52 UTC
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.