Bug 225109 - mdadm doesn't autostart lvm member
mdadm doesn't autostart lvm member
Product: Fedora
Classification: Fedora
Component: mdadm (Show other bugs)
All Linux
medium Severity medium
: ---
: ---
Assigned To: Doug Ledford
Depends On:
  Show dependency treegraph
Reported: 2007-01-28 18:56 EST by Douglas E. Warner
Modified: 2007-11-30 17:11 EST (History)
0 users

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2007-01-29 11:52:55 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Douglas E. Warner 2007-01-28 18:56:07 EST
Description of problem:
Software RAID-5 array with four SATA and one LVM member does not autostart the 
LVM member.
Essentially, starting the array with:
mdadm -A -s
will start the array degraded with the four SATA members but will not start 
the array with the LVM member even if LVM has been initialized.

I believe there's an additional problem that my device is /dev/md0 which gets 
autostarted before LVM has been initialized.  I've commented out the lines in 
rc.sysinit to test starting up the RAID array after LVM has been started but 
mdadm still doesn't include the LVM member.

Version-Release number of selected component (if applicable):
# mdadm --version
mdadm - v2.5.4 - 13 October 2006

# uname -r

How reproducible:
Build a RAID-5 array with an LVM member.  My array's setup is currently:
# mdadm --detail /dev/md0
        Version : 00.90.03
  Creation Time : Sat Jan 13 17:35:47 2007
     Raid Level : raid5
     Array Size : 1562834944 (1490.44 GiB 1600.34 GB)
    Device Size : 390708736 (372.61 GiB 400.09 GB)
   Raid Devices : 5
  Total Devices : 5
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Sun Jan 28 17:30:26 2007
          State : clean, degraded, recovering
 Active Devices : 4
Working Devices : 5
 Failed Devices : 0
  Spare Devices : 1

         Layout : left-symmetric
     Chunk Size : 64K

 Rebuild Status : 70% complete

           UUID : 5bc899dd:77d71cad:4423409f:259f9db5
         Events : 0.67768

    Number   Major   Minor   RaidDevice State
       0       8       33        0      active sync   /dev/sdc1
       1       8       17        1      active sync   /dev/sdb1
       2       8       49        2      active sync   /dev/sdd1
       3       8       81        3      active sync   /dev/sdf1
       5     253        2        4      spare rebuilding   /dev/fake400G/lvol0

Steps to Reproduce:
1. create the RAID-5 device
2. comment out the RAID lines in rc.sysinit similar to:
update_boot_stage RCraid
#[ -x /sbin/nash ] && echo "raidautorun /dev/md0" | nash --quiet
#if [ -f /etc/mdadm.conf ]; then
#    /sbin/mdadm -A -s
3. verify the LVM device exists and is initialized:
# lvdisplay
  --- Logical volume ---
  LV Name                /dev/fake400G/lvol0
  VG Name                fake400G
  LV UUID                5C05Qx-gHu9-PZf2-COxd-1fmn-5d3h-uywp17
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                386.25 GB
  Current LE             24720
  Segments               2
  Allocation             inherit
  Read ahead sectors     0
  Block device           253:2
4. Attempt to autostart the RAID device:
# mdadm -A -s
Actual results:
The RAID-5 device is started without the LVM member:
# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sdf1[3] sdd1[2] sdc1[0] sdb1[1]
      1562834944 blocks level 5, 64k chunk, algorithm 2 [5/4] [UUUU_]

After adding the device the array will rebuild:
# mdadm --manage /dev/md0 --add /dev/mapper/fake400G-lvol0
# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 dm-2[5] sdf1[3] sdd1[2] sdc1[0] sdb1[1]
      1562834944 blocks level 5, 64k chunk, algorithm 2 [5/4] [UUUU_]
      [==============>......]  recovery = 74.1% (289719936/390708736) 
finish=37.7min speed=44611K/sec

Expected results:
The array should get autostarted with the LVM member so that the array does 
not need rebuilt.
Comment 1 Doug Ledford 2007-01-29 11:52:55 EST
The reason the lvm member is getting left out is likely that the superblock
generation numbers don't match up properly.  At a minimum, running raid over
some lvm and some non-lvm devices is going to be tricky.  Here's some of the
things you will have to get right:

1.  Don't use the Linux Raid Autodetect partition type on the non-lvm devices. 
During boot up, the initrd will look for all partitions labeled as type Linux
Raid Autodetect (0xfd) and tell the kernel to try and autostart them.  However,
lvm devices don't have regular partition tables, and therefore no partition
type, so they will *never* get added to this autostart list.  As a result,
especially with only one device on lvm, if all the other devices have their type
set to the autostart type, then the kernel could start the device in degraded
mode, and once that happens, the only way to get the lvm device back into the
array is to add it as a spare and let it rebuild.  So, switch any partitions
that are type 0xfd to 0x83 (Linux) and you should be good on this point.

2.  Once the partitions are no longer autodetect type, then the initrd will not
automatically start the array.  If your array is a / device then that's bad,
otherwise it's OK (let me know if it is and I can tack a patch to mkinitrd that
solves the problem by starting the / device using mdadm in the generated initrd
image).  The /etc/rc.d/rc.sysinit script will invoke a second attempt at raid
autorun (which will do nothing), then if there is an /etc/mdadm.conf file, the
script will invoke mdadm to assemble non-autostart arrays.  This is where you
will want your array to be started at.  As long as you have the line:

DEVICE partitions

in your mdadm.conf file (and the necessary ARRAY lines too), it will find raid
members on started lvm devices.  In order to make sure the lvm is started before
the raid, you can verify that the initrd starts the lvm devices.  If the initrd
doesn't start the lvm device in question, then running mkinitrd and building a
new initrd image that forces that lvm device to be started will solve that issue.

3.  Finally, you have to make sure to stop the md device prior to stopping the
lvm subsystem at shutdown or else it's possible that the final superblock update
that marks each of the devices as clean and in sync might get lost on the lvm
device, which would prevent the lvm device from being included at the next
reboot and leave you in exactly the same position you are now.  The preferred
sequence would be umount md device, completely stop md device, completely stop
lvm device, flush hard disk cache.  Whether or not the kernel/initscripts does
things in this order on your system I'm not sure.  I can't remember if lvm or md
devices are stopped first by default.  And if you were to use md devices in an
lvm group, then the order would need to be reversed, so I'm not sure this is
something we can automatically get right anyway.  So, you will just need to
check the ordering of shutdown scripts in /etc/rc.d/rc6.d and possibly reorder
the lvm/md shutdown in order to make it shut down cleanly.

If you do all of these things, then it *should* work.  But, as you can probably
tell, this isn't an expected to work "out of the box" situation.  As such, I'm
going to close this as CANTFIX, but if you have any more questions, feel free to
post them and I'll respond when I can.

Note You need to log in before you can comment on or make changes to this bug.