Bug 846335

Summary: Dracut fails to incrementally assemble nested raid configurations
Product: [Fedora] Fedora Reporter: Ian Dall <ian>
Component: dracutAssignee: dracut-maint
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 17CC: dracut-maint, harald, jonathan, trevor
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-12-20 15:53:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Patch to enable dracut to work woth stacked raids
none
Proposed patch
none
mdadm.conf
none
Sample /proc/mdstat none

Description Ian Dall 2012-08-07 13:32:45 UTC
Description of problem:
Dracut fails to incrementally assemble nested raid configurations. If root is on such a raid of raids then the system fails to boot and dracut drops to a shell prompt.

Version-Release number of selected component (if applicable):

dracut-018-96.git20120724.fc17

How reproducible:
Always

Steps to Reproduce:
assuming sda, sdb, sdc and sdd
1.create level0 raid md0 with components sda1, sdb1
2.create level0 raid md1 with components sdc1, sdd1
3.create level1 raid md2 with components md0 and md1
4. put root on md2, run dracut and reboot
  
Actual results:
Failure in initramfs to find the root partition and drops to a shell prompt.

Expected results:
Successful boot.

Additional info:

This is a regression. It used to work in Fedora14, but am not clear when it stopped working.

The issue is in /lib/dracut/modules.d/90mdraid/65-md-incremental-imsm.rules, the line KERNEL=="md*", GOTO="md_end", which prevents md* being considered to be raid components. The equivalent /lib/udev/rules.d/65-md-incremental.rules has code to allow nested raid. The changes needed seem to be a bit more sbtle than simply removing the KERNEL=="md*" line.

Comment 1 Harald Hoyer 2012-08-17 12:49:59 UTC
why would someone want to stack raids?

Comment 2 Ian Dall 2012-08-18 08:55:15 UTC
Good question. mdraid has had this feature for a long time (maybe forever). It allows you to have a mirror of striped volumes (for example), or a mirror of a striped volume and a single block device. This is largely, but not completely, made redundant by raid10. The mirror of raids has advantages when there is disparity in the speed of the devices. So, for example, you might have a raid0 of 15k rpm SAS disks, mirrored by one or a few big slow disks or an iscsi block device. Bid slow disks are practically free so this is a cost effective way of getting the performance of the striped 15k disks plus redundancy without wasting any of the 15k disks space. mdraid has a --write-mostly option to support this.

Whether the performance is achieved depends rather on the read/write ratio. I put relatively static filesystems - such as root - on this volume.

Comment 3 Ian Dall 2012-08-18 09:02:14 UTC
Created attachment 605314 [details]
Patch to enable dracut to work woth stacked raids

The attached patch works for me. I'm not a udev guru so I might have missed something.

Comment 4 Harald Hoyer 2012-08-22 11:03:58 UTC
Created attachment 606195 [details]
Proposed patch

Comment 5 Harald Hoyer 2012-08-22 11:04:23 UTC
Can you try with my proposed patch?

Comment 6 Ian Dall 2012-08-23 14:48:59 UTC
That works for me. Thanks!

Comment 7 Harald Hoyer 2012-09-05 13:31:33 UTC
upstream commit 5746f04994745ac4ad01b4ae26f6c368ac725fb3

Comment 8 Fedora Update System 2012-09-27 10:22:30 UTC
dracut-018-60.git20120927.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/dracut-018-60.git20120927.fc16

Comment 9 Fedora Update System 2012-09-27 10:23:30 UTC
dracut-018-105.git20120927.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/dracut-018-105.git20120927.fc17

Comment 10 Fedora Update System 2012-09-27 10:30:56 UTC
dracut-018-105.git20120927.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/dracut-018-105.git20120927.fc17

Comment 11 Fedora Update System 2012-09-28 08:20:38 UTC
Package dracut-018-105.git20120927.fc17:
* should fix your issue,
* was pushed to the Fedora 17 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing dracut-018-105.git20120927.fc17'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-14953/dracut-018-105.git20120927.fc17
then log in and leave karma (feedback).

Comment 12 Fedora Update System 2012-12-20 15:53:38 UTC
dracut-018-105.git20120927.fc17 has been pushed to the Fedora 17 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 13 Trevor Cordes 2015-10-22 08:31:33 UTC
I'm seeing this problem in Fedora 21 on an nested (1 on 0) array I just created.  Ian, can I ask you to paste the mdadm.conf you used when you saw this bug fixed?  Also, your fdisk -l /dev/sd* and /dev/md* regarding the relevant constituents of your arrays?  I want to check what I'm doing before opening a new bug.

Thanks!

Comment 14 Trevor Cordes 2015-10-22 09:17:34 UTC
Also, why change the *imsm* file?  Are your arrays using Intel Matrix Storage Manager (isn't that what imsm means?)?  Or am I wrong what imsm stands for?  Why would md (without imsm superblocks) rely on a *imsm* udev file?  Why aren't the same "nested ability" rules in the plain *md* udev files?

Comment 15 Ian Dall 2015-10-22 13:35:57 UTC
I'm not currently using quite the same configuration but I'll attach mdadm.conf and /proc/mdstat. I don't know what your referring to with *imsm* files. They are not something I use and they don't seem to be in the patch.

I am still on Fedora20 which doesn't show any problems with assembling these arrays.

Comment 16 Ian Dall 2015-10-22 13:39:01 UTC
Created attachment 1085528 [details]
mdadm.conf

Config file to allow auto configuring nested arrays

Comment 17 Ian Dall 2015-10-22 13:43:21 UTC
Created attachment 1085554 [details]
Sample /proc/mdstat

/proc/mdstat showing md3 is a raid 1 with members sde (a ssd disk) and md1 (a raid0 of spinning rust).

Comment 18 Trevor Cordes 2015-10-24 21:35:02 UTC
Thanks Ian, with your help I was able to figure out my problem.  I'm not sure it's a bug, deserving of a new bz (maybe someone can offer guidance), but here it is for others who have the same problem.

My setup was RAID-0 on a drive partition, then the RAID-0 was partitioned, then those partitions became components of other RAID-1 arrays.  They wouldn't assemble at boot no matter what, but would work fine while the system was active after I added them to the arrays.  Nothing I did in mdadm.conf and grub2.conf would have these show up at boot.  One of my arrays was root.

The problem was the partitions in the middle.  As soon as I got rid of the paritions-on-RAID-0 and instead made it RAID-0-on-partitions the problem went away and my arrays survived reboot (with an almost empty mdadm.conf too).  There seems to be a problem with the nesting detection code when one of the arrays is itself partitioned.

You might say that partitioning a RAID-0 array is stupid, and that's a valid point, but I had my reasons.  Regardless, I do believe any combination of nested RAID and partitions, any X levels deep, theoretically should work on linux.  Certainly if mdadm allows me to make a frankenstein, the boot scripts should allow him to survive a reboot without manual intervention.  If someone agrees that this is a bug, I'll make a new ticket and create some simple test cases.