Red Hat Bugzilla – Bug 846335
Dracut fails to incrementally assemble nested raid configurations
Last modified: 2015-10-24 17:35:02 EDT
Description of problem:
Dracut fails to incrementally assemble nested raid configurations. If root is on such a raid of raids then the system fails to boot and dracut drops to a shell prompt.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
assuming sda, sdb, sdc and sdd
1.create level0 raid md0 with components sda1, sdb1
2.create level0 raid md1 with components sdc1, sdd1
3.create level1 raid md2 with components md0 and md1
4. put root on md2, run dracut and reboot
Failure in initramfs to find the root partition and drops to a shell prompt.
This is a regression. It used to work in Fedora14, but am not clear when it stopped working.
The issue is in /lib/dracut/modules.d/90mdraid/65-md-incremental-imsm.rules, the line KERNEL=="md*", GOTO="md_end", which prevents md* being considered to be raid components. The equivalent /lib/udev/rules.d/65-md-incremental.rules has code to allow nested raid. The changes needed seem to be a bit more sbtle than simply removing the KERNEL=="md*" line.
why would someone want to stack raids?
Good question. mdraid has had this feature for a long time (maybe forever). It allows you to have a mirror of striped volumes (for example), or a mirror of a striped volume and a single block device. This is largely, but not completely, made redundant by raid10. The mirror of raids has advantages when there is disparity in the speed of the devices. So, for example, you might have a raid0 of 15k rpm SAS disks, mirrored by one or a few big slow disks or an iscsi block device. Bid slow disks are practically free so this is a cost effective way of getting the performance of the striped 15k disks plus redundancy without wasting any of the 15k disks space. mdraid has a --write-mostly option to support this.
Whether the performance is achieved depends rather on the read/write ratio. I put relatively static filesystems - such as root - on this volume.
Created attachment 605314 [details]
Patch to enable dracut to work woth stacked raids
The attached patch works for me. I'm not a udev guru so I might have missed something.
Created attachment 606195 [details]
Can you try with my proposed patch?
That works for me. Thanks!
upstream commit 5746f04994745ac4ad01b4ae26f6c368ac725fb3
dracut-018-60.git20120927.fc16 has been submitted as an update for Fedora 16.
dracut-018-105.git20120927.fc17 has been submitted as an update for Fedora 17.
* should fix your issue,
* was pushed to the Fedora 17 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing dracut-018-105.git20120927.fc17'
as soon as you are able to.
Please go to the following url:
then log in and leave karma (feedback).
dracut-018-105.git20120927.fc17 has been pushed to the Fedora 17 stable repository. If problems still persist, please make note of it in this bug report.
I'm seeing this problem in Fedora 21 on an nested (1 on 0) array I just created. Ian, can I ask you to paste the mdadm.conf you used when you saw this bug fixed? Also, your fdisk -l /dev/sd* and /dev/md* regarding the relevant constituents of your arrays? I want to check what I'm doing before opening a new bug.
Also, why change the *imsm* file? Are your arrays using Intel Matrix Storage Manager (isn't that what imsm means?)? Or am I wrong what imsm stands for? Why would md (without imsm superblocks) rely on a *imsm* udev file? Why aren't the same "nested ability" rules in the plain *md* udev files?
I'm not currently using quite the same configuration but I'll attach mdadm.conf and /proc/mdstat. I don't know what your referring to with *imsm* files. They are not something I use and they don't seem to be in the patch.
I am still on Fedora20 which doesn't show any problems with assembling these arrays.
Created attachment 1085528 [details]
Config file to allow auto configuring nested arrays
Created attachment 1085554 [details]
/proc/mdstat showing md3 is a raid 1 with members sde (a ssd disk) and md1 (a raid0 of spinning rust).
Thanks Ian, with your help I was able to figure out my problem. I'm not sure it's a bug, deserving of a new bz (maybe someone can offer guidance), but here it is for others who have the same problem.
My setup was RAID-0 on a drive partition, then the RAID-0 was partitioned, then those partitions became components of other RAID-1 arrays. They wouldn't assemble at boot no matter what, but would work fine while the system was active after I added them to the arrays. Nothing I did in mdadm.conf and grub2.conf would have these show up at boot. One of my arrays was root.
The problem was the partitions in the middle. As soon as I got rid of the paritions-on-RAID-0 and instead made it RAID-0-on-partitions the problem went away and my arrays survived reboot (with an almost empty mdadm.conf too). There seems to be a problem with the nesting detection code when one of the arrays is itself partitioned.
You might say that partitioning a RAID-0 array is stupid, and that's a valid point, but I had my reasons. Regardless, I do believe any combination of nested RAID and partitions, any X levels deep, theoretically should work on linux. Certainly if mdadm allows me to make a frankenstein, the boot scripts should allow him to survive a reboot without manual intervention. If someone agrees that this is a bug, I'll make a new ticket and create some simple test cases.