+++ This bug was initially created as a clone of Bug #735306 +++ Description of problem: Current kernel/mdadm code does not support members >2TB when 0.9X metadata is used. Yet mdadm does not refuse to grow an existing array with this metadata version to >2TB members. Attempting to do so will render the array unusable. Version-Release number of selected component (if applicable): mdadm-3.2.2-6.fc15.x86_64 How reproducible: Always Steps to Reproduce: 1. create an array with 0.9x metadata using <2TB members 2. grow/replace the members to >2TB 3. mdadm --grow /dev/md0 --size max Actual results: mdadm does not refuse the operation or warn the user and renders the array unusable Expected results: mdadm refuses the operation Additional info: kernel-2.6.38.6-27.fc15.x86_64 --- Additional comment from p.zandbergen on 2011-09-02 08:32:57 EDT --- It has been suggested on the linux-raid mailinglist that mdadm ought to have complained in step 2 as well, the step where >2TB members were added to the array. --- Additional comment from Jes.Sorensen on 2011-10-20 09:39:17 EDT --- Pim, When you're saying 2TB members, do you mean individual drives > 2TB or a raid assembly that is larger than 2TB? Jes --- Additional comment from p.zandbergen on 2011-10-20 10:50:29 EDT --- Jes, My point is that mdadm should refuse to add drives >2TB to an array with 0.9X metadata and/or should refuse to grow this array in a way that would require individual drives to be >2TB. However, it now turns out that the real limit for individual drives in a 0.9X metadata array is 4TB. The fact that damage occurs long before that is another bug. See this thread in the linux-raid mailinglist http://marc.info/?l=linux-raid&m=131489162105812&w=2 Pim --- Additional comment from Jes.Sorensen on 2011-10-25 04:11:49 EDT --- Pim, Thanks for the details, looks like we need to apply the following three patches to mdadm in F15 and F16 and one patch to the F15 kernel (below). I am listing them all here for reference before cloning the bug to the right release versions. Jes commit 20a4675688e0384a1b4eac61b05f60fbf7747df9 Author: NeilBrown <neilb> Date: Thu Sep 8 13:08:51 2011 +1000 Grow: refuse to grow a 0.90 array beyond 2TB A kernel bug makes handling for arrays using more than 2TB per device incorrect, and the kernel doesn't stop an array from growing beyond any limit. This is fixed in 3.1 So prior to 3.1, make sure not to ask for an array to grow bigger than 2TB per device. Signed-off-by: NeilBrown <neilb> commit 11b391ece9fa284a151362537af093aa44883696 Author: NeilBrown <neilb> Date: Thu Sep 8 13:05:31 2011 +1000 Discourage large devices from being added to 0.90 arrays. 0.90 arrays can only use up to 4TB per device. So when a larger device is added, complain a bit. Still allow it if --force is given as there could be a valid use. Signed-off-by: NeilBrown <neilb> commit 01619b481883926f13da2b1b88f3125359a6a08b Author: NeilBrown <neilb> Date: Thu Sep 8 12:20:36 2011 +1000 Fix component size checks in validate_super0. A 0.90 array can use at most 4TB of each device - 2TB between 2.6.39 and 3.1 due to a kernel bug. The test for this in validate_super0 is very wrong. 'size' is sectors and the number it is compared against is just confusing. So fix it all up and correct the spelling of terabytes and remove a second redundant test on 'size'. Signed-off-by: NeilBrown <neilb> ========================================================================= This patch is also needed for the F15 kernel: From 27a7b260f71439c40546b43588448faac01adb93 Mon Sep 17 00:00:00 2001 From: NeilBrown <neilb> Date: Sat, 10 Sep 2011 17:21:28 +1000 Subject: [PATCH] md: Fix handling for devices from 2TB to 4TB in 0.90 metadata. 0.90 metadata uses an unsigned 32bit number to count the number of kilobytes used from each device. This should allow up to 4TB per device. However we multiply this by 2 (to get sectors) before casting to a larger type, so sizes above 2TB get truncated. Also we allow rdev->sectors to be larger than 4TB, so it is possible for the array to be resized larger than the metadata can handle. So make sure rdev->sectors never exceeds 4TB when 0.90 metadata is in used. Also the sanity check at the end of super_90_load should include level 1 as it used ->size too. (RAID0 and Linear don't use ->size at all). Reported-by: Pim Zandbergen <P.Zandbergen> Cc: stable Signed-off-by: NeilBrown <neilb> --- drivers/md/md.c | 12 ++++++++++-- 1 files changed, 10 insertions(+), 2 deletions(-)
mdadm-3.2.2-14.fc16 has been submitted as an update for Fedora 16. https://admin.fedoraproject.org/updates/mdadm-3.2.2-14.fc16
Package mdadm-3.2.2-14.fc16: * should fix your issue, * was pushed to the Fedora 16 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing mdadm-3.2.2-14.fc16' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2011-15731 then log in and leave karma (feedback).
mdadm-3.2.2-15.fc16 has been submitted as an update for Fedora 16. https://admin.fedoraproject.org/updates/mdadm-3.2.2-15.fc16
mdadm-3.2.2-15.fc16 has been pushed to the Fedora 16 stable repository. If problems still persist, please make note of it in this bug report.