Bug 748731 - mdadm may grow an array beyond supported limits
mdadm may grow an array beyond supported limits
Product: Fedora
Classification: Fedora
Component: mdadm (Show other bugs)
x86_64 Linux
unspecified Severity high
: ---
: ---
Assigned To: Jes Sorensen
Fedora Extras Quality Assurance
Depends On: 735306 748732
  Show dependency treegraph
Reported: 2011-10-25 04:13 EDT by Jes Sorensen
Modified: 2011-12-14 18:38 EST (History)
6 users (show)

See Also:
Fixed In Version: mdadm-3.2.2-15.fc16
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 735306
Last Closed: 2011-12-14 18:38:06 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Jes Sorensen 2011-10-25 04:13:02 EDT
+++ This bug was initially created as a clone of Bug #735306 +++

Description of problem:
Current kernel/mdadm code does not support members >2TB when 0.9X metadata is used. Yet mdadm does not refuse to grow an existing array with this metadata version to >2TB members. Attempting to do so will render the array unusable.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. create an array with 0.9x metadata using <2TB members
2. grow/replace the members to >2TB
3. mdadm --grow /dev/md0 --size max
Actual results:
mdadm does not refuse the operation or warn the user
and renders the array unusable

Expected results:
mdadm refuses the operation

Additional info:

--- Additional comment from p.zandbergen@macroscoop.nl on 2011-09-02 08:32:57 EDT ---

It has been suggested on the linux-raid mailinglist that mdadm ought to have
complained in step 2 as well, the step where >2TB members were added to the array.

--- Additional comment from Jes.Sorensen@redhat.com on 2011-10-20 09:39:17 EDT ---


When you're saying 2TB members, do you mean individual drives > 2TB or
a raid assembly that is larger than 2TB?


--- Additional comment from p.zandbergen@macroscoop.nl on 2011-10-20 10:50:29 EDT ---


My point is that mdadm should refuse to add drives >2TB to an array with 0.9X metadata and/or should refuse to grow this array in a way that would require individual drives to be >2TB.

However, it now turns out that the real limit for individual drives in a 0.9X metadata array is 4TB. The fact that damage occurs long before that is another bug.

See this thread in the linux-raid mailinglist


--- Additional comment from Jes.Sorensen@redhat.com on 2011-10-25 04:11:49 EDT ---


Thanks for the details, looks like we need to apply the following three
patches to mdadm in F15 and F16 and one patch to the F15 kernel (below).

I am listing them all here for reference before cloning the bug to the
right release versions.


commit 20a4675688e0384a1b4eac61b05f60fbf7747df9
Author: NeilBrown <neilb@suse.de>
Date:   Thu Sep 8 13:08:51 2011 +1000

    Grow: refuse to grow a 0.90 array beyond 2TB
    A kernel bug makes handling for arrays using more than 2TB per device
    incorrect, and the kernel doesn't stop an array from growing beyond
    any limit.
    This is fixed in 3.1
    So prior to 3.1, make sure not to ask for an array to grow bigger than
    2TB per device.
    Signed-off-by: NeilBrown <neilb@suse.de>

commit 11b391ece9fa284a151362537af093aa44883696
Author: NeilBrown <neilb@suse.de>
Date:   Thu Sep 8 13:05:31 2011 +1000

    Discourage large devices from being added to 0.90 arrays.
    0.90 arrays can only use up to 4TB per device.  So when a larger
    device is added, complain a bit.  Still allow it if --force is given
    as there could be a valid use.
    Signed-off-by: NeilBrown <neilb@suse.de>

commit 01619b481883926f13da2b1b88f3125359a6a08b
Author: NeilBrown <neilb@suse.de>
Date:   Thu Sep 8 12:20:36 2011 +1000

    Fix component size checks in validate_super0.
    A 0.90 array can use at most 4TB of each device - 2TB between
    2.6.39 and 3.1 due to a kernel bug.
    The test for this in validate_super0 is very wrong.  'size' is sectors
    and the number it is compared against is just confusing.
    So fix it all up and correct the spelling of terabytes and remove
    a second redundant test on 'size'.
    Signed-off-by: NeilBrown <neilb@suse.de>


This patch is also needed for the F15 kernel:

From 27a7b260f71439c40546b43588448faac01adb93 Mon Sep 17 00:00:00 2001
From: NeilBrown <neilb@suse.de>
Date: Sat, 10 Sep 2011 17:21:28 +1000
Subject: [PATCH] md: Fix handling for devices from 2TB to 4TB in 0.90

0.90 metadata uses an unsigned 32bit number to count the number of
kilobytes used from each device.
This should allow up to 4TB per device.
However we multiply this by 2 (to get sectors) before casting to a
larger type, so sizes above 2TB get truncated.

Also we allow rdev->sectors to be larger than 4TB, so it is possible
for the array to be resized larger than the metadata can handle.
So make sure rdev->sectors never exceeds 4TB when 0.90 metadata is in

Also the sanity check at the end of super_90_load should include level
1 as it used ->size too. (RAID0 and Linear don't use ->size at all).

Reported-by: Pim Zandbergen <P.Zandbergen@macroscoop.nl>
Cc: stable@kernel.org
Signed-off-by: NeilBrown <neilb@suse.de>
 drivers/md/md.c |   12 ++++++++++--
 1 files changed, 10 insertions(+), 2 deletions(-)
Comment 1 Fedora Update System 2011-11-10 05:00:22 EST
mdadm-3.2.2-14.fc16 has been submitted as an update for Fedora 16.
Comment 2 Fedora Update System 2011-11-10 20:24:00 EST
Package mdadm-3.2.2-14.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing mdadm-3.2.2-14.fc16'
as soon as you are able to.
Please go to the following url:
then log in and leave karma (feedback).
Comment 3 Fedora Update System 2011-11-23 05:47:26 EST
mdadm-3.2.2-15.fc16 has been submitted as an update for Fedora 16.
Comment 4 Fedora Update System 2011-12-14 18:38:06 EST
mdadm-3.2.2-15.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.