Bug 139207 - Support for drives over 2TB is buggy
Support for drives over 2TB is buggy
Status: CLOSED CANTFIX
Product: Fedora
Classification: Fedora
Component: kernel (Show other bugs)
5
i386 Linux
medium Severity medium
: ---
: ---
Assigned To: Dave Jones
Brian Brock
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2004-11-13 19:46 EST by Glen
Modified: 2015-01-04 17:12 EST (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2006-10-30 17:30:20 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
dmesg output after I run dd (234.47 KB, text/plain)
2005-06-30 23:09 EDT, Glen
no flags Details
More of what dmesg spits out (227.71 KB, text/plain)
2005-06-30 23:10 EDT, Glen
no flags Details

  None (edit)
Description Glen 2004-11-13 19:46:25 EST
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5)
Gecko/20041111 Firefox/1.0

Description of problem:
I have two RAID boxes, SB3140s (best I can tell ya, they're some
generic things from taiwan), that convert ide drives into what looks
to the operating system as one big SCSI drive.  Currently the size of
each RAID unit is roughly 3.5 Terabytes.

I hooked up just one of the drives to my system, booted Fedora Core 3
and the dmesg command is showing me this:

Fusion MPT base driver 3.01.16
Copyright (c) 1999-2004 LSI Logic Corporation
ACPI: PCI interrupt 0000:02:02.0[A] -> GSI 5 (level, low) -> IRQ 5
mptbase: Initiating ioc0 bringup
ioc0: 53C1030: Capabilities={Initiator}
ACPI: PCI interrupt 0000:01:04.0[A] -> GSI 5 (level, low) -> IRQ 5
mptbase: Initiating ioc1 bringup
ioc1: 53C1030: Capabilities={Initiator}
Fusion MPT SCSI Host driver 3.01.16
scsi0 : ioc0: LSI53C1030, FwRev=01030f00h, Ports=1, MaxQ=222, IRQ=5
  Vendor: SB-3140   Model:                   Rev: 0001
  Type:   Direct-Access                      ANSI SCSI revision: 03
SCSI device sda: 4294950912 512-byte hdwr sectors (2199015 MB)
SCSI device sda: drive cache: write back
 sda: unknown partition table
Attached scsi disk sda at scsi0, channel 0, id 3, lun 0
scsi: host 0 channel 0 id 3 lun 0x00000200080c0400 has a LUN larger
than currently supported.
scsi: host 0 channel 0 id 3 lun 0xff010000ffffffff has a LUN larger
than currently supported.
scsi: host 0 channel 0 id 3 lun 0x0002202020202020 has a LUN larger
than currently supported.
scsi: host 0 channel 0 id 3 lun808529923 has a LUN larger than allowed
by the host adapter
scsi: host 0 channel 0 id 3 lun3078 has a LUN larger than allowed by
the host adapter
scsi1 : ioc1: LSI53C1030, FwRev=01030700h, Ports=1, MaxQ=222, IRQ=5
  Vendor: MAXTOR    Model: ATLAS10K4_36SCA   Rev: DFV0
  Type:   Direct-Access                      ANSI SCSI revision: 03
SCSI device sdb: 71833096 512-byte hdwr sectors (36779 MB)
SCSI device sdb: drive cache: write back
 sdb: sdb1
 sdb1: <netbsd: sdb5 sdb6 sdb7 sdb8 >
Attached scsi disk sdb at scsi1, channel 0, id 0, lun 0
  Vendor: SDR       Model: GEM318            Rev: 0
  Type:   Processor                          ANSI SCSI revision: 02

Also, even though the size of the drive should be 3.5 Terabytes, fdisk
is saying this to me
Disk /dev/sda: 2199.0 GB, 2199014866944 bytes
255 heads, 63 sectors/track, 267348 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1      267348  2147472778+  83  Linux:

I also experimented with a different SCSI controller figuring that my
particular controller might be the culprit, but with an adaptec
controller that uses the aic79xx module I got pretty much the exact
same thing.  In fact with that controller Fedora Core 3 didn't even
detect the drive and it wasn't until after I installed Fedora and
played around loading modules that I even got the kernel to recognize
my disk (an adaptec 39320 SCSI controller by the way).

I've also played with this system by splitting the RAIDS in half, so
they each total about 1.7TB, and this works, but if I put two RAIDS on
the same scsi id but with different luns I still get those weird
errors in dmesg.

Version-Release number of selected component (if applicable):
kernel-smp-2.6.9-1.667

How reproducible:
Always

Steps to Reproduce:
1. Boot up Fedora with an SB3140 RAID unit attached
2.
3.
    

Actual Results:  I get weird errors about the drive.

Expected Results:  I expect to be able to access all 3.5 Terabytes

Additional info:
Comment 1 Glen 2005-06-28 20:12:27 EDT
I've got a new unit, that's around 5 TB in size.  I booted up Fedora Core 4 with
the thing attached, and this time dmesg showed the kernel detecting the right
size.  As soon as I try and write to the thing however it starts spitting out
all sorts of SCSI errors.  Looks like I'll need to split this thing in half as well.
Comment 2 Glen 2005-06-30 23:07:28 EDT
My SCSI adapter uses the aic79xx module by the way.  I recompiled a kernel with
all the debugging options turned on for that driver and I'm attaching the junk
that dmesg spits out, if that's any help.
Comment 3 Glen 2005-06-30 23:09:48 EDT
Created attachment 116218 [details]
dmesg output after I run dd

This is what dmesg spits out after I run this:
dd if=/dev/zero of=/dev/sda bs=1K count=10000
Comment 4 Glen 2005-06-30 23:10:55 EDT
Created attachment 116219 [details]
More of what dmesg spits out

Here's more of what dmesg spits out after I try and write zeros to my drive.
Comment 5 Glen 2005-07-02 18:02:24 EDT
Looks like I misdiagnosed my latest problem just a little bit.  I guess I'm so
used to large disks not working that I just assumed.  I split up my RAID into
two RAID's, each roughly 2TB in size.  Once again, when I went to format them I
get the same errors popping up.  It seems my adapter doesn't work under Linux. 
The adapter in question is an Adaptec 39320A-R.  As I mentioned before it uses
the aic79xx driver.

I'm pretty sure my issue isn't a hardware problem either.  I put Windows server
2003 on my system for a little bit and it was able to handle the 5 terabyte disk
just fine.  I even upgraded my adapter's firmware, but that didn't help at all
under Linux.

Also, an LSI adapter seems to handle the drives just fine under Linux, although
I haven't tested a 5 TB disk with the LSI adapter just yet, only a 2 TB array. 
If I have issues with size and the LSI adapter I'll let you know.
Comment 6 Glen 2005-07-02 19:38:01 EDT
Well shoot.  It looks like I spoke too soon again.  I'm getting similar errors
now with the LSI cards, and this is just on a 2TB array.  I'm beginning to
suspect that these RAID units are designed to only work with Windows.  I guess
this is what be get for buying bargin basement generic RAID units?

There is a 64bit LBA mode I can put the RAID unit in, but then my server can't
see it at all.  Perhaps I need a special adapter for that?
Comment 7 Tom Coughlan 2005-07-05 15:48:41 EDT
You must enable 64bit LBAs in the storage device to address > 2 TB. 

With FC4 you should not need to do anything special to enable 64bit LBAs
(CONFIG_LBD=y by default). 

Please post the messages you get with the mpt fusion driver, and the storage set
for 64bit LBA mode. 

Comment 8 Glen 2005-07-07 14:09:40 EDT
Actually, I lied a little bit.  When I put the RAID into "64bit LBA" mode my
server can see it, but it only lists it as being 2TB.  Even my SCSI adapter's
bios screen lists it as being 2TB.  It's only when I put the RAID into "Windows"
mode does it see the whole 5TB.

And now you guys are going to laugh at my shoddy hardware, but when I drop the
speed down from U320 to U160 speeds it starts to work.  I also tried turning off
QAS and tagged queueing, but the only thing that seems to help is dropping the
speed down.  So I'm thinking it's just my RAID unit is crappy.  Or maybe it's
still a bug with Linux?  I don't think so though, as it's the same thing with
every distro I've tried, and also with FreeBSD, NetBSD, and DragonflyBSD.

Of course now the issue is that the user space tools (fdisk, sfdisk, parted,
mke2fs) don't seem to support a 5TB disk, so it looks like I'll need to split my
RAID up after all.
Comment 9 Glen 2005-07-07 14:17:58 EDT
Hmm, but the funny thing is how the RAID unit works in U320 mode when the server
is running Windows Server 2003.  I've contacted the guys that sold us this junky
thing and they say it works for them with Fedora and that they're going to help
me out.  But then I suspect they don't even test it with anything but Windows. 
So far they've just told me idiot stuff like how I can't have the RAID set up as
one big 5TB RAID, when it doesn't matter either way.  *Sigh*

Oh well.  I'll let you guys know if I figure anything else out.
Comment 10 Dave Jones 2005-07-15 17:46:43 EDT
[This comment has been added as a mass update for all FC4 kernel bugs.
 If you have migrated this bug from an FC3 bug today, ignore this comment.]

Please retest your problem with todays 2.6.12-1.1398_FC4 update.

If your problem involved being unable to boot, or some hardware not being
detected correctly, please make sure your /etc/modprobe.conf is correct *BEFORE*
installing any kernel updates.
If in doubt, you can recreate this file using..

mv /etc/sysconfig/hwconf /etc/sysconfig/hwconf.bak
mv /etc/modprobe.conf /etc/modprobe.conf.bak
kudzu


Thank you.
Comment 11 Adrian Reber 2005-07-16 02:32:40 EDT
FYI: I also have problems with drives bigger than 2TB on a qlogic controller.
See bug #150991
Comment 12 Jesse Keating 2005-07-17 14:13:48 EDT
Ok guys, please make sure that you use a gpt disk label on these > 2TB volumes.
 parted mklabel gpt

Only a GPT disk label can manage a physical volume that is > 2TB in size.  This
does mean that you will not be able to boot from it. Neither grub nor lilo
support booting from gpt at this time.  Anaconda has yet to be modified to
detect these large volumes and do the right thing, however RH is accepting
patches.  I just haven't had time to make the patches myself.

Please let me know if you have trouble w/ these volumes after using parted to
make a gpt disk label and then partitioning w/ parted.  Leave fdisk alone.
Comment 13 Adrian Reber 2005-07-17 17:30:55 EDT
Using a gpt disk label fixes the error for me with the qla2300 module:

/dev/sdf1             3.6T  121M  3.6T   1% /ftp/pub/.3
Comment 14 Glen 2005-07-21 22:47:00 EDT
I get this when I run parted:

You found a bug in GNU Parted.  Please email a bug report to bug-parted@gnu.org
containing the version (1.6.22), and the following message:

The sector size on /dev/sda is 4096 bytes.  Parted is known not to work properly
with drives with sector sizes other than 512 bytes
Comment 15 Tom Coughlan 2005-07-25 09:40:15 EDT
Glen,

Does the o.s. report that the sector size on /dev/sda is 4096 bytes? (It does
not in your original post, it says "512-byte hdwr sectors", but maybe something
has changed since then.)

Tom
Comment 16 Dave Jones 2005-09-30 03:18:49 EDT
Mass update to all FC4 bugs:

An update has been released (2.6.13-1.1526_FC4) which rebases to a new upstream
kernel (2.6.13.2). As there were ~3500 changes upstream between this and the
previous kernel, it's possible your bug has been fixed already.

Please retest with this update, and update this bug if necessary.

Thanks.
Comment 17 Dave Jones 2005-11-10 15:27:36 EST
2.6.14-1.1637_FC4 has been released as an update for FC4.
Please retest with this update, as a large amount of code has been changed in
this release, which may have fixed your problem.

Thank you.
Comment 18 Dave Jones 2005-12-27 22:10:14 EST
Glen, any update to comment #15 ?
Comment 19 Glen 2005-12-28 14:58:19 EST
Well I run rhel 4 primarily on my fileservers these days.  But then I don't know
why we're wasting the money and not going centos, or solaris even.  Well, I know
why, I stupidly assumed university pricing included tech support.  Of course
I've heard many a time that Red Hat tech support is basically worthless anyway,
so no big loss I spose.  Then again, ever since the switch to rhel you bugzilla
guys do seem to be pretty helpful, so I guess I can't complain too much.  Sorry
to go off on a tangent, I know all that's neither here nor there.  I guess I
needed to vent a little bit.

Anyway, on the remaining Fedora file servers I do run (fc4, latest updates and
all that) it tells me the same thing when I run dmesg, 512 byte hardware
sectors.  Ditto for all my rhel 4 machines.  But then, barring the 3TB logical
volumes I'm experimenting with, everything is under 2TB at this point.  Not that
that matters probably.

On some newer units I have, I have the option to build the raid with 64 bit
lba's, but when I do that, they aren't even recognized I don't think.  But then
I don't think my scsi adapters list the things correctly in the scsi bios
either, so it maybe just be a problem that way.  If I'm just ignorant here in
expecting these 64 bit lba thingys to work with any old u320 adapter then let me
know please.

Oh, and fedora, rhel, either way, I still get the weird messages about my luns
being to big.  Not that it really seems to hurt anything.  It just means I can't
put more than one thing on the same scsi id.
Comment 20 Dave Jones 2006-02-03 02:26:04 EST
This is a mass-update to all currently open kernel bugs.

A new kernel update has been released (Version: 2.6.15-1.1830_FC4)
based upon a new upstream kernel release.

Please retest against this new kernel, as a large number of patches
go into each upstream release, possibly including changes that
may address this problem.

This bug has been placed in NEEDINFO_REPORTER state.
Due to the large volume of inactive bugs in bugzilla, if this bug is
still in this state in two weeks time, it will be closed.

Should this bug still be relevant after this period, the reporter
can reopen the bug at any time. Any other users on the Cc: list
of this bug can request that the bug be reopened by adding a
comment to the bug.

If this bug is a problem preventing you from installing the
release this version is filed against, please see bug 169613.

Thank you.
Comment 21 Glen 2006-02-05 22:32:53 EST
With this latest fc4 kernel, kernel-smp-2.6.15-1.1830_FC4, I'm not seeing the
weird messages about luns anymore.  So that's a good thing.  Unfortunately I'm
not in a position to test out filesystems over 2TB at the moment.  If I get a
chance though I'll definitely report back my findings.
Comment 22 Dave Jones 2006-09-16 23:17:41 EDT
[This comment added as part of a mass-update to all open FC4 kernel bugs]

FC4 has now transitioned to the Fedora legacy project, which will continue to
release security related updates for the kernel.  As this bug is not security
related, it is unlikely to be fixed in an update for FC4, and has been migrated
to FC5.

Please retest with Fedora Core 5.

Thank you.
Comment 23 Dave Jones 2006-10-16 20:32:41 EDT
A new kernel update has been released (Version: 2.6.18-1.2200.fc5)
based upon a new upstream kernel release.

Please retest against this new kernel, as a large number of patches
go into each upstream release, possibly including changes that
may address this problem.

This bug has been placed in NEEDINFO state.
Due to the large volume of inactive bugs in bugzilla, if this bug is
still in this state in two weeks time, it will be closed.

Should this bug still be relevant after this period, the reporter
can reopen the bug at any time. Any other users on the Cc: list
of this bug can request that the bug be reopened by adding a
comment to the bug.

In the last few updates, some users upgrading from FC4->FC5
have reported that installing a kernel update has left their
systems unbootable. If you have been affected by this problem
please check you only have one version of device-mapper & lvm2
installed.  See bug 207474 for further details.

If this bug is a problem preventing you from installing the
release this version is filed against, please see bug 169613.

If this bug has been fixed, but you are now experiencing a different
problem, please file a separate bug for the new problem.

Thank you.
Comment 24 John Thacker 2006-10-30 17:30:20 EST
Closing per lack of response to previous request for information.
This bug was originally filed against a much earlier version of Fedora
Core, and significant changes have taken place since the last version
for which this bug is confirmed.

Note that FC3 and FC4 are supported by Fedora Legacy for security
fixes only.  Please install a still supported version and retest.  If
it still occurs on FC5 or FC6, please reopen and assign to the correct
version.  Otherwise, if this a security issue, please change the
product to Fedora Legacy.  Thanks, and we are sorry that we did not
get to this bug earlier.

Note You need to log in before you can comment on or make changes to this bug.