Bug 439391 - Extremely poor disk performance on HP dc7800 desktop PC (Intel Q35 Express chipset)
Extremely poor disk performance on HP dc7800 desktop PC (Intel Q35 Express ch...
Status: CLOSED WONTFIX
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: kernel (Show other bugs)
5.2
x86_64 Linux
low Severity high
: rc
: ---
Assigned To: Jeff Burrell
Red Hat Kernel QE team
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-03-28 10:19 EDT by Ole Holm Nielsen
Modified: 2013-11-03 20:59 EST (History)
10 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-11-03 20:59:45 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Ole Holm Nielsen 2008-03-28 10:19:20 EDT
Description of problem:
The built-in SATA hard disk of the HP dc7800 desktop PC (with Intel Q35 Express
chipset) is incorrectly configured, and disk I/O performance is more than 10
times lower than it should be. Users experience a very slow system. 
We test I/O performance by writing a large file to disk:
# time dd if=/dev/zero of=temp bs=1024k count=5000
  5242880000 bytes (5.2 GB) copied, 978.086 seconds, 5.4 MB/s
Another symptom of the problem is that DMA is turned off (and cannot be turned on):
# hdparm  -d /dev/hda
  using_dma    =  0 (off)
The fact that the disk is denoted as /dev/hda in stead of the expected /dev/sda
is yet another symptom of the problem.

We have installed Fedora Core 9 on the same hardware. FC9 detects the hard disk
as /dev/sda, and writing a big file as above gives a disk I/O speed of 55-60
MB/sec. We have also built and installed the kernel.org Linux kernel version
2.4.26.2 on the RHEL 5.2 system, and this kernel fixes the problem as well.

Conclusion: The RHEL kernel 2.6.18-84.el5 (and older) doesn't support the Intel
Q35 Express chipset correctly.  The solution would seem to be backporting the
support of Intel Q35 Express from later Linux kernels.

Version-Release number of selected component (if applicable):
RHEL 5.2 Beta (and any previous version)

How reproducible:
Always

Steps to Reproduce:
1. Install RHEL onto HP dc7800 desktop PC
2. Test hard disk I/O performance
3.
  
Actual results:
I/O performance is about 5 MB/sec. SATA disk device is /dev/hda.

Expected results:
I/O performance should be above 50 MB/sec. SATA disk device is /dev/sda.

Additional info:
The HP dc7800 desktop PC is Certified and Supported by RedHat, see
https://hardware.redhat.com/show.cgi?id=427817
Workaround: Install newer Linux kernel or Fedora Core.
Comment 1 Tony Camuso 2008-03-29 06:37:03 EDT
Ole, 

Can you boot the newer kernels without using pci=conf1 ?


If so, then if you boot the newer kernels with pci=conf1, does the SD get listed
as HD and performance get trashed?

If so, then this may be a manifestation of the MMCONF problem, where extended
PCI space is not available to device drivers. 

Comment 2 Ole Holm Nielsen 2008-03-31 05:54:35 EDT
Firstly, I was advised in Bug 439387 to use the kernel parameter pci=nommconf in
stead of pci=conf1.  Either of these boot flags seem to work well on the dc7800
hardware.

I tested booting the dc7800 with RHEL 5.2 Beta using the kernel.org Linux kernel
2.4.26.2 *without* pci=nommconf. In this case the boot freezes after a couple of
seconds just after the "Booting the kernel" message. Apparently the kernel.org
people didn't provide boot support for the Intel Q35 Express chipset (or it
needs a special kernel config).

When using the pci=nommconf flag, disk I/O performance is great with kernel
2.4.26.2 (55-60 MBytes/sec), and it's still abysmal with the RHEL 5.2 kernel (5
MBytes/sec).
Comment 3 Geoff Gustafson 2008-03-31 14:27:57 EDT
Make sure you have SATA mode in the BIOS set to AHCI or Native/Enhanced mode,
and not Legacy/Compatible mode.
Comment 4 Ole Holm Nielsen 2008-03-31 16:09:27 EDT
This dc7800 PC has default BIOS settings which I believe is AHCI (will check it).
But if BIOS settings was the problem, how do you explain that kernel 2.4.26.2 is
more than 10 times faster that the RHEL 5.2 kernel ?  We need to focus on the
fact that newer kernels seem to handle disk I/O on the Intel Q35 Express chipset
very well, in contrast to the RHEL kernels.
Comment 5 Ole Holm Nielsen 2008-04-01 03:45:37 EDT
I checked the HP dc7800 BIOS setup now: The default settings are used in my
case, and in the notation that the HP BIOS uses the following settings are made:

SATA Defaults: Multisector transfers: 16
      Transfer mode: Max UDMA
      Translation mode: Automatic
SATA0 160 GB SATA hard disk:
      Emulation type: hard disk
      Multisector transfers: 16
      Translation mode: Automatic

Even though AHCI isn't mentioned anywhere in the BIOS, I trust that HP has
implemented the maximum performance settings for the SATA disks.
Comment 6 Bryan Christ 2008-04-01 10:08:27 EDT
Perhaps I am missing something.  According to comment #2, the system operates at
native speeds with pci=nommconf.  If this is the case, then this really isn't a
bug at all just another instance of the same issue (nommconf) that Tony has been
working for some time now.
Comment 7 Bryan Christ 2008-04-01 10:11:50 EDT
Reply to comment#5... You can enable AHCI by putting the system in "RAID" mode
(through F10 system).  Don't worry, it's fake-raid and you can just use it like
a normal SATA controller in this mode.  This effectively exposes AHCI for the
Linux kernel.  I should also mention that "RAID" mode is not available on the
dc7800 ultra-slim model (only the Convertible Minitower and Small Form Factor).
Comment 8 Ole Holm Nielsen 2008-04-01 15:21:32 EDT
Bryan, the system operates at native speed ONLY with kernel 2.6.24.2 from
kernel.org, NOT with any RHEL kernels ! This is the bug that I'm trying to get
RedHat to take a look at. Apparently I must try to make myself completely clear
regarding the status of RHEL kernels on the HP dc7800:
 * RHEL 5.x kernel = Bad I/O performance :-(
 * Kernel 2.6.24.2 = Excellent I/O performance :-)
Any clarifications needed regarding this statement ?

Setting the BIOS to SATA Emulation=RAID is possible, but then the boot
terminates with a Kernel Panic after a few seconds.  And why even try such a
non-default option when it's known that kernel 2.6.24.2 does things correctly ?
Comment 9 Ole Holm Nielsen 2008-05-30 10:49:51 EDT
I'm reconfirming the above serious disk performance problem with the new RedHat
Desktop 5.2. Disk I/O performance is still an abysmal 5 MB/sec :-(

We have been forced to install a later kernel from kernel.org on our HP dc7800
Linux PCs because of this problem.

Hopefully support of the Intel Q35 Express chipset can be backported to RedHat
5.2 from later kernels.
Comment 10 Mogens Kjaer 2008-07-07 08:32:20 EDT
Boot with:

pci=nomsi,nommconf hda=noprobe hdc=noprobe
Comment 11 Ole Holm Nielsen 2008-07-07 08:57:44 EDT
(In reply to comment #10)
> Boot with:
> 
> pci=nomsi,nommconf hda=noprobe hdc=noprobe
> 
PROBLEM SOLVED: With these parameters I now get the expected I/O performance of
about 70 MB/s in stead of 5 MB/s:

# time dd if=/dev/zero of=temp bs=1024k count=5000
5000+0 records in
5000+0 records out
5242880000 bytes (5.2 GB) copied, 67.7216 seconds, 77.4 MB/s

Also, the disk is correctly recognized as /dev/sda in stead of /dev/hda.

This workaround should definitely be added to the hardware support pages at 
https://hardware.redhat.com/list.cgi?product=Red+Hat+Hardware+Certification&quicksearch=dc7800
and for any other systems using the Intel Q35 Express chipset.
Comment 12 Andreas Mohr 2009-11-16 08:06:51 EST
Same thing for DC7700.

I'm afraid it's not too helpful that when trying to rectify this unacceptable performance by going the way of custom-building a mainline kernel (something I'm doing _all the time_ on other installations), one then runs into another tarpit as mentioned in "Kernel Panic + Intel SATA": http://lkml.indiana.edu/hypermail/linux/kernel/0510.3/0563.html , caused by weak device name support in initrd (nash mkrootdev issue, no root fs found no matter how one tries to correct it, with hundreds of people stumped in various Google results on RHEL5, FC6 etc.).

Given these issues with medieval SATA support RHEL5 appears to be way _too_ stable (probably even Debian stable with its infamous up-to-dateness is quite a bit more modern than this, as shocking as this statement may be). Buying further RHEL subscriptions has become less likely at this place... (supporting further Linux development is a very nice thing in theory, but once it actually starts hampering local productivity on _multiple_ occasions, certain reconsiderations do take place). Or, IOW, when actively paying for things there are specific expectations of at least equal behaviour as compared to certain other solutions.
And I just don't see this, both in the restrictive behaviour of package feed management and in general runtime behaviour.

Time to rethink the overall development / marketing model, maybe?
Comment 13 Tony Camuso 2009-11-16 09:39:01 EST
Please assign this bug to bryan.christ@hp.com of HP desktop systems.
Comment 14 Tony Camuso 2010-09-21 11:24:23 EDT
Pleasse assign this bug to Jeff.Burrell@hp.com
Comment 15 Robert Townley 2010-10-31 22:27:40 EDT
(In reply to comment #12)
> 
> Time to rethink the overall development / marketing model, maybe?

Our dc7800's came with WinVista and were the slowest new machines i ever used running Windows Vista.
Comment 16 Robert Townley 2010-10-31 22:36:10 EDT
Our dc7800's came with WinVista and were the slowest new machines i ever used.  Slower than our much older 2.4GhZ Pentium IVs.
Sometimes i seriously wonder if vPro is part of the problem.
Comment 17 Edouard Bourguignon 2010-11-30 10:34:24 EST
Same problem here with JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 02) on kernel 2.6.18-194.26.1.el5

default boot options:
dd if=/dev/zero of=./test.img bs=1M count=1024
1024+0 enregistrements lus
1024+0 enregistrements écrits
1073741824 octets (1,1 GB) copiés, 173,375 seconde, 6,2 MB/s

"pci=nomsi,nommconf hda=noprobe hdc=noprobe" boot options:
dd if=/dev/zero of=/tmp/test.img bs=1M count=1024
1024+0 enregistrements lus
1024+0 enregistrements écrits
1073741824 octets (1,1 GB) copiés, 9,31306 seconde, 115 MB/s

Motherboard is a recent Gigabyte GA-D510UD. Disks on Intel Corporation N10/ICH7 Family SATA IDE Controller (rev 02) works fine even with default boot parameters.

Any idea when this will be fixed? No problem on Fedora13.
Comment 18 syed azam 2010-11-30 12:45:15 EST
check in the HP DC7800 BIOS that storage option is native AHCI.
Comment 19 syed azam 2010-11-30 12:45:16 EST
check in the HP DC7800 BIOS that storage option is native AHCI.
Comment 20 Edouard Bourguignon 2011-01-07 07:49:25 EST
Since BIOS version F4 for the Gigabyte GA-D510UD I don't have the problem anymore. Can boot without any parameter, disk performance is ok.
Comment 21 John Feeney 2013-11-03 20:59:45 EST
This Bugzilla has been reviewed by Red Hat and is not planned on being
addressed in Red Hat Enterprise Linux 5, and therefore is being closed.
If this bug is critical to production systems, please contact your Red
Hat support representative and provide a sufficient business justification
in order to re-open it.

Note You need to log in before you can comment on or make changes to this bug.