Bug 103305 - Performance issue with sync on RHEL 2.1
Performance issue with sync on RHEL 2.1
Product: Red Hat Enterprise Linux 2.1
Classification: Red Hat
Component: kernel (Show other bugs)
i686 Linux
medium Severity medium
: ---
: ---
Assigned To: Larry Woodman
Brian Brock
Depends On:
  Show dependency treegraph
Reported: 2003-08-28 14:27 EDT by Heather Conway
Modified: 2007-11-30 17:06 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2003-09-26 11:25:03 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Heather Conway 2003-08-28 14:27:01 EDT
Description of problem:
When using the sync option to mount an ext3 filesystem on a LUN, the 
performance degrades to that of a floppy drive. 
Dell PE2650
RHEL 2.1 updated to v2.4.9-e.24
Qlogic QLA2340
QLogic driver v6.04.01 and BIOS v1.34
PowerPath v3.0.3 b065

Version-Release number of selected component (if applicable):
kernel-enterprise-2.4.9-e.24, kernel-smp-2.4.9-e.24, kernel-smp-2.4.9-e.16

How reproducible:
Every time

Steps to Reproduce:
An example with PowerPath:
- mount /dev/emcpowera1 /share
- rsync -rvP /boot /share => 6574740.86 bytes/sec
- rm -rf /share/boot
- mount -o remount,sync /dev/emcpowera1 /share
- mount => to verify that the sync option is enabled
- rsync -rvP /boot /share => 1122518.24 bytes/sec

An example WITHOUT PowerPath:
- mount /dev/sdd1 /share
- rsync -rvP /boot /share => 9204649.60 bytes/sec
- rm -rf /share/boot
- mount -o remount,sync /dev/sdd1 /share
- mount 
- rsync -rvP /boot /share => 1122518.24 bytes/sec
Actual results:

Expected results:

Additional info:
Comment 1 Heather Conway 2003-09-12 13:48:16 EDT
I ran the test again on a Dell 2600.  The following are the results I got:
With PowerPath:
 - mount /dev/emcpowerc1 /share
- rsync -rvP /boot /share => 1360727.20 bytes/sec
- rm -rf /share/boot
- mount -o remount,sync /dev/emcpowera1 /share
- mount 
- rsync -rvP /boot /share => 427.60 bytes/sec

Without PowerPath:
 - mount /dev/sdd1 /share
- rsync -rvP /boot /share => 140070.29 bytes/sec
- rm -rf /share/boot
- mount -o remount,sync /dev/sdd1 /share
- mount 
- rsync -rvP /boot /share => 155689.67 bytes/sec
Comment 2 Arjan van de Ven 2003-09-26 10:43:49 EDT
Just to be clear, this is the qlogic driver binary as Red Hat ships it, right?
Comment 3 Stephen Tweedie 2003-09-26 11:25:03 EDT
This is not a bug: synchronous IO is slow.

Instead of having IOs submitted in large, contiguous chunks to the disk --- both
to the journal and to the files themselves: mount -o sync forces each 4k write
to flush the data to disk, wait for that to complete, then flush the transaction
to the journal, wait for that to complete, then flush the commit block to the
journal, and wait for _that_ to complete.  That's 2 seeks and 3 pipeline stalls
per IO.

So the basic slowdown without EMC modules is entirely expected.  The huge
slowdown with PowerPath is something that EMC will need to debug: one
observation is that drivers with high throughput but high latencies will be
disproportionately affected by all forms of synchronous IO, so if powerpath is
using any form of deferred command completion then that could explain much of
the performance loss.

Note You need to log in before you can comment on or make changes to this bug.