Bug 806405 - max_hw_sectors_kb not there in RHS2
max_hw_sectors_kb not there in RHS2
Status: CLOSED CURRENTRELEASE
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: unclassified (Show other bugs)
2.0
Unspecified Unspecified
medium Severity medium
: Release Candidate
: RHGS 2.0.0
Assigned To: Anthony Towns
M S Vishwanath Bhat
:
Depends On:
Blocks: 817967
  Show dependency treegraph
 
Reported: 2012-03-23 12:13 EDT by Ben England
Modified: 2016-05-31 21:55 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-11-13 01:09:02 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Ben England 2012-03-23 12:13:49 EDT
Description of problem:

We need to tune the block device parameter max_hw_sectors_kb, which is important getting the I/O request size high enough to fully exploit the hardware RAID controller and many drives behind it.

Specifically, we need to raise max_hw_sectors_kb, which requires /opt/modprobe.d/megaraid_sas.conf containing:

options megaraid_sas max_sectors=2048

then rebuild initramfs as follows

# cd boot
# dracut initramfs-2.6.32-220.7.1.el6.x86_64.img.new 2.6.32-220.7.1.el6.x86_64
# mv initramfs-2.6.32-220.7.1.el6.x86_64.im{g,g.orig}
# initramfs-2.6.32-220.7.1.el6.x86_64.im{g.new,g}
# reboot

I think anaconda script should be able to do this, particularly the initramfs generation, or maybe the .iso could be set up with the right initramfs to begin with.

Version-Release number of selected component (if applicable):

RHS 2 beta

How reproducible:

always
Comment 1 Anthony Towns 2012-04-04 04:03:53 EDT
I think to do this we just want to populate /etc/modprobe.d/megaraid_sas.conf in the %pre part of the kickstart; the initramfs should then get generated correctly when the kernel is installed.
Comment 2 Ben England 2012-04-09 21:29:05 EDT
I'm no anaconda expert, I was just trying to describe the end result,  there may well be a much simpler way to get the same result, /sys/block/sdb/queue/max_sectors_kb > 128.
Comment 4 Anthony Towns 2012-05-24 19:00:24 EDT
megaraid_sas.conf added to /etc/modprobe.d via appliance-base as of build RHS-2.0-20120524.n.0-RHS-x86_64-DVD1.is
Comment 5 Sachin Ghai 2012-06-01 02:41:23 EDT
verified with following rpms by updating RHS-2.0-20120524.n.0-RHS-x86_64-DVD1.iso

glusterfs-fuse-3.3.0qa45-1.el6.x86_64
glusterfs-3.3.0qa45-1.el6.x86_64
glusterfs-server-3.3.0qa45-1.el6.x86_64
glusterfs-rdma-3.3.0qa45-1.el6.x86_64
org.apache.hadoop.fs.glusterfs-glusterfs-0.20.2_0.1-1.noarch
glusterfs-geo-replication-3.3.0qa45-1.el6.x86_64

megaraid_sas.conf is added to /etc/modprobe.d and having :
options megaraid_sas max_sectors=2048

---
[root@dhcp201-154 run]# cat /etc/modprobe.d/megaraid_sas.conf
options megaraid_sas max_sectors=2048
[root@dhcp201-154 run]# cd 
[root@dhcp201-154 ~]# cd /etc/modprobe.d/
[root@dhcp201-154 modprobe.d]# ls
anaconda.conf   dist-alsa.conf  dist-oss.conf
blacklist.conf  dist.conf       megaraid_sas.conf
[root@dhcp201-154 modprobe.d]# ls | grep megaraid_sas.conf 
megaraid_sas.conf
[root@dhcp201-154 modprobe.d]# 
--
Comment 6 Ben England 2012-06-04 15:21:10 EDT
I did not see this problem anymore in RHS-2.0-20120530-something.iso, I think you can close this bug
Comment 7 Ben England 2013-04-07 14:27:03 EDT
When I checked this for Anshi build dated March 20, I did not see /etc/modprobe.d/megaraid_sas.conf in place, and the max_hw_sectors_kb was 280 on gprfs041.sbu.lab.eng.bos.redhat.com.    I tried to fix using same procedure as before, adding /etc/modprobe.d/megaraid_sas.conf with line

options megaraid_sas max_sectors=2048

and re-running dracut utility to remake /boot/initramfs for my kernel.

I then saw this error from dmesg:

megasas: INIT adapter done
megasas: max_sectors should be > 0and <= 560 (or < 1MB for GEN2 controller)

So what I'm saying is that we may want to remove this optimization which worked for older megaraid controllers but apparently will not work for newer ones.    This is disappointing because at 1 GB/sec throughput, 280 KB = 280 usec, not much time for Linux to process an I/O request.

max_sectors_kb was set to max_hw_sectors_kb, which is the maximum that this driver will support, it appears.

Note You need to log in before you can comment on or make changes to this bug.