Bug 240523 - Big problem with raw device binding with EMC powerpath devices, on dual core server Dell PowerEdge 2900, for Oracle RAC database server
Summary: Big problem with raw device binding with EMC powerpath devices, on dual core ...
Alias: None
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: initscripts   
(Show other bugs)
Version: 4.3
Hardware: i686
OS: Linux
Target Milestone: ---
: ---
Assignee: Bill Nottingham
QA Contact: Brock Organ
Depends On:
TreeView+ depends on / blocked
Reported: 2007-05-18 05:18 UTC by Hai Wu
Modified: 2014-03-17 03:06 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2008-12-08 21:15:32 UTC
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

Description Hai Wu 2007-05-18 05:18:51 UTC
Description of problem:
Some random raw devices could not be bound to emcpower devices in boot-up
process per setting in '/etc/sysconfig/rawdevices' file, it always complains
that emcpower block device file can not be found. Manually run
'/etc/init.d/rawdevices start' will always work, after server boots up.

Version-Release number of selected component (if applicable):
EMC PowerPath 4.5.1 for Linux

How reproducible:

Steps to Reproduce:
1. Install EMC PowerPath 4.5.1 for Linux on Dell PowerEdge 2900 (DUAL CORE
server), using qlogic card. Have 4-path for each LUN (The same issue is not
happening to 2-path LUN).
2. put the following (similar) in /etc/sysconfigure/rawdevices file:
/dev/raw/raw1 /dev/emcpowerk1
/dev/raw/raw2 /dev/emcpowerj1
/dev/raw/raw3 /dev/emcpowera1
/dev/raw/raw4 /dev/emcpowern1
/dev/raw/raw5 /dev/emcpowero1
/dev/raw/raw6 /dev/emcpowerp1
/dev/raw/raw7 /dev/emcpowerq1
/dev/raw/raw8 /dev/emcpowerr1
/dev/raw/raw9 /dev/emcpowert1
/dev/raw/raw10 /dev/emcpowerl1
/dev/raw/raw11 /dev/emcpowerm1
/dev/raw/raw12 /dev/emcpowers1
/dev/raw/raw13 /dev/emcpoweru1
/dev/raw/raw14 /dev/emcpowerb1
/dev/raw/raw15 /dev/emcpowere1
/dev/raw/raw16 /dev/emcpowerf1
/dev/raw/raw17 /dev/emcpowerg1
/dev/raw/raw18 /dev/emcpowerh1
/dev/raw/raw19 /dev/emcpowerc1
/dev/raw/raw20 /dev/emcpowerd1
/dev/raw/raw21 /dev/emcpoweri1

Reboot the system.
Actual results:
Most of the time, it would complain about one EMC device file could not be
found, for example, /dev/emcpowerd1 could not be found. Sometimes, it complains
about 2 EMC device files could not be found. In some rare cases, it would work fine.

Expected results:
It should always work.

Additional info:
I have to change file /etc/init.d/rawdevices to add one line:
change from:
       raw $RAW $BLOCK
change to:
       sleep 1
       raw $RAW $BLOCK

Does it sound like an EMC issue or RHEL supporting issue with dual core?


Comment 1 Bill Nottingham 2007-05-18 06:01:59 UTC
Seems like an issue in how long it takes EMC to become available after it has

Comment 2 Hai Wu 2007-05-18 06:15:28 UTC
EMC powerpath inserts 3/4 lines into file /etc/rc.sysinit, to call
/etc/init.d/PowerPath start, I tried to put 5 seconds delay after that, it
didn't help. I went into the script /etc/init.d/PowerPath, and add some delay,
that didn't help.

The only place that it helps is this /etc/sysconfig/rawdevices file. I tried to
just sleep 0.2 seconds, and that also helps, I tested that for a few times, and
it worked fine.

It is a timing issue, but I don't know who needs to fix this. If we don't use
rawdevices, EMC powerpath always works fine. It is the interoperation between
these two that created this problem. It could also due to udev did not create
emcpower device in time for raw command to take that as its input.

Comment 3 Hai Wu 2007-05-18 06:25:11 UTC
Just tried to put 60 seconds delay after /etc/init.d/PowerPath start in the boot
up, and that didn't help, it complained that 2 emcpower device files were not found.

Comment 4 Bill Nottingham 2007-05-18 06:42:17 UTC
Is udev actually what's creating the emc devices ,or is that done by the
PowerPath script?

Comment 5 Hai Wu 2007-05-18 07:53:30 UTC
I am not sure, but in that script, it has some comments:
    #  Wait for udev to finish creating emcpower devices
    for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15; do

From this script, I can' tell how emc asks udev to create emcpower devices.

Comment 6 Bill Nottingham 2007-05-18 08:11:16 UTC
Hm, perhaps that script should run 'udevsettle' or similar to wait for events to

Comment 7 Bill Nottingham 2007-05-18 08:11:41 UTC
Actually, udevsettle probably postdates RHEL 4 - never mind.

Comment 8 Andy Speagle 2007-06-01 22:05:09 UTC
I was able to make it work by adding a 15 second sleep before 
the "pp_start_lvm" call in the "configure_pp" function in /etc/init.d/PowerPath:

--- snip

sleep 15

--- snip

This seemed to allow enough time for things to settle down on my system.  Of 
course, while this helps with my rawdevice startup bindings; I'm still having 
problems with LVM not finding the emcpower devices.... *grumble* ... it's 
definitely a timing issue that EMC needs to work on.  

I don't like having to hack /etc/rc.sysinit and other files to make a system 
wait for emcpower devices to show up.

I'm going to make some modifications to the ugly /etc/init.d/PowerPath script 
to make this work consistently.  Perhaps I can convince EMC to fix their 

Comment 9 Bill Nottingham 2008-12-08 21:15:32 UTC
Given that this seems to be solely an issue with EMC powerpath itself, I don't see anything we can fix in the initscripts themselves. Therefore, closing. (Apologies for the delay.)

Note You need to log in before you can comment on or make changes to this bug.