Bug 457226 - [NetApp 5.3 bug] multipathd fails to start if max_fds option is configured as "unlimited" in multipath.conf with 1024 paths.
Summary: [NetApp 5.3 bug] multipathd fails to start if max_fds option is configured as...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: device-mapper-multipath
Version: 5.2
Hardware: All
OS: Linux
medium
medium
Target Milestone: rc
: ---
Assignee: Ben Marzinski
QA Contact: Cluster QE
URL:
Whiteboard:
Depends On: 251346
Blocks: 373081 RHEL5u3_relnotes 457228
TreeView+ depends on / blocked
 
Reported: 2008-07-30 12:24 UTC by Rajashekhar M A
Modified: 2010-03-14 21:28 UTC (History)
21 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Previously, setting max_fds to "unlimited" in /etc/multipath.conf would prevent the multipathd daemon from starting. If number of open file descriptors needd to be set to the system maximum, max_fds should be set to "max"
Clone Of:
Environment:
Last Closed: 2009-01-20 22:09:07 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
multipath.conf settings (1.29 KB, text/plain)
2008-07-30 12:24 UTC, Rajashekhar M A
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2009:0232 0 normal SHIPPED_LIVE device-mapper-multipath bug-fix and enhancement update 2009-01-20 16:06:34 UTC

Description Rajashekhar M A 2008-07-30 12:24:30 UTC
Description of problem:

On a RHEL2.2 host, I have 256 luns mapped each with 4 FCP paths giving a total
of 1024 paths. The FCP HBA driver on the host properly detects all these paths
i.e. all 1024 entries are present in the sysfs. But when I start multipathd with
the option max_fds set to "unlimited" in multipath.conf file, the daemon fails
to start up:

# /etc/init.d/multipathd start
Starting multipathd daemon:                                [  OK  ]
# /etc/init.d/multipathd status
multipathd dead but pid file exists

I see the following message in /var/log/messages --
multipathd: can't set open fds limit to -1 : Operation not permitted


Version-Release number of selected component (if applicable):

# rpm -qa | grep device
device-mapper-event-1.02.24-1.el5
device-mapper-1.02.24-1.el5
device-mapper-1.02.24-1.el5
device-mapper-multipath-0.4.7-17.el5


How reproducible:

Always


Steps to Reproduce:

1. Map 256 luns to a RHEL5.2 host with 4 FCP paths each thereby giving a total
of 1024 paths.
2. Add the option "max_fds unlimited" in defaults section in multipath.conf file.
3. Configure maps for all the devices, by running "multipath -v3" command.
4. Start the multipathd daemon.

Actual results:

The multipathd daemon gets killed during startup for the above setup.
# /etc/init.d/multipathd status
multipathd dead but pid file exists


Expected results:

The multipathd daemon should start properly, with fd limit set to a possible
maximum value.

Additional info:

1. If max_fds option is configured with a sufficiently high numerical value,
multipathd starts up fine.

2. /etc/multipath.conf settings are attached.

Comment 1 Rajashekhar M A 2008-07-30 12:24:30 UTC
Created attachment 312991 [details]
multipath.conf settings

Comment 2 Rajashekhar M A 2008-07-30 12:26:22 UTC
Sorry for the typo in the description section, its "On a RHEL5.2 host...".

Comment 5 Dave Wysochanski 2008-07-30 16:39:33 UTC
Ben's proposed action from that bug 251346 - remove the "unlimited", provide a
hard upper limit (based on the kernel's limit) and call it "max".

Comment #20 From Ben Marzinski (bmarzins)  	 on 2008-04-22 13:42
EST  	[reply]   	 Private

Unfortunately, unlimited does seem like it will not work do to an in-kernel
limit of 1048576 open file descriptors. However, for all practical purposes,
this limit shouldn't be a big problem.  I'll remove the "unlimited" option in
future releases. It might be best to replace it with a "max" option, that simply
sets the limit to the in-kernel max.

Comment 6 Don Domingo 2008-08-03 22:29:20 UTC
tagging for inclusion in the RHEL5.3 release notes, including as quoted in "Release Notes" field of this bug.

Comment 7 Don Domingo 2008-08-03 22:29:20 UTC
Added New Release Notes Contents.

Comment 8 Ben Marzinski 2008-09-02 22:34:05 UTC
Fixed.  I've changed "unlimited" to "max" which sets the number of fds to the NR_OPEN kernel value.

Comment 10 Andrius Benokraitis 2008-09-03 13:52:50 UTC
Release note updated. If any revisions are required, please set the 
"requires_release_notes"  flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

Diffed Contents:
@@ -1,3 +1 @@
-In /etc/multipath.conf, setting max_fds to unlimited will prevent the multipathd
+In /etc/multipath.conf, setting max_fds to unlimited will prevent the multipathd daemon from starting up properly. As such, you should use a sufficiently high value instead for this setting.-daemon from starting up properly. As such, you should use a sufficiently high
-value instead for this setting.

Comment 11 Ben Marzinski 2008-09-03 17:46:50 UTC
Release note updated. If any revisions are required, please set the 
"requires_release_notes"  flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

Diffed Contents:
@@ -1 +1 @@
-In /etc/multipath.conf, setting max_fds to unlimited will prevent the multipathd daemon from starting up properly. As such, you should use a sufficiently high value instead for this setting.+In /etc/multipath.conf, "umlimited" is no longer an appropriate value for the "max_fds" option. Instead, you should use the value "max" which sets the number of open file descriptors to the system maximum.

Comment 13 Ryan Lerch 2008-09-04 00:37:28 UTC
Release note updated. If any revisions are required, please set the 
"requires_release_notes"  flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

Diffed Contents:
@@ -1 +1,3 @@
-In /etc/multipath.conf, "umlimited" is no longer an appropriate value for the "max_fds" option. Instead, you should use the value "max" which sets the number of open file descriptors to the system maximum.+In /etc/multipath.conf, "umlimited" is no longer an appropriate value for the "max_fds" option. Instead, you should use the value "max" which sets the number of open file descriptors to the system maximum.
+
+Previously, setting max_fds to unlimited in /etc/multipath.conf would prevent the multipathd daemon from starting.

Comment 14 Ben Marzinski 2008-09-04 20:46:53 UTC
In response to comment #12, multipathd does now start up with over 1K LUNs, so 251346 is no longer an issue.  This bug (457226) is now resolved.  All the customers need to know is that they can use "max" now to do what "unlimited" was supposed to do.

The updated release note in comment #13 looks great.

Comment 16 Ryan Lerch 2008-10-24 00:07:46 UTC
Release note updated. If any revisions are required, please set the 
"requires_release_notes"  flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

Diffed Contents:
@@ -1,3 +1 @@
-In /etc/multipath.conf, "umlimited" is no longer an appropriate value for the "max_fds" option. Instead, you should use the value "max" which sets the number of open file descriptors to the system maximum.
+Previously, setting max_fds to "unlimited" in /etc/multipath.conf would prevent the multipathd daemon from starting. If number of open file descriptors needd to be set to the system maximum,  max_fds should be set to "max"-
-Previously, setting max_fds to unlimited in /etc/multipath.conf would prevent the multipathd daemon from starting.

Comment 17 Naveen Reddy 2008-10-30 16:23:33 UTC
I have verified this by setting the max_fds option to "max" in RHEL5U3 Beta, but still the multipathd goes dead when I start the multipathd with 1024 paths.

# /etc/init.d/multipathd status
multipathd dead but pid file exists

If a change the max_fds to some high value (say 4096), multipathd starts correctly.

Comment 18 Ben Marzinski 2008-11-04 15:50:14 UTC
This is fixed. I'm rebuilding a package now.

Comment 20 Naveen Reddy 2008-11-14 06:35:43 UTC
I have verified it in the RHEL5.3 GASnapshot2 (device-mapper-multipath-0.4.7-22.el5). This issue is fixed.

Comment 23 errata-xmlrpc 2009-01-20 22:09:07 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2009-0232.html


Note You need to log in before you can comment on or make changes to this bug.