Bug 134794 - curl issues with select vs FD_SETSIZE
curl issues with select vs FD_SETSIZE
Product: Red Hat Enterprise Linux 3
Classification: Red Hat
Component: curl (Show other bugs)
All Linux
medium Severity high
: ---
: ---
Assigned To: Jindrich Novy
Depends On:
Blocks: 201599
  Show dependency treegraph
Reported: 2004-10-06 08:58 EDT by Joe Orton
Modified: 2013-07-02 19:02 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2006-12-16 13:40:35 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Joe Orton 2004-10-06 08:58:45 EDT
Description of problem:
curl does not work if file descriptor numbers are pushed above
FD_SETSIZE.  This happens, for example, in Apache configurations where
a large number of log files are configured, typical of large-scale
virtual hosting scenarios.

Version-Release number of selected component (if applicable):
curl-7.10.6-4.1 (or any version)

How reproducible:

Steps to Reproduce:
The "bumpfd" program just bumps the rlimits and opens N files to
reproduce the issue:

0. become root
1. curl http://cvs.apache.org/~jorton/bumpfd.c > bumpfd.c
2. gcc -o bumpfd bumpfd.c
3. ./bumpfd 2048 curl http://www.apache.org/

Actual results:
Segmentation fault

Expected results:
Web page

Additional info:
This bug appears to be known to upstream but not being actively worked
on, and is a deep issue since fd_set * is exposed in the curl API.
Comment 1 Daniel Stenberg 2004-10-10 07:15:40 EDT
Known: yes

Worked on: no

Known portable fix: no

Appreciate patch: yes
Comment 2 Joe Orton 2004-10-10 13:15:05 EDT
The "portable fix" is to use poll() where available and select() where
Comment 3 Daniel Stenberg 2004-11-11 09:36:26 EST
1. What about those systems without poll()?

2. libcurl has APIs that provide fd_set data, those are not easily
fixed with simply switching to poll.
Comment 4 Joe Orton 2004-11-11 09:56:07 EST
Yes, as I mentioned fixing this really requires changing the curl API
to not expose the fd_set pointers directly, maybe doing so only
through some abstract type. e.g.

curl_multi_fdsets(CURLM *mh, int *read_fds, *write_fds, *except_fds)

where each int * array is a list of fd numbers terminated by a "-1",
that is easy enough for the caller to translate into a struct pollfd
array or an fd_set as desired.

Or alternatively curl could expose the existing curl_multi_fdset as-is
IFF compiled to use select() i.e. on Windows, and it could expose a
struct poll * through a new curl_multi_pollfds() function IFF compiled
to use poll().
Comment 7 Daniel Stenberg 2004-11-20 04:45:05 EST
This issue is now fixed in the curl CVS, to be included in the
upcoming 7.12.3 release.
Comment 8 Joe Orton 2004-12-13 09:26:07 EST
Great news, Daniel!
Comment 12 RHEL Product and Program Management 2006-12-16 13:40:36 EST
Product Management has reviewed and declined this request.  You may appeal this
decision by reopening this request. 

Note You need to log in before you can comment on or make changes to this bug.