Red Hat Bugzilla – Bug 134794
curl issues with select vs FD_SETSIZE
Last modified: 2013-07-02 19:02:19 EDT
Description of problem:
curl does not work if file descriptor numbers are pushed above
FD_SETSIZE. This happens, for example, in Apache configurations where
a large number of log files are configured, typical of large-scale
virtual hosting scenarios.
Version-Release number of selected component (if applicable):
curl-7.10.6-4.1 (or any version)
Steps to Reproduce:
The "bumpfd" program just bumps the rlimits and opens N files to
reproduce the issue:
0. become root
1. curl http://cvs.apache.org/~jorton/bumpfd.c > bumpfd.c
2. gcc -o bumpfd bumpfd.c
3. ./bumpfd 2048 curl http://www.apache.org/
This bug appears to be known to upstream but not being actively worked
on, and is a deep issue since fd_set * is exposed in the curl API.
Worked on: no
Known portable fix: no
Appreciate patch: yes
The "portable fix" is to use poll() where available and select() where
1. What about those systems without poll()?
2. libcurl has APIs that provide fd_set data, those are not easily
fixed with simply switching to poll.
Yes, as I mentioned fixing this really requires changing the curl API
to not expose the fd_set pointers directly, maybe doing so only
through some abstract type. e.g.
curl_multi_fdsets(CURLM *mh, int *read_fds, *write_fds, *except_fds)
where each int * array is a list of fd numbers terminated by a "-1",
that is easy enough for the caller to translate into a struct pollfd
array or an fd_set as desired.
Or alternatively curl could expose the existing curl_multi_fdset as-is
IFF compiled to use select() i.e. on Windows, and it could expose a
struct poll * through a new curl_multi_pollfds() function IFF compiled
to use poll().
This issue is now fixed in the curl CVS, to be included in the
upcoming 7.12.3 release.
Great news, Daniel!
Product Management has reviewed and declined this request. You may appeal this
decision by reopening this request.