Red Hat Bugzilla – Bug 134794
curl issues with select vs FD_SETSIZE
Last modified: 2013-07-02 19:02:19 EDT
Description of problem: curl does not work if file descriptor numbers are pushed above FD_SETSIZE. This happens, for example, in Apache configurations where a large number of log files are configured, typical of large-scale virtual hosting scenarios. Version-Release number of selected component (if applicable): curl-7.10.6-4.1 (or any version) How reproducible: Always Steps to Reproduce: The "bumpfd" program just bumps the rlimits and opens N files to reproduce the issue: 0. become root 1. curl http://cvs.apache.org/~jorton/bumpfd.c > bumpfd.c 2. gcc -o bumpfd bumpfd.c 3. ./bumpfd 2048 curl http://www.apache.org/ Actual results: Segmentation fault Expected results: Web page Additional info: This bug appears to be known to upstream but not being actively worked on, and is a deep issue since fd_set * is exposed in the curl API.
Known: yes Worked on: no Known portable fix: no Appreciate patch: yes
The "portable fix" is to use poll() where available and select() where not.
1. What about those systems without poll()? 2. libcurl has APIs that provide fd_set data, those are not easily fixed with simply switching to poll.
Yes, as I mentioned fixing this really requires changing the curl API to not expose the fd_set pointers directly, maybe doing so only through some abstract type. e.g. curl_multi_fdsets(CURLM *mh, int *read_fds, *write_fds, *except_fds) where each int * array is a list of fd numbers terminated by a "-1", that is easy enough for the caller to translate into a struct pollfd array or an fd_set as desired. Or alternatively curl could expose the existing curl_multi_fdset as-is IFF compiled to use select() i.e. on Windows, and it could expose a struct poll * through a new curl_multi_pollfds() function IFF compiled to use poll().
This issue is now fixed in the curl CVS, to be included in the upcoming 7.12.3 release.
Great news, Daniel!
Product Management has reviewed and declined this request. You may appeal this decision by reopening this request.