Description of problem:
Whilst porting an in-house application, which uses libcurl, from RHEL6 to RHEL7 we noticed that the time taken by libcurl to fetch a page had substantially increased. Even an attempt at fetching a non-existent page has increased from 10-20ms to around 150ms. This with just plain HTTP.
Our first thought was that the API has changed in some subtle way and the fault must be in our code, but then we thought to try measuring the performance of the curl tool itself. To our surprise, we found the same performance disparity.
* Adding the DNS address of the HTTP server to /etc/hosts (and specifying files,dns in /etc/nsswitch.conf) does not improve things
* but, performance when using an IP address in the URL is fine (~ 10-20ms) for both RHEL6 and RHEL7
... which kind of suggests that the delay is in libcurl's DNS code rather than in the upstream resolver.
We did notice that when using DNS names in the URL, curl forks off a process to do the name lookup and then poll()s. There's no such fork when using an IP address in the URL. The poll() timesout after a period of time - we suspect this is the cause of the delay.
Version-Release number of selected component (if applicable):
Use the "curl" command to fetch a non-existent page off a working HTTP server.
Steps to Reproduce:
1. Setup an HTTP server, say myserver.mydomain.org
2. time curl http://myserver.mydomain.org/non-existent-page.html
Running curl on an RHEL 6.4 box, the time taken by curl is ~10-20ms.
Running curl on an RHEL 7.0 box, the time taken by curl is ~150ms.
Time taken should be same under both OS releases.
This "bug" was reported to the Curl mailing list, and Daniel Stenberg suggested "this is simply because Redhat switched their build to the threaded resolver and in 7.29.0 we switched some libcurl internals which made us have that performance issue. I think this problem is solved in a more recent libcurl version."
Thank you for reporting the bug here, Alastair!
This was originally reported at the upstream mailing-list:
I was able to reproduce the delay on my RHEl-7 box by resolving localhost:
$ time curl http://localhost -v --trace-time
20:37:13.240666 * About to connect() to localhost port 80 (#0)
20:37:13.240969 * Trying ::1...
20:37:13.241285 * Connection refused
20:37:13.241477 * Trying 127.0.0.1...
20:37:13.241648 * Connection refused
20:37:13.241755 * Failed connect to localhost:80; Connection refused
20:37:13.241851 * Closing connection 0
curl: (7) Failed connect to localhost:80; Connection refused
It seems to be caused by the following upstream patch:
... that was applied on Fedora 19:
The following upstream commit seems to fix it:
fixed in curl-7.29.0-22.fc19
While testing the package, Stefan Kremen discovered that the fixed version of RHEL-7 libcurl is still _slightly_ slower in resolving hosts from /etc/hosts, compared to RHEL-6 libcurl.
Unfortunately, this behavior is expected and nearly impossible to fix. The RHEL-6 version of libcurl uses the blocking interface of the system resolver directly whereas the RHEL-7 version of libcurl uses threads to implement a non-blocking interface on top of the blocking interface of the system resolver. Since the system resolver provides only blocking API, polling is necessary.
On the other hand, single-threaded applications are no longer blocked during name resolving in RHEL-7. Moreover, it is no longer needed to use setjmp/longjmp to implement timeouts, which is known to occasionally crash the python interpreter in RHEL-6.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.