Bug 1572438
| Summary: | libcurl w/ ssl (nss) consumes memory | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Marcus Watts <mwatts> |
| Component: | nss-pem | Assignee: | Kamil Dudka <kdudka> |
| Status: | CLOSED WONTFIX | QA Contact: | BaseOS QE Security Team <qe-baseos-security> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 7.7-Alt | CC: | fsumsal, kdudka, mwatts, rmarigny, vumrao |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-11-19 13:38:56 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1643104 | ||
| Attachments: | |||
Created attachment 1427444 [details]
massif output - rhel7 w/ libcurl-7.29.0-47.el7.x86_64 + nss-3.34.0-4.el7.x86_64
This is the full output I got using massif with rhel7, libcurl+nss. This output includes the memory ramp described, plus much more detailed accountings of memory usage at various points during execution. You have to install the appropriate debug information to get all this detail.
Created attachment 1427446 [details]
massif output: xenial w/ curl+openssl.
xenial ships with 3 different versions of libcurl. Each
is linked with a different soname so runtimes for all 3
can be loaded at the same time. Which version is used is selected at at compile time for the application by which version of the dev library is present.
This run:
libcurl3:amd64: 7.47.0-1ubuntu2.7
libssl1.0.0:amd64: 1.0.2g-1ubuntu4.12
This shows the flat memory usage I expect to see.
Created attachment 1427462 [details]
massif output: xenial w/ curl+gnutls.
This is output from xenial with curl+gnutls. This is only included to document that this combination is less desirable.
libcurl3-gnutls:amd64 7.47.0-1ubuntu2.7
libgnutls30:amd64 3.4.10-4ubuntu1.4
This has flat bounded output. However, the peaks are far larger than for openssl. I haven't bothered to document execution time for this bug report, but execution time is very noticeably slower with gnutls.
I'm not going to bother with the xenial libcurl+nss combination, because that doesn't look for ca certs the same way as gnutls or openssl and doesn't have a good workflow for adding new ca certs.
Thank you for providing the reproducer! I can confirm that libcurl-7.29.0-47.el7 consumes less memory than libcurl-7.29.0-46.el7. The remaining memory waste is not in libcurl but in nss-pem. It is a known issue.
nss-pem is used to load CA certificates from PEM files. The memory consumption grows per CA cert (re)load (and they are reloaded per each connection). Hence you can significantly reduce the memory consumption by giving libcurl fewer CA certificates from PEM file, or even make the memory consumption constant by not loading CA certificates from PEM file at all.
Could something like this be used as a workaround?
--- a/curl5.cc
+++ b/curl5.cc
@@ -273,7 +273,7 @@ doit(int id, struct doit_stats *ds)
if (Vflag) {
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
}
- if (capath)
+ //if (capath)
curl_easy_setopt(curl, CURLOPT_CAINFO, capath);
if (clock_gettime(CLOCK_MONOTONIC_RAW, ts) < 0) {
std::cerr << "gettime failed " << errno << std::endl;
$ c++ -g -o curl5 curl5.cc -lcurl -lpthread -std=c++11
$ mkdir $HOME/nssdb
$ certutil -d sql:$HOME/nssdb -N
$ certutil -d sql:$HOME/nssdb -A -n cacert -t C,, <ca-cert.pem
$ export SSL_DIR=$HOME/nssdb
$ valgrind --tool=massif --massif-out-file=/tmp/t.%p curl5 -t 1 -m 1 1024 https://...
There is no known fix for this bug. nss-pem is already in maintenance mode because it is going to be substituted by OpenSSL in RHEL-8. Closing as WONTFIX. Note that the memory consumption of (lib)curl running on top of NSS was already significantly improved in RHEL-7.6 per bug #1510247. Development Management has reviewed and declined this request. You may appeal this decision by reopening this request. |
Created attachment 1427442 [details] source to test program Description of problem: Version-Release number of selected component (if applicable): tested with: libcurl-7.29.0-47.el7.x86_64 nss-3.34.0-4.el7.x86_64 How reproducible: Steps to Reproduce: 1. setup a local web server w/ ssl. 2. setup rhel instance w/ c++ compiler, libcurl-devel, valgrind 3. c++ -g -o curl5 curl5.cc -lcurl -lpthread -std=c++11 4. valgrind --massif-out-file=/tmp/t.%p curl5 -t 1 -m 1 1024 https:<<your-web-servers-fqdn>> 5. ms_print /tmp/t.* >/tmp/out 6. sed 30q /tmp/out Actual results: chart showing ramp up of memory usage - from about 2.5m to about 5m. Expected results: chart showing bounded flat memory usage - all peaks < 1.5m. Additional info: