Bug 1117959

Summary:	Can curl HTTPS requests make fewer access system calls? [rhel-7]
Product:	Red Hat Enterprise Linux 7	Reporter:	Hubert Kario <hkario>
Component:	nss-softokn	Assignee:	Elio Maldonado Batiz <emaldona>
Status:	CLOSED ERRATA	QA Contact:	Hubert Kario <hkario>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	7.0	CC:	emaldona, hkario, kdudka, pchavan, rrelyea, sforsber, thatsafunnyname
Target Milestone:	rc	Keywords:	Regression
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:	nss-softokn-3.16.2-1.el7	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:	1044666	Environment:
Last Closed:	2015-03-05 08:28:09 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1044666
Bug Blocks:

Description Hubert Kario 2014-07-09 17:26:09 UTC

Marking as regression since the bug will be fixed in RHEL 6.6.

Reproducible using nss-softokn-3.15.4-2.el7.x86_64

+++ This bug was initially created as a clone of Bug #1044666 +++

Description of problem:

When using curl 7.19.7-37.el6_4, or libcurl on RHEL 6.4 to make a HTTPS request there are many "access" system calls to paths/directories/files that do not exist, this bloats the dentry_cache.

Version-Release number of selected component (if applicable):

curl 7.19.7-37.el6_4.x86_64 on RHEL 6.4
nss 3.14.3-4.el6_4.x86_64

I also tested 3.15.1-15.el6, and 7.29.0-12.el7 on RHEL 7.0 Beta.

How reproducible:

Very, strace curl making a HTTPS request.
Watch dentry_cache size in "slabtop".

Steps to Reproduce:

strace -fc -e trace=access curl 'https://www.google.com' > /dev/null

Actual results:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
131   262  131   262    0     0   1564      0 --:--:-- --:--:-- --:--:-- 13789
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
100.00    0.000598           0      1576      1574 access
------ ----------- ----------- --------- --------- ----------------
100.00    0.000598                  1576      1574 total

Expected results:

Less calls to directories that do not exist, dentry_cache not bloated in size.
I did not find an option to change this behaviour, I was able to use the environment variable SSL_DIR to change the path being used but this did not change the behaviour.

Additional info:
 
strace -f -e trace=access curl 'https://www.google.com'
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb", W_OK)          = -1 EACCES (Permission denied)
access("/hhoudini/.pki/nssdb/.2777916463_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916464_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916465_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916466_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916467_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916468_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916469_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916470_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916471_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916472_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916473_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916474_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916475_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916476_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916477_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916478_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916479_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916480_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916481_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916482_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916483_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916484_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916485_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916486_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916487_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916488_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916489_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916490_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916491_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916492_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916493_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916494_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916495_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916496_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916497_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916498_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916499_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916500_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916501_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916502_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916503_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916504_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916505_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916506_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916507_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916508_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916509_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916510_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916511_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916512_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916513_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916514_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916515_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916516_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916517_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916518_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916519_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916520_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916521_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916522_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916523_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916524_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916525_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916526_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916527_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916528_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916529_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916530_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916531_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916532_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916533_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916534_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916535_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916536_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916537_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916538_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916539_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916540_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916541_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916542_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916543_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916544_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916545_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916546_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916547_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916548_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916549_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916550_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916551_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916552_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916553_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916554_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916555_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916556_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916557_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916558_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916559_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916560_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916561_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916562_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916563_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916564_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916565_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916566_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916567_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916568_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916569_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/.2777916570_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/cert9.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/cert8.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/cert7.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/cert6.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/cert5.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/cert4.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/cert3.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/cert2.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/key3.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/key2.db", F_OK) = -1 ENOENT (No such file or directory)
access("/hhoudini/.pki/nssdb/secmod.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916496_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916497_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916498_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916499_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916500_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916501_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916502_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916503_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916504_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916505_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916506_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916507_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916508_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916509_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916510_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916511_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916512_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916513_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916514_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916515_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916516_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916517_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916518_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916519_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916520_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916521_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916522_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916523_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916524_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916525_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916526_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916527_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916528_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916529_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916530_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916531_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916532_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916533_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916534_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916535_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916536_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916537_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916538_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916539_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916540_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916541_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916542_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916543_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916544_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916545_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916546_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916547_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916548_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916549_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916550_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916551_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916552_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916553_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916554_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916555_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916556_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916557_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916558_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916559_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916560_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916561_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916562_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916563_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916564_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916565_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916566_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916567_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916568_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916569_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916570_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916571_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916572_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916573_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916574_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916575_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916576_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916577_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916578_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916579_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916580_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916581_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916582_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916583_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916584_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916585_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916586_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916587_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916588_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916589_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916590_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916591_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916592_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916593_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916594_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916595_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916596_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916597_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916598_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916599_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916600_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916601_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916602_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916603_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916604_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916605_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916606_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/.2777916607_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/cert9.db", F_OK) = 0
access("/etc/pki/nssdb/cert9.db-journal", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/key4.db", F_OK)  = 0
access("/etc/pki/nssdb/key4.db-journal", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/key4.db-journal", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/key4.db-journal", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/key4.db-journal", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/cert9.db-journal", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/cert9.db-journal", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/cert9.db-journal", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/cert9.db-journal", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/cert9.db-journal", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/cert9.db-journal", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/cert9.db-journal", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/cert9.db-journal", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/cert9.db-journal", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/cert9.db-journal", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/cert9.db-journal", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/cert9.db-journal", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/cert9.db-journal", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/cert9.db-journal", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/pki/nssdb/cert9.db-journal", F_OK) = -1 ENOENT (No such file or directory)
 
Just to show how big the dentry_cache was:
 
  OBJS ACTIVE  USE OBJ SIZE  SLABS OBJ/SLAB CACHE SIZE NAME
182485800 182485794   5%    0.19K 9124290       20  36497160K dentry
2816736 1883738  66%    0.10K  76128       37    304512K buffer_head
393547 359443  91%    0.20K  20713       19     82852K vm_area_struct
231910 190008  81%    0.55K  33130        7    132520K radix_tree_node
194887 157962  81%    0.05K   2531       77     10124K anon_vma_chain
174380 118313  67%    0.19K   8719       20     34876K filp
169924 137070  80%    0.04K   1847       92      7388K anon_vma
159975 159922  99%    1.02K  53325        3    213300K nfs_inode_cache
 
Also reported here:
  http://curl.haxx.se/mail/tracker-2013-06/0114.html

--- Additional comment from Peter Edwards on 2013-12-19 10:56:22 EST ---

I noticed:
 
valgrind --trace-children=yes curl 'https://google.com'
 
reports "possibly lost: 176 bytes in 4 blocks"
 
40 bytes in 1 blocks are possibly lost in loss record 44 of 101
   at 0x4A072CB: malloc (vg_replace_malloc.c:263)
   by 0x314B20D78B: ??? (in /lib64/libnspr4.so)
   by 0x314B225014: ??? (in /lib64/libnspr4.so)
   by 0x314B226210: PR_Socket (in /lib64/libnspr4.so)
   by 0x38AD840637: Curl_nss_connect (in /usr/lib64/libcurl.so.4.1.1)
   by 0x38AD838481: Curl_ssl_connect (in /usr/lib64/libcurl.so.4.1.1)
   by 0x38AD816ECA: Curl_http_connect (in /usr/lib64/libcurl.so.4.1.1)
   by 0x38AD81D681: Curl_protocol_connect (in /usr/lib64/libcurl.so.4.1.1)
   by 0x38AD823B3B: Curl_connect (in /usr/lib64/libcurl.so.4.1.1)
   by 0x38AD82BBAF: Curl_perform (in /usr/lib64/libcurl.so.4.1.1)
   by 0x40875F: ??? (in /usr/bin/curl)
   by 0x3146E1ECDC: (below main) (in /lib64/libc-2.12.so)
 
40 bytes in 1 blocks are possibly lost in loss record 45 of 101
   at 0x4A072CB: malloc (vg_replace_malloc.c:263)
   by 0x314B20D78B: ??? (in /lib64/libnspr4.so)
   by 0x314B225014: ??? (in /lib64/libnspr4.so)
   by 0x314B2253D8: PR_OpenFile (in /lib64/libnspr4.so)
   by 0x5742CB4: ???
   by 0x574052A: ???
   by 0x57458C8: ???
   by 0x574C521: ???
   by 0x314B6470C9: ??? (in /usr/lib64/libnss3.so)
   by 0x314B647362: PK11_CreateGenericObject (in /usr/lib64/libnss3.so)
   by 0x38AD83F49F: ??? (in /usr/lib64/libcurl.so.4.1.1)
   by 0x38AD83F626: ??? (in /usr/lib64/libcurl.so.4.1.1)
 
48 bytes in 1 blocks are possibly lost in loss record 49 of 101
   at 0x4A072CB: malloc (vg_replace_malloc.c:263)
   by 0x314B20D779: ??? (in /lib64/libnspr4.so)
   by 0x314B225014: ??? (in /lib64/libnspr4.so)
   by 0x314B226210: PR_Socket (in /lib64/libnspr4.so)
   by 0x38AD840637: Curl_nss_connect (in /usr/lib64/libcurl.so.4.1.1)
   by 0x38AD838481: Curl_ssl_connect (in /usr/lib64/libcurl.so.4.1.1)
   by 0x38AD816ECA: Curl_http_connect (in /usr/lib64/libcurl.so.4.1.1)
   by 0x38AD81D681: Curl_protocol_connect (in /usr/lib64/libcurl.so.4.1.1)
   by 0x38AD823B3B: Curl_connect (in /usr/lib64/libcurl.so.4.1.1)
   by 0x38AD82BBAF: Curl_perform (in /usr/lib64/libcurl.so.4.1.1)
   by 0x40875F: ??? (in /usr/bin/curl)
   by 0x3146E1ECDC: (below main) (in /lib64/libc-2.12.so)
 
48 bytes in 1 blocks are possibly lost in loss record 50 of 101
   at 0x4A072CB: malloc (vg_replace_malloc.c:263)
   by 0x314B20D779: ??? (in /lib64/libnspr4.so)
   by 0x314B225014: ??? (in /lib64/libnspr4.so)
   by 0x314B2253D8: PR_OpenFile (in /lib64/libnspr4.so)
   by 0x5742CB4: ???
   by 0x574052A: ???
   by 0x57458C8: ???
   by 0x574C521: ???
   by 0x314B6470C9: ??? (in /usr/lib64/libnss3.so)
   by 0x314B647362: PK11_CreateGenericObject (in /usr/lib64/libnss3.so)
   by 0x38AD83F49F: ??? (in /usr/lib64/libcurl.so.4.1.1)
   by 0x38AD83F626: ??? (in /usr/lib64/libcurl.so.4.1.1)
 
I don't know if these are false.  I found:
 
  https://bugzilla.redhat.com/show_bug.cgi?id=1007603#c13
 
nss is also using sqlite.
 
Thanks.

--- Additional comment from Kamil Dudka on 2013-12-19 11:47:32 EST ---

The certificate database is manipulated by NSS.  I am switching the component...

--- Additional comment from Peter Edwards on 2013-12-19 12:01:42 EST ---

I created a support request:
  https://access.redhat.com/support/cases/01002015/

--- Additional comment from Peter Edwards on 2013-12-20 06:47:29 EST ---

Here is a reproducer showing how a single process making sequential HTTPS requests using curl can cause significant increases in the dentry cache (and slab), after running:
 
echo 2 > /proc/sys/vm/drop_caches
 
as root, then run this command as non root, replace the url with a HTTPS server you are responsible for:
 
/bin/egrep 'Slab|claim' /proc/meminfo ; /usr/bin/slabtop -o | /bin/egrep 'Slab|SLAB|dent' ; /usr/bin/perl -e 'foreach(1..1000){print q{.}.`/usr/bin/curl --silent https://replace-with-a-server-you-admin.com/does_not_exist.html > /dev/null`};print qq{\n}' ; /usr/bin/slabtop -o | /bin/egrep 'Slab|SLAB|dent' ;  /bin/egrep 'Slab|claim' /proc/meminfo
 
First run:
 
Slab:             291652 kB
SReclaimable:     212536 kB
SUnreclaim:        79116 kB
 Active / Total Slabs (% used)      : 68277 / 68291 (100.0%)
  OBJS ACTIVE  USE OBJ SIZE  SLABS OBJ/SLAB CACHE SIZE NAME
 51320  18890  36%    0.19K   2566       20     10264K dentry
...<removed>...
 Active / Total Slabs (% used)      : 74055 / 74079 (100.0%)
  OBJS ACTIVE  USE OBJ SIZE  SLABS OBJ/SLAB CACHE SIZE NAME
145380 145380 100%    0.19K   7269       20     29076K dentry
Slab:             314800 kB
SReclaimable:     235300 kB
SUnreclaim:        79500 kB
 
after 15 more runs the 16th run looks like this:
 
Slab:             646524 kB
SReclaimable:     567352 kB
SUnreclaim:        79172 kB
 Active / Total Slabs (% used)      : 157004 / 157014 (100.0%)
  OBJS ACTIVE  USE OBJ SIZE  SLABS OBJ/SLAB CACHE SIZE NAME
1792780 1792780 100%    0.19K  89639       20    358556K dentry
...<removed>...
 Active / Total Slabs (% used)      : 162842 / 162861 (100.0%)
  OBJS ACTIVE  USE OBJ SIZE  SLABS OBJ/SLAB CACHE SIZE NAME
1907720 1907720 100%    0.19K  95386       20    381544K dentry
Slab:             669860 kB
SReclaimable:     590196 kB
SUnreclaim:        79664 kB
 
Slab more than doubled in size and the dentry cache size and number of objects increased to 37 times the initial sizes.
 
It would be great if curl (libcurl) could be used to make HTTPS connections/requests and not have the nss library nss-3.14.3/mozilla/security/nss/lib/softoken/sdb.c:sdb_measureAccess() "find out how expensive the access system call is for non-existant files in the given directory.  Return the number of operations done in 33 ms".  When sdb_measureAccess() is called from sdb_init it uses the environment variable NSS_SDB_USE_CACHE to control if the cache is used or not ("yes" or "no"), however in s_open it is called without examining the environment variable NSS_SDB_USE_CACHE.
 
    /* how long does it take to test for a non-existant file in our working
     * directory? Allows us to test if we may be on a network file system */
    accessOps = sdb_measureAccess(directory);
 
Changed from 200 iterations in:
 https://hg.mozilla.org/projects/nss/diff/4d876e0ee318/security/nss/lib/softoken/sdb.c 
added in:
 https://hg.mozilla.org/projects/nss/annotate/198b92c76b08/security/nss/lib/softoken/sdb.c
for:
 https://bugzilla.mozilla.org/show_bug.cgi?id=391294 
 
On a RHEL5.9 the same command does not cause a significant increase in dentry_cache/slab, curl being built with OpenSSL and not NSS.
 
Thanks.

Comment 5 Elio Maldonado Batiz 2014-10-23 16:26:38 UTC

This was fixed upstream for nss-3.16 and we picked up the fix when we reabased to 3.16.2, Marking it as modified on nss-softokn-3.16.2-1.el7 as this the first build containing the fix.

Comment 10 errata-xmlrpc 2015-03-05 08:28:09 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0364.html