Bug 1042992

Description of problem:
Can't get following applications to pickup "new installed ca's, 
- ldaptools to pickup ca's 
- sssd 
- pkcs11_inspect, 

Maybe I'm missing something here, but I've spent a great deal of time the last days trying to figure this certificate situation on Rhel. Finally I found bz #466626 which shed some light upon the situation and made me feel some hope. It looked like it was only on Fedora, but further digging seems to suggest that this is actually implemented (at least partially) on Rhel. 

So, my goal was to install our root-ca into a rhel-system and make "the system" aware of it, no matter if the application was nss/openssl/tls-based. 

> man update-ca-trust suggests that this is actually ridiculously (if I understood everything correctly) easy (as I was hoping for), 

- update-ca-trust enable
- add our root-ca under /etc/pki/ca-trust/source/anchors/
- update-ca-trust extract

That's it ! 
Awesome. 
And it worked. 
Almost. Sort of.
Or I'm missing something. 

Here's whats working after following the above mention method, 

> openssl s_client -verify 5 -connect xxxxx.xxxx.xx:443
> curl -w "Verify: %{ssl_verify_result}\n" --head https://xxxxx.xxxx.xx

Both openssl/curl uses openssl, right ? 


> firefox https://xxxx.xxxx.xx
> google-chrome https://xxxx.xxxx.xx

Both of the browsers use nssdb, right ? 
And by using nssdb, does that mean that they implicitly make use of the "libnssckbi.so" (that is linked to p11-kit-trust.so after running update-ca-trust enable). To be honest I'm a bit confused when applications make us of the library and if so that is handeled by the application itself or if't "implicit" when using the nss-tools. I find this area somewhat confusing. However, the browsers are working as expected so I'm satisfied withe the result. 


> java testxxxx443 (openJDK) 
Modifying the java-class (so it connects to our host instead of bugzilla) found at https://fedoraproject.org/wiki/Features/SharedSystemCertificates:Testing






Here's whats *NOT* working after following the above mention method,

> gnutls-cli xxxxx.xxxx.xx
> gnutls-cli bugzilla.redhat.com 
- The hostname in the certificate matches 'bugzilla.redhat.com'.
- Peer's certificate issuer is unknown
- Peer's certificate is NOT trusted
Interestingly enough not even bugzilla.redhat.com (or any host for that matter) is recognized, which makes me believe that there is something really fishy with tls. 

> gnutls-cli xxxxx.xxxx.xx --x509cafile /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 
> gnutls-cli bugzilla.redhat.com --x509cafile /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 
Works like a charm though. 

Same goes with the ldaptools*/sssd, 
> ldapsearch -ZZ cn=xx cn -h xxxxx.xxxx.xx -d1
- TLS: can't connect: TLS error -8179:Peer's Certificate issuer is not recognized..


So, using TLS-based tools, it seems that they wont work unless i specifically specify a cafile/cadir-option. 
Both ldap/sssd and works with the options, 
/etc/openldap/ldap.conf,  tls_cacert = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
/etc/sssd/sssd.conf, ldap_tls_cacert = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem (this actually default to whatever is set in /etc/openldap/ldap.conf, but just for the clarity I specify it)

And that's fine I guess, I was just under the impression that I wouldn't need to specify the cacert when using this "new methodology", am I missing something ? 



> pkcs11_inspect debug
-DEBUG:pam_config.c:238: Using config file /etc/pam_pkcs11/pam_pkcs11.conf
-DEBUG:pkcs11_lib.c:182: Initializing NSS ...
-DEBUG:pkcs11_lib.c:192: Initializing NSS ... database=/etc/pki/nssdb
-DEBUG:pkcs11_lib.c:210: ...  NSS Complete
-DEBUG:pkcs11_inspect.c:69: loading pkcs #11 module...
-DEBUG:pkcs11_lib.c:222: Looking up module in list
-DEBUG:pkcs11_lib.c:225: modList = 0x1522270 next = 0x1554830
-DEBUG:pkcs11_lib.c:226: dllName= <null> 
-DEBUG:pkcs11_lib.c:225: modList = 0x1554830 next = 0x0
-DEBUG:pkcs11_lib.c:226: dllName= /usr/lib/libiidp11.so 
-DEBUG:pkcs11_inspect.c:78: initialising pkcs #11 module...
-PIN for token: 
-DEBUG:pkcs11_lib.c:48: PIN = [XXXX]
-DEBUG:pkcs11_lib.c:746: cert 0: found (Instant EID IP9 (identification):xxx), "E=xxx@xx,CN=xx,OU=People,DC=xx,DC=xxxx,DC=xx"
-DEBUG:mapper_mgr.c:172: Retrieveing mapper module list
-DEBUG:mapper_mgr.c:73: Loading static module for mapper 'cn'
-DEBUG:mapper_mgr.c:197: Inserting mapper [cn] into list
-DEBUG:pkcs11_inspect.c:128: Found '1' certificate(s)
-DEBUG:pkcs11_inspect.c:132: verifing the certificate #1
-DEBUG:cert_vfy.c:34: Verifying Cert: Instant EID IP9 (identification):xxx (E=xx@xx,CN=xx,OU=People,DC=xx,DC=xxxx,DC=xx)
-DEBUG:cert_vfy.c:38: Couldn't verify Cert: Peer's Certificate issuer is not recognized.
-ERROR:pkcs11_inspect.c:142: verify_certificate() failed: 
-DEBUG:mapper_mgr.c:214: unloading mapper module list
-DEBUG:mapper_mgr.c:137: calling mapper_module_end() cn
-DEBUG:mapper_mgr.c:148: Module cn is static: don't remove
-DEBUG:pkcs11_inspect.c:163: releasing pkcs #11 module...
-DEBUG:pkcs11_inspect.c:166: Process completed

This result is also a bit confusing. 
Since pkcs11_inspect makes use of the nssdb I thought that it would be able to lookup ca's trough the "p11-kit-trust.so"-module, but my confusion here is a fact. 
I can only make pkcs11_inspect work by manually adding our rootca with the certutil-command specifying the /etc/pki/nssdb. 


> java testxxxx443 (oracle) 
Modifying the java-class (so it connects to our host instead of bugzilla) found at https://fedoraproject.org/wiki/Features/SharedSystemCertificates:Testing
- The oracle-java makes use of the "/usr/lib/jvm/jre-x.x.x-oracle.x86_64/lib/security/cacerts"-database, and not the /etc/pki/ca-trust/exported/java/cacert as I was hoping for. 



So, to summarize this amazingly long "bug-report", 
- I would love to make use of the new "SharedSystemCertificates methodology"
- What's going on with the TLS-based applications, why don't they pick up the "root-ca-bundle" automagically ? 
- Why isn't pkcs11_inspect picking up the "root-ca-bundle" with the help of the "p11-kit-trust.so" (some explanation regarding nssdb/libnssckbi.so/p11-kit-trust.so and when they are loaded/used would be extremely cool). 
- And isn't oracles-java suppose to use the /etc/pki/ca-trust/exported/java/cacert-store ? 



Version-Release number of selected component (if applicable):
cat /etc/redhat-release 
Red Hat Enterprise Linux Workstation release 6.5 (Santiago)
sssd-1.9.2-129.el6.x86_64
openldap-2.4.23-32.el6_4.1.x86_64
pam_pkcs11-0.6.2-13.el6.x86_64


How reproducible:
Always ? 


Steps to Reproduce:
I've tried to summarize everything in some order in the description. 


Actual results:
openssl-based applications seems to work. 
Some nss-based applications seems to work (firefox/google-chrome) while others like pkcs11_inspect doesn't. 

Expected results:
I would expect that every application that made use of nss/tls/openssl/java? used the "SharedSystemCertificates"-store.

Additional info:
I would be more than happy to assist further in this issue since I want really want it to be that simple to add a "system-wide" ca.