Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1632664

Summary: sssd does not display meaningful error CERT_VerifyCertificateNow failed [-8102]
Product: Red Hat Enterprise Linux 7 Reporter: amitkuma
Component: sssdAssignee: SSSD Maintainers <sssd-maint>
Status: CLOSED CANTFIX QA Contact: sssd-qe <sssd-qe>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.6CC: amitkuma, fedoraproject, grajaiya, jhrozek, lslebodn, mzidek, pbrezina, sbose, tscherf
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-03-23 16:30:14 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
opensc debug logs debug=3 none

Description amitkuma 2018-09-25 10:43:20 UTC
Description of problem:
Configured Smart card Authentication for active directory user Login using Smart card and SSSD.

Followed: https://docs.pagure.org/SSSD.sssd/design_pages/smartcard_authentication_testing_with_ad.html

sssd.conf
[sssd]
domains = atest.com
config_file_version = 2
services = nss, pam, ssh
[ssh]
debug_level = 10
[pam]
pam_cert_auth = True
[domain/atest.com]
ad_server = windows2008-2.atest.com
ad_domain = atest.com
krb5_realm = ATEST.COM
realmd_tags = manages-system joined-with-adcli
cache_credentials = True
id_provider = ad
krb5_store_password_if_offline = True
default_shell = /bin/bash
ldap_id_mapping = True
use_fully_qualified_names = false
fallback_homedir = /home/%u@%d
access_provider = ad
ldap_user_certificate = userCertificate;binary
[nss]
#debug_level = 10

# modutil -dbdir /etc/pki/nssdb -list

Listing of PKCS #11 Modules
-----------------------------------------------------------
  1. NSS Internal PKCS #11 Module
	   uri: pkcs11:library-manufacturer=Mozilla%20Foundation;library-description=NSS%20Internal%20Crypto%20Services;library-version=3.36
	 slots: 2 slots attached
	status: loaded

	 slot: NSS Internal Cryptographic Services
	token: NSS Generic Crypto Services
	  uri: pkcs11:token=NSS%20Generic%20Crypto%20Services;manufacturer=Mozilla%20Foundation;serial=0000000000000000;model=NSS%203

	 slot: NSS User Private Key and Certificate Services
	token: NSS Certificate DB
	  uri: pkcs11:token=NSS%20Certificate%20DB;manufacturer=Mozilla%20Foundation;serial=0000000000000000;model=NSS%203

  2. CoolKey PKCS #11 Module
	library name: libcoolkeypk11.so
	   uri: pkcs11:library-manufacturer=Mozilla%20Foundation;library-description=CoolKey%20PKCS%20%2311%20Module%20%20%20%20%20;library-version=1.0
	 slots: There are no slots attached to this module
	status: loaded

  3. OpenSC
	library name: /usr/lib64/opensc-pkcs11.so
	   uri: pkcs11:library-manufacturer=OpenSC%20Project;library-description=OpenSC%20smartcard%20framework;library-version=0.16
	 slots: There are no slots attached to this module
	status: loaded
-----------------------------------------------------------

//Added Root CA certificate in nssdb
#  certutil -L -d /etc/pki/nssdb -h all 
Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI
RHCS CA Cert                                                 CT,C,C

# openssl x509 -in rooCA.crt -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1 (0x1)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: O = gsslab.pnq2.redhat.com Security Domain, OU = pki-tomcat, CN = CA Signing Certificate
        Validity
            Not Before: Sep 10 10:07:11 2018 GMT
            Not After : Sep 10 10:07:11 2038 GMT
        Subject: O = gsslab.pnq2.redhat.com Security Domain, OU = pki-tomcat, CN = CA Signing Certificate
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:f5:cb:cf:e2:6b:33:85:88:23:a1:05:33:8b:e2:
                    d7:c8:0e:d1:ae:6e:7d:5f:94:8f:96:de:91:4b:35:
                    05:7e:20:b2:94:a4:87:df:a3:51:d3:8d:b8:7c:9e:
                    e0:3e:f9:f6:9c:52:c7:4a:48:99:15:51:d1:f2:32:
                    a9:75:0f:ef:9e:c4:8b:35:d3:93:bc:de:00:e3:ac:
                    62:e4:d3:7f:9d:2d:9d:8d:c1:2e:be:d5:8d:6d:b7:
                    3b:02:74:8b:f3:14:6e:45:12:e3:fc:05:7f:ca:4c:
                    59:34:3a:90:47:8a:93:69:a6:c1:d6:61:1b:e6:ff:
                    0e:e0:b1:ea:15:08:0e:a8:b2:26:64:9a:44:14:a4:
                    33:fc:35:f0:ef:3a:a9:e5:08:33:20:e8:44:cc:09:
                    6c:ec:22:c0:8b:53:9f:c7:cf:d1:39:d4:af:41:15:
                    10:11:c8:87:c5:81:ca:5a:3b:c7:d7:83:5f:99:ad:
                    f7:29:0a:d7:11:3c:e8:aa:cc:17:c4:a6:23:ef:29:
                    c8:27:cd:55:c5:32:88:0d:9f:25:ad:30:d7:9c:51:
                    71:f1:d9:63:4d:a0:89:c4:9b:59:35:24:f5:65:43:
                    be:18:77:e3:b4:35:a4:0a:4d:ae:54:79:60:b9:a0:
                    3f:b3:d5:de:82:ea:e2:d5:52:0d:1a:81:8f:83:ce:
                    c9:cd
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Authority Key Identifier: 
                keyid:85:C1:17:3F:38:A1:E7:57:72:33:5A:52:9B:5A:98:63:8D:9B:EA:CF

            X509v3 Basic Constraints: critical
                CA:TRUE
            X509v3 Key Usage: critical
                Digital Signature, Non Repudiation, Certificate Sign, CRL Sign
            X509v3 Subject Key Identifier: 
                85:C1:17:3F:38:A1:E7:57:72:33:5A:52:9B:5A:98:63:8D:9B:EA:CF
            Authority Information Access: 
                OCSP - URI:http://vm255-214.gsslab.pnq2.redhat.com:8080/ca/ocsp

    Signature Algorithm: sha256WithRSAEncryption
         d1:b3:ce:a0:c7:2b:4b:cb:ad:0e:f0:ec:68:8a:49:0b:2c:ee:
         de:74:28:ba:2a:49:4e:21:42:52:a4:04:c1:e4:99:62:c4:ca:
         15:86:67:7c:6a:e2:73:1f:42:cc:72:92:92:37:8a:6a:5d:d8:
         88:16:78:1e:c5:d5:40:41:43:7d:3f:c4:72:69:f3:ac:6c:17:
         7a:ed:a2:3b:39:18:a3:74:12:91:4a:bc:ff:02:5c:cb:f4:b4:
         13:bc:79:96:28:ec:2a:39:09:7a:96:79:b6:b9:d5:22:89:52:
         ca:45:51:a8:41:b9:4a:8f:02:b8:e0:13:b6:43:59:eb:cb:dd:
         e4:87:67:ec:3d:e5:f9:5a:4c:9f:45:2c:5b:85:d6:c8:9e:68:
         03:2c:a3:23:aa:50:34:da:75:c4:16:be:e6:12:02:8f:72:bc:
         11:a2:7c:bd:b2:19:f4:9f:f8:74:2a:03:12:c7:9d:25:f2:45:
         d0:5b:a2:75:55:f7:50:8a:8a:33:3f:80:c6:a6:a5:8f:9a:3f:
         18:6e:f8:4d:09:3e:ee:05:d5:9d:5d:e1:ab:9a:bb:b2:11:10:
         3b:0a:d5:cb:7e:63:9f:11:7d:15:ff:a5:7f:05:be:9a:9a:1d:
         0b:ee:71:53:91:51:5b:1f:3e:0c:f3:b2:69:06:b3:1b:95:c2:
         1e:c0:59:43


Certificate Present on Smart-card:
# openssl x509 -in cert-prasad.pem -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 17 (0x11)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: O = gsslab.pnq2.redhat.com Security Domain, OU = pki-tomcat, CN = CA Signing Certificate
        Validity
            Not Before: Sep 25 08:16:01 2018 GMT
            Not After : Sep 24 08:16:01 2023 GMT
        Subject: O = Token Key User, UID = prasad
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (1024 bit)
                Modulus:
                    00:83:b8:ea:1f:a9:74:dc:4a:43:a9:6c:72:d5:46:
                    4a:67:09:ab:de:33:d4:40:07:32:d6:19:3f:f6:ab:
                    03:57:bd:6d:5a:e7:74:f6:56:c0:5e:b7:be:bd:b3:
                    58:bb:52:c6:5b:22:30:68:a0:e1:9b:61:7e:94:bf:
                    f9:d8:4b:70:29:85:71:c3:b0:1a:33:92:2f:89:fd:
                    e7:16:b7:15:e7:51:45:4c:3a:b2:94:36:9a:d5:89:
                    67:e9:ea:f4:45:97:38:93:81:f5:4e:ee:ef:2a:67:
                    3a:e5:81:d5:ce:e1:0d:07:c3:c7:7d:38:22:19:18:
                    46:f4:8e:42:74:28:67:cc:25
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Key Encipherment
            X509v3 Subject Alternative Name: 
                email:$request.mail$
            X509v3 Subject Key Identifier: 
                01:84:DF:58:D0:D2:A0:BC:82:C3:DE:55:4F:CE:4A:BD:6C:D4:92:B0
            X509v3 Authority Key Identifier: 
                keyid:85:C1:17:3F:38:A1:E7:57:72:33:5A:52:9B:5A:98:63:8D:9B:EA:CF

            X509v3 Basic Constraints: 
                CA:FALSE
    Signature Algorithm: sha256WithRSAEncryption
         a0:fc:99:9e:fb:05:33:bc:b0:0d:93:4f:72:98:b4:0c:a3:5c:
         9b:2d:ea:e5:f5:d8:f5:af:b5:82:e4:71:49:8b:38:94:99:65:
         70:8f:14:9c:e9:ee:57:75:26:a3:95:5c:de:39:a7:7c:62:33:
         74:0a:e5:50:d2:86:30:f5:00:90:4a:68:ea:c8:9a:c2:76:ae:
         29:8f:40:3b:0f:e8:35:fc:e3:e0:39:9d:db:a8:44:68:34:85:
         a6:3c:2d:5b:43:83:23:85:f0:58:20:15:45:7e:eb:52:16:34:
         29:c4:9e:43:ae:76:28:08:41:f0:e8:65:44:18:f3:e6:8d:18:
         e1:10:22:2d:dc:22:1a:0d:4c:b2:9c:9c:0c:df:ce:a1:56:41:
         2c:04:e6:54:7b:42:16:bb:db:91:67:98:ea:eb:b3:fc:0f:54:
         6d:36:5c:61:f7:9a:ce:04:da:2e:2b:31:44:e6:ba:70:cd:5a:
         90:2f:d5:88:29:22:b0:c3:ca:d7:0c:57:c8:68:92:a5:32:d8:
         ac:80:a1:33:47:47:f8:4a:82:71:f0:30:0e:ee:f6:25:2d:4c:
         3c:43:74:6e:ee:b3:5d:77:f2:5d:9f:6d:ce:28:48:05:21:25:
         07:e4:d5:d3:27:3d:45:14:d1:8a:25:35:8e:bd:dd:c1:a6:96:
         6d:39:ea:e2

//Loaded Certificate in AD user entry
# vim test-prasad.ldif
dn: CN=prasad,CN=Users,DC=atest,DC=com
changetype: modify
add: userCertificate
userCertificate:< file:cert-prasad.der
# ldapmodify -Y GSSAPI -H ldap://windows2008-2.atest.com -f test-prasad.ldif    


//Done ldapsearch on AD to verify certificate present
# kinit prasad
#ldapsearch -Y GSSAPI -H ldap://windows2008-2.atest.com -b 'dc=atest,dc=com' samAccountName=prasad
SASL/GSSAPI authentication started
SASL username: prasad
SASL SSF: 256
SASL data security layer installed.
# extended LDIF
#
# LDAPv3
# base <dc=atest,dc=com> with scope subtree
# filter: samAccountName=prasad
# requesting: ALL
#

# prasad, Users, atest.com
dn: CN=prasad,CN=Users,DC=atest,DC=com
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: user
cn: prasad
userCertificate:: MIIDBzCCAe+gAwIBAgIBETANBgkqhkiG9w0BAQsFADBnMS8wLQYDVQQKDCZn
 c3NsYWIucG5xMi5yZWRoYXQuY29tIFNlY3VyaXR5IERvbWFpbjETMBEGA1UECwwKcGtpLXRvbWNhd
 DEfMB0GA1UEAwwWQ0EgU2lnbmluZyBDZXJ0aWZpY2F0ZTAeFw0xODA5MjUwODE2MDFaFw0yMzA5Mj
 QwODE2MDFaMDExFzAVBgNVBAoMDlRva2VuIEtleSBVc2VyMRYwFAYKCZImiZPyLGQBAQwGcHJhc2F
 kMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCDuOofqXTcSkOpbHLVRkpnCaveM9RABzLWGT/2
 qwNXvW1a53T2VsBet769s1i7UsZbIjBooOGbYX6Uv/nYS3AphXHDsBozki+J/ecWtxXnUUVMOrKUN
 prViWfp6vRFlziTgfVO7u8qZzrlgdXO4Q0Hw8d9OCIZGEb0jkJ0KGfMJQIDAQABo3gwdjAOBgNVHQ
 8BAf8EBAMCBSAwGQYDVR0RBBIwEIEOJHJlcXVlc3QubWFpbCQwHQYDVR0OBBYEFAGE31jQ0qC8gsP
 eVU/OSr1s1JKwMB8GA1UdIwQYMBaAFIXBFz84oedXcjNaUptamGONm+rPMAkGA1UdEwQCMAAwDQYJ
 KoZIhvcNAQELBQADggEBAKD8mZ77BTO8sA2TT3KYtAyjXJst6uX12PWvtYLkcUmLOJSZZXCPFJzp7
 ld1JqOVXN45p3xiM3QK5VDShjD1AJBKaOrImsJ2rimPQDsP6DX84+A5nduoRGg0haY8LVtDgyOF8F
 ggFUV+61IWNCnEnkOudigIQfDoZUQY8+aNGOEQIi3cIhoNTLKcnAzfzqFWQSwE5lR7Qha725FnmOr
 rs/wPVG02XGH3ms4E2i4rMUTmunDNWpAv1YgpIrDDytcMV8hokqUy2KyAoTNHR/hKgnHwMA7u9iUt
 TDxDdG7us1138l2fbc4oSAUhJQfk1dMnPUUU0YolNY693cGmlm056uI=
givenName: prasad
distinguishedName: CN=prasad,CN=Users,DC=atest,DC=com
instanceType: 4
whenCreated: 20180925081811.0Z
whenChanged: 20180925083605.0Z
displayName: prasad
uSNCreated: 643380
uSNChanged: 643391
name: prasad
objectGUID:: L7jSsilXhES4pyVWr6fCQw==
userAccountControl: 66048
badPwdCount: 0
codePage: 0
countryCode: 0
badPasswordTime: 0
lastLogoff: 0
lastLogon: 131823435017656250
pwdLastSet: 131823370914687500
primaryGroupID: 513
objectSid:: AQUAAAAAAAUVAAAAN2MWDbhMfvyVveAccwQAAA==
accountExpires: 9223372036854775807
logonCount: 2
sAMAccountName: prasad
sAMAccountType: 805306368
userPrincipalName: prasad
objectCategory: CN=Person,CN=Schema,CN=Configuration,DC=atest,DC=com
dSCorePropagationData: 16010101000000.0Z
lastLogonTimestamp: 131823381655625000

# search reference
ref: ldap://ForestDnsZones.atest.com/DC=ForestDnsZones,DC=atest,DC=com

# search reference
ref: ldap://DomainDnsZones.atest.com/DC=DomainDnsZones,DC=atest,DC=com

# search reference
ref: ldap://atest.com/CN=Configuration,DC=atest,DC=com

# search result
search: 4
result: 0 Success

# numResponses: 5
# numEntries: 1
# numReferences: 3

//Retrieve Info and Check USERCERTIFICATE attribute to contain user certificate blob
[root@amitkuma ~]# sss_cache -u prasad
[root@amitkuma ~]# sss_cache -E
[root@amitkuma ~]# id prasad
uid=107601139(prasad) gid=107600513(domain users) groups=107600513(domain users)
[root@amitkuma ~]# ldbsearch -H /var/lib/sss/db
# record 4
dn: name=prasad,cn=users,cn=atest.com,cn=sysdb
createTimestamp: 1537864727
fullName: prasad
gecos: prasad
gidNumber: 107600513
name: prasad
objectCategory: user
uidNumber: 107601139
objectSIDString: S-1-5-21-219570999-4236135608-484490645-1139
uniqueID: b2d2b82f-5729-4484-b8a7-2556afa7c243
originalDN: CN=prasad,CN=Users,DC=atest,DC=com
originalModifyTimestamp: 20180925083605.0Z
entryUSN: 643391
userPrincipalName: prasad
adUserAccountControl: 66048
userCertificate:: MIIDBzCCAe+gAwIBAgIBETANBgkqhkiG9w0BAQsFADBnMS8wLQYDVQQKDCZn
 c3NsYWIucG5xMi5yZWRoYXQuY29tIFNlY3VyaXR5IERvbWFpbjETMBEGA1UECwwKcGtpLXRvbWNhd
 DEfMB0GA1UEAwwWQ0EgU2lnbmluZyBDZXJ0aWZpY2F0ZTAeFw0xODA5MjUwODE2MDFaFw0yMzA5Mj
 QwODE2MDFaMDExFzAVBgNVBAoMDlRva2VuIEtleSBVc2VyMRYwFAYKCZImiZPyLGQBAQwGcHJhc2F
 kMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCDuOofqXTcSkOpbHLVRkpnCaveM9RABzLWGT/2
 qwNXvW1a53T2VsBet769s1i7UsZbIjBooOGbYX6Uv/nYS3AphXHDsBozki+J/ecWtxXnUUVMOrKUN
 prViWfp6vRFlziTgfVO7u8qZzrlgdXO4Q0Hw8d9OCIZGEb0jkJ0KGfMJQIDAQABo3gwdjAOBgNVHQ
 8BAf8EBAMCBSAwGQYDVR0RBBIwEIEOJHJlcXVlc3QubWFpbCQwHQYDVR0OBBYEFAGE31jQ0qC8gsP
 eVU/OSr1s1JKwMB8GA1UdIwQYMBaAFIXBFz84oedXcjNaUptamGONm+rPMAkGA1UdEwQCMAAwDQYJ
 KoZIhvcNAQELBQADggEBAKD8mZ77BTO8sA2TT3KYtAyjXJst6uX12PWvtYLkcUmLOJSZZXCPFJzp7
 ld1JqOVXN45p3xiM3QK5VDShjD1AJBKaOrImsJ2rimPQDsP6DX84+A5nduoRGg0haY8LVtDgyOF8F
 ggFUV+61IWNCnEnkOudigIQfDoZUQY8+aNGOEQIi3cIhoNTLKcnAzfzqFWQSwE5lR7Qha725FnmOr
 rs/wPVG02XGH3ms4E2i4rMUTmunDNWpAv1YgpIrDDytcMV8hokqUy2KyAoTNHR/hKgnHwMA7u9iUt
 TDxDdG7us1138l2fbc4oSAUhJQfk1dMnPUUU0YolNY693cGmlm056uI=
nameAlias: prasad
isPosix: TRUE
memberof: name=Domain Users,cn=groups,cn=atest.com,cn=sysdb
initgrExpireTimestamp: 1537870127
lastUpdate: 1537866169
dataExpireTimestamp: 1
distinguishedName: name=prasad,cn=users,cn=atest.com,cn=sysdb

//Check whether SSSD can retrieve public Keys from cached user entry (User certificate)
[root@amitkuma ~]# /usr/bin/sss_ssh_authorizedkeys prasad
[root@amitkuma ~]# 

//tail -f /var/log/sssd/sssd_ssh.log
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [get_client_cred] (0x4000): Client creds: euid[0] egid[0] pid[9737].
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [setup_client_idle_timer] (0x4000): Idle timer re-set for client [0x55b4cb3d2380][18]
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [accept_fd_handler] (0x0400): Client connected!
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [sss_cmd_get_version] (0x0200): Received client version [0].
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [sss_cmd_get_version] (0x0200): Offered version [0].
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [ssh_protocol_parse_request] (0x0400): Requested domain [<ALL>]
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [ssh_cmd_get_user_pubkeys] (0x0400): Requesting SSH user public keys for [prasad] from [<ALL>]
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [cache_req_set_plugin] (0x2000): CR #1: Setting "User by name" plugin
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [cache_req_send] (0x0400): CR #1: New request 'User by name'
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [cache_req_process_input] (0x0400): CR #1: Parsing input name [prasad]
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [sss_parse_name_for_domains] (0x0200): name 'prasad' matched without domain, user is prasad
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [cache_req_set_name] (0x0400): CR #1: Setting name [prasad]
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [cache_req_select_domains] (0x0400): CR #1: Performing a multi-domain search
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [cache_req_search_domains] (0x0400): CR #1: Search will check the cache and check the data provider
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [cache_req_validate_domain_type] (0x2000): Request type POSIX-only for domain atest.com type POSIX is valid
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [cache_req_set_domain] (0x0400): CR #1: Using domain [atest.com]
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [cache_req_prepare_domain_data] (0x0400): CR #1: Preparing input data for domain [atest.com] rules
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [cache_req_search_send] (0x0400): CR #1: Looking up prasad
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [cache_req_search_ncache] (0x0400): CR #1: Checking negative cache for [prasad]
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [sss_ncache_check_str] (0x2000): Checking negative cache for [NCE/USER/atest.com/prasad]
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [cache_req_search_ncache] (0x0400): CR #1: [prasad] is not present in negative cache
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [cache_req_search_cache] (0x0400): CR #1: Looking up [prasad] in cache
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [ldb] (0x4000): Added timed event "ltdb_callback": 0x55b4cb3da4c0

(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [ldb] (0x4000): Added timed event "ltdb_timeout": 0x55b4cb3da580

(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [ldb] (0x4000): Running timer event 0x55b4cb3da4c0 "ltdb_callback"

(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [ldb] (0x4000): Destroying timer event 0x55b4cb3da580 "ltdb_timeout"

(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [ldb] (0x4000): Ending timer event 0x55b4cb3da4c0 "ltdb_callback"

(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [ldb] (0x4000): Added timed event "ltdb_callback": 0x55b4cb3da4c0

(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [ldb] (0x4000): Added timed event "ltdb_timeout": 0x55b4cb3da580

(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [ldb] (0x4000): Running timer event 0x55b4cb3da4c0 "ltdb_callback"

(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [ldb] (0x4000): Destroying timer event 0x55b4cb3da580 "ltdb_timeout"

(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [ldb] (0x4000): Ending timer event 0x55b4cb3da4c0 "ltdb_callback"

(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [cache_req_search_send] (0x0400): CR #1: Returning [prasad] from cache
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [cache_req_search_ncache_filter] (0x0400): CR #1: This request type does not support filtering result by negative cache
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [cache_req_create_and_add_result] (0x0400): CR #1: Found 1 entries in domain atest.com
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [cache_req_done] (0x0400): CR #1: Finished: Success
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [ldb] (0x4000): Added timed event "ltdb_callback": 0x55b4cb3e37d0

(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [ldb] (0x4000): Added timed event "ltdb_timeout": 0x55b4cb3d00a0

(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [ldb] (0x4000): Running timer event 0x55b4cb3e37d0 "ltdb_callback"

(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [ldb] (0x4000): Destroying timer event 0x55b4cb3d00a0 "ltdb_timeout"

(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [ldb] (0x4000): Ending timer event 0x55b4cb3e37d0 "ltdb_callback"

(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [cert_to_ssh_key] (0x0020): CERT_VerifyCertificateNow failed [-8102].
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [get_valid_certs_keys] (0x0040): cert_to_ssh_key failed, ignoring.
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [ssh_protocol_done] (0x4000): Sending reply: success
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [client_recv] (0x0200): Client disconnected!
(Tue Sep 25 15:38:42 2018) [sssd[ssh]] [client_close_fn] (0x2000): Terminated client [0x55b4cb3d2380][18]


Version-Release number of selected component (if applicable):
# rpm -qa | grep sssd
sssd-common-pac-1.16.0-19.el7_5.5.x86_64
sssd-ldap-1.16.0-19.el7_5.5.x86_64
sssd-krb5-1.16.0-19.el7_5.5.x86_64
python-sssdconfig-1.16.0-19.el7_5.5.noarch
sssd-ipa-1.16.0-19.el7_5.5.x86_64
sssd-tools-1.16.0-19.el7_5.5.x86_64
sssd-ad-1.16.0-19.el7_5.5.x86_64
sssd-1.16.0-19.el7_5.5.x86_64
sssd-krb5-common-1.16.0-19.el7_5.5.x86_64
sssd-proxy-1.16.0-19.el7_5.5.x86_64
sssd-dbus-1.16.0-19.el7_5.5.x86_64
redhat-internal-sssd-config-0.3-8.el7.csb.noarch
sssd-common-1.16.0-19.el7_5.5.x86_64
sssd-client-1.16.0-19.el7_5.5.x86_64


How reproducible:
all times in local env

Further Queries:
1. This is sssd checking client certificate on smart-card against CA cert present in nssdb, Can we disable this check?
2. Is KU=Digital Signature required all times in client certificate? Is this critical extension and required all times?


Actual results:
#/usr/bin/sss_ssh_authorizedkeys prasad
does not return public keys

Expected results:
#/usr/bin/sss_ssh_authorizedkeys prasad
Should return public keys

Additional info:
none

Comment 2 Sumit Bose 2018-09-25 13:49:13 UTC
(In reply to amitkuma from comment #0)
> Description of problem:
> Configured Smart card Authentication for active directory user Login using
> Smart card and SSSD.
> 
> Followed:
> https://docs.pagure.org/SSSD.sssd/design_pages/
> smartcard_authentication_testing_with_ad.html
> 

...

> 
> (Tue Sep 25 15:38:42 2018) [sssd[ssh]] [cert_to_ssh_key] (0x0020):
> CERT_VerifyCertificateNow failed [-8102].

In the next RHEL version the SSH responder will use p11_child to verify the certificate which will print the description of the error code as well. For older SSSD versions please use e.g https://www-archive.mozilla.org/projects/security/pki/nss/ref/ssl/sslerr.html to translate the error codes.

> (Tue Sep 25 15:38:42 2018) [sssd[ssh]] [get_valid_certs_keys] (0x0040):
> cert_to_ssh_key failed, ignoring.
> (Tue Sep 25 15:38:42 2018) [sssd[ssh]] [ssh_protocol_done] (0x4000): Sending
> reply: success
> (Tue Sep 25 15:38:42 2018) [sssd[ssh]] [client_recv] (0x0200): Client
> disconnected!
> (Tue Sep 25 15:38:42 2018) [sssd[ssh]] [client_close_fn] (0x2000):
> Terminated client [0x55b4cb3d2380][18]
> 
> 
> Version-Release number of selected component (if applicable):
> # rpm -qa | grep sssd
> sssd-common-pac-1.16.0-19.el7_5.5.x86_64
> sssd-ldap-1.16.0-19.el7_5.5.x86_64
> sssd-krb5-1.16.0-19.el7_5.5.x86_64
> python-sssdconfig-1.16.0-19.el7_5.5.noarch
> sssd-ipa-1.16.0-19.el7_5.5.x86_64
> sssd-tools-1.16.0-19.el7_5.5.x86_64
> sssd-ad-1.16.0-19.el7_5.5.x86_64
> sssd-1.16.0-19.el7_5.5.x86_64
> sssd-krb5-common-1.16.0-19.el7_5.5.x86_64
> sssd-proxy-1.16.0-19.el7_5.5.x86_64
> sssd-dbus-1.16.0-19.el7_5.5.x86_64
> redhat-internal-sssd-config-0.3-8.el7.csb.noarch
> sssd-common-1.16.0-19.el7_5.5.x86_64
> sssd-client-1.16.0-19.el7_5.5.x86_64
> 
> 
> How reproducible:
> all times in local env
> 
> Further Queries:
> 1. This is sssd checking client certificate on smart-card against CA cert
> present in nssdb, Can we disable this check?

Please see the 'no_verification' argument of the certificate_verification option explained in man sssd.conf.

> 2. Is KU=Digital Signature required all times in client certificate? Is this
> critical extension and required all times?

This restriction should be lifted in the next RHEL version as well.

> 
> 
> Actual results:
> #/usr/bin/sss_ssh_authorizedkeys prasad
> does not return public keys
> 
> Expected results:
> #/usr/bin/sss_ssh_authorizedkeys prasad
> Should return public keys
> 
> Additional info:
> none

Comment 3 amitkuma 2018-09-26 06:21:33 UTC
After specifying no_verification.
    [sssd]
    certificate_verification = no_verification 

#  /usr/bin/sss_ssh_authorizedkeys prasad
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAgQCDuOofqXTcSkOpbHLVRkpnCaveM9RABzLWGT/2qwNXvW1a53T2VsBet769s1i7UsZbIjBooOGbYX6Uv/nYS3AphXHDsBozki+J/ecWtxXnUUVMOrKUNprViWfp6vRFlziTgfVO7u8qZzrlgdXO4Q0Hw8d9OCIZGEb0jkJ0KGfMJQ==

But ssh fails.
# ssh -I /usr/lib64/pkcs11/opensc-pkcs11.so prasad@localhost
Enter PIN for 'prasad (prasad)':       <<Entered correct PIN
C_Login failed: 5
sign_and_send_pubkey: signing failed: error in libcrypto
no such identity: /root/.ssh/id_ecdsa: No such file or directory
no such identity: /root/.ssh/id_ed25519: No such file or directory
Password: 

5 means general error.
./src/pkcs11/pkcs11.h:#define CKR_GENERAL_ERROR	(5UL)

I suppose ssh is not able to send public Key to Server?
Though Public key is present in sssd cache entry.

With coolkey
# ssh -I /usr/lib64/pkcs11/libcoolkeypk11.so prasad@localhost
Enter PIN for 'prasad': 
C_Login failed: 179
sign_and_send_pubkey: signing failed: error in libcrypto
no such identity: /root/.ssh/id_ecdsa: No such file or directory
no such identity: /root/.ssh/id_ed25519: No such file or directory
Password:

Comment 4 Sumit Bose 2018-09-26 08:19:34 UTC
If C_Login fails ssh cannot access the private key on the card.

Does login by other means work, e.g. p11toll --provider /usr/lib64/pkcs11/opensc-pkcs11.so --list-all --login?

Does 'ssh -v -v -v ....' show more details?

You might want to enablee debugging for OpenSC in /etc/opensc-x86_64.conf to get more details about the communication with the Smartcard.

Comment 5 amitkuma 2018-09-26 08:48:09 UTC
# p11tool --provider /usr/lib64/pkcs11/opensc-pkcs11.so --list-all --login
Token 'prasad (prasad)' with URL 'pkcs11:model=PKCS%2315%20emulated;manufacturer=4090%20GemAlto%20%28Infineon%29;serial=9061450c01231622;token=prasad%20%28prasad%29' requires user PIN
Enter PIN: 
Object 0:
	URL: pkcs11:model=PKCS%2315%20emulated;manufacturer=4090%20GemAlto%20%28Infineon%29;serial=9061450c01231622;token=prasad%20%28prasad%29;id=%01;object=signing%20key%20for%20prasad;type=private
	Type: Private key
	Label: signing key for prasad
	Flags: CKA_PRIVATE; CKA_SENSITIVE; 
	ID: 01

Object 1:
	URL: pkcs11:model=PKCS%2315%20emulated;manufacturer=4090%20GemAlto%20%28Infineon%29;serial=9061450c01231622;token=prasad%20%28prasad%29;id=%01;object=signing%20key%20for%20prasad;type=public
	Type: Public key
	Label: signing key for prasad
	ID: 01

Object 2:
	URL: pkcs11:model=PKCS%2315%20emulated;manufacturer=4090%20GemAlto%20%28Infineon%29;serial=9061450c01231622;token=prasad%20%28prasad%29;id=%01;object=signing%20key%20for%20prasad;type=cert
	Type: X.509 Certificate
	Label: signing key for prasad
	ID: 01

Object 3:
	URL: pkcs11:model=PKCS%2315%20emulated;manufacturer=4090%20GemAlto%20%28Infineon%29;serial=9061450c01231622;token=prasad%20%28prasad%29;id=%02;object=encryption%20key%20for%20prasad;type=private
	Type: Private key
	Label: encryption key for prasad
	Flags: CKA_WRAP/UNWRAP; CKA_PRIVATE; CKA_SENSITIVE; 
	ID: 02

Object 4:
	URL: pkcs11:model=PKCS%2315%20emulated;manufacturer=4090%20GemAlto%20%28Infineon%29;serial=9061450c01231622;token=prasad%20%28prasad%29;id=%02;object=encryption%20key%20for%20prasad;type=public
	Type: Public key
	Label: encryption key for prasad
	Flags: CKA_WRAP/UNWRAP; 
	ID: 02

Object 5:
	URL: pkcs11:model=PKCS%2315%20emulated;manufacturer=4090%20GemAlto%20%28Infineon%29;serial=9061450c01231622;token=prasad%20%28prasad%29;id=%02;object=encryption%20key%20for%20prasad;type=cert
	Type: X.509 Certificate
	Label: encryption key for prasad
	ID: 02

# 


# ssh -vvv -I /usr/lib64/pkcs11/opensc-pkcs11.so prasad@localhost
OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 60: Applying options for *
debug1: Executing proxy command: exec /usr/bin/sss_ssh_knownhostsproxy -p 22 localhost
debug1: permanently_set_uid: 0/0
debug1: permanently_drop_suid: 0
debug1: provider /usr/lib64/pkcs11/opensc-pkcs11.so: manufacturerID <OpenSC Project> cryptokiVersion 2.20 libraryDescription <OpenSC smartcard framework> libraryVersion 0.16
debug1: provider /usr/lib64/pkcs11/opensc-pkcs11.so slot 0: label <prasad (prasad)> manufacturerID <4090 GemAlto (Infineon)> model <PKCS#15 emulate> serial <9061450c0123162> flags 0x40c
debug1: have 1 keys
debug1: have 2 keys
debug1: pkcs11_provider_unref: 0x55e61ad05b80 refcount 3
debug1: pkcs11_provider_unref: 0x55e61ad05b80 refcount 3
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_rsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4
debug1: match: OpenSSH_7.4 pat OpenSSH* compat 0x04000000
debug2: fd 5 setting O_NONBLOCK
debug2: fd 4 setting O_NONBLOCK
debug1: Authenticating to localhost:22 as 'prasad'
debug3: hostkeys_foreach: reading file "/root/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /root/.ssh/known_hosts:140
debug3: load_hostkeys: loaded 1 keys from localhost
debug3: hostkeys_foreach: reading file "/var/lib/sss/pubconf/known_hosts"
debug3: order_hostkeyalgs: prefer hostkeyalgs: ecdsa-sha2-nistp256-cert-v01,ecdsa-sha2-nistp384-cert-v01,ecdsa-sha2-nistp521-cert-v01,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01,ecdsa-sha2-nistp384-cert-v01,ecdsa-sha2-nistp521-cert-v01,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519-cert-v01,ssh-rsa-cert-v01,ssh-dss-cert-v01,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ssh-dss
debug2: ciphers ctos: chacha20-poly1305,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm,aes256-gcm,aes128-cbc,aes192-cbc,aes256-cbc
debug2: ciphers stoc: chacha20-poly1305,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm,aes256-gcm,aes128-cbc,aes192-cbc,aes256-cbc
debug2: MACs ctos: umac-64-etm,umac-128-etm,hmac-sha2-256-etm,hmac-sha2-512-etm,hmac-sha1-etm,umac-64,umac-128,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm,umac-128-etm,hmac-sha2-256-etm,hmac-sha2-512-etm,hmac-sha1-etm,umac-64,umac-128,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib,zlib
debug2: compression stoc: none,zlib,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm,aes256-gcm,aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc
debug2: ciphers stoc: chacha20-poly1305,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm,aes256-gcm,aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc
debug2: MACs ctos: umac-64-etm,umac-128-etm,hmac-sha2-256-etm,hmac-sha2-512-etm,hmac-sha1-etm,umac-64,umac-128,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm,umac-128-etm,hmac-sha2-256-etm,hmac-sha2-512-etm,hmac-sha1-etm,umac-64,umac-128,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib
debug2: compression stoc: none,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305 MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305 MAC: <implicit> compression: none
debug1: kex: curve25519-sha256 need=64 dh_need=64
debug1: kex: curve25519-sha256 need=64 dh_need=64
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:4eMH4pM5oecTvLcLkOD2TQ6KcPOyg66obgBD48bFi7o
debug3: hostkeys_foreach: reading file "/root/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /root/.ssh/known_hosts:140
debug3: load_hostkeys: loaded 1 keys from localhost
debug3: hostkeys_foreach: reading file "/var/lib/sss/pubconf/known_hosts"
debug1: Host 'localhost' is known and matches the ECDSA host key.
debug1: Found key in /root/.ssh/known_hosts:140
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 134217728 blocks
debug2: key: /usr/lib64/pkcs11/opensc-pkcs11.so (0x55e61ad249e0)
debug2: key: /usr/lib64/pkcs11/opensc-pkcs11.so (0x55e61ad26410)
debug2: key: /root/.ssh/id_rsa ((nil))
debug2: key: /root/.ssh/id_dsa ((nil))
debug2: key: /root/.ssh/id_ecdsa ((nil)), explicit
debug2: key: /root/.ssh/id_ed25519 ((nil)), explicit
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup gssapi-keyex
debug3: remaining preferred: gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_is_enabled gssapi-keyex
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context
debug2: we did not send a packet, disable method
debug3: authmethod_lookup gssapi-with-mic
debug3: remaining preferred: publickey,keyboard-interactive,password
debug3: authmethod_is_enabled gssapi-with-mic
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure.  Minor code may provide more information
Server not found in Kerberos database

debug3: send packet: type 50
debug2: we sent a gssapi-with-mic packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
debug2: we did not send a packet, disable method
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /usr/lib64/pkcs11/opensc-pkcs11.so
debug3: send_pubkey_test
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
debug1: Offering RSA public key: /usr/lib64/pkcs11/opensc-pkcs11.so
debug3: send_pubkey_test
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 60
debug1: Server accepts key: pkalg rsa-sha2-512 blen 151
debug2: input_userauth_pk_ok: fp SHA256:KJa3SdGtNR5YcL3G1CQdt5DcpdWGrKdWv17xEzBbYrY
debug3: sign_and_send_pubkey: RSA SHA256:KJa3SdGtNR5YcL3G1CQdt5DcpdWGrKdWv17xEzBbYrY
Enter PIN for 'prasad (prasad)': 
C_Login failed: 5
sign_and_send_pubkey: signing failed: error in libcrypto
debug1: Trying private key: /root/.ssh/id_rsa
debug3: no such identity: /root/.ssh/id_rsa: No such file or directory
debug1: Trying private key: /root/.ssh/id_dsa
debug3: no such identity: /root/.ssh/id_dsa: No such file or directory
debug1: Trying private key: /root/.ssh/id_ecdsa
no such identity: /root/.ssh/id_ecdsa: No such file or directory
debug1: Trying private key: /root/.ssh/id_ed25519
no such identity: /root/.ssh/id_ed25519: No such file or directory
debug2: we did not send a packet, disable method
debug3: authmethod_lookup keyboard-interactive
debug3: remaining preferred: password
debug3: authmethod_is_enabled keyboard-interactive
debug1: Next authentication method: keyboard-interactive
debug2: userauth_kbdint
debug3: send packet: type 50
debug2: we sent a keyboard-interactive packet, wait for reply
debug3: receive packet: type 60
debug2: input_userauth_info_req
debug2: input_userauth_info_req: num_prompts 1
Password: 

# id prasad
uid=107601139(prasad) gid=107600513(domain users) groups=107600513(domain users)
[root@amitkuma ~]# sss_cache -E
[root@amitkuma ~]# sss_cache -u prasad
[root@amitkuma ~]# id prasad
uid=107601139(prasad) gid=107600513(domain users) groups=107600513(domain users)
#

Comment 6 amitkuma 2018-09-27 13:03:01 UTC
Created attachment 1487778 [details]
opensc debug logs debug=3

Comment 7 amitkuma 2018-10-02 08:43:31 UTC
Hey Sumit.
Any updates!!

Comment 8 Sumit Bose 2018-10-25 14:00:54 UTC
ah, sorry for the delay. I asked Jakub Jelen about the logs and here is what he replied:

"""
Hello Sumit,
in the bug description, I see first issue that both coolkey and OpenSC
pkcs11 modules are in NSS DB, which might cause some problems, but they
should not let the SSH itself fail, as it is using the OpenSC pkcs11
module directly. But first things first. Are the pkcs11 modules needed
in the NSS DB in this use case at all?

The debug log is from three initialization of pkcs11 module from two
places. The first one can be called 0x7faf3b5ee840 by the first column,
and that is probably the ssh process from where we ask for a PIN. But
in the meantime, some other process tries to search something on the
card (sssd through NSS DB?) and it closes its session before we attempt
to login:

pkcs11-session.c:164:C_CloseAllSessions: C_CloseAllSessions(0x0)
slot.c:418:slot_get_token: Slot(id=0x0): get token
slot.c:436:slot_get_token: Slot-get-token returns OK
pkcs11-session.c:129:sc_pkcs11_close_all_sessions: real
C_CloseAllSessions(0x0) 1
pkcs11-session.c:98:sc_pkcs11_close_session: real
C_CloseSession(0x5577988cf7e0)
pkcs11-global.c:304:C_Finalize: C_Finalize()
ctx.c:846:sc_cancel: called
reader-pcsc.c:677:pcsc_cancel: called
slot.c:184:card_removed: Gemalto PC Twin Reader 00 00: card removed

This boils down to two issues:
 * Calling C_Finalize() from one process disconnects the card on PCSC
level, which unpowers the card and it loses all the state information.
This might be worked around (already changed upstream and in RHEL8) by
setting disconnect_action=leave instead of reset or unpower in
opensc.conf.

 * Coolkey cards have additional security measure in form of nonce,
that is passed as part of some APDUs to the card, which is making sure
we talk to the same application and nobody hijacked our "authenticated"
connection. This might not matter until we log in, but might matter
later if these issues will continue.


I would suggest checking what and why is touching the card while
running ssh (sssd checking known hosts initializes NSS DB?), if that is
needed or if some different DB would make sense to be used in that
case.

Later I would suggest changing the configuration option
disconnect_action=leave which should workaround this problem in opensc.

Additionally, if the coolkey cards are used as the only cards, there
might be issues with matching them (we had some bug), so setting
explicitly card_drivers=coolkey,internal might avoid some failures with
matching.

Let me know if you will need some clarifications or if it will help you
to move on with this.
"""

So I would first suggest to try ssh from a different host. If that works than the certificate validation done by SSSD and the login process of the ssh client step on each others toes as Jakub assumes.

bye,
Sumit

Comment 9 amitkuma 2018-10-26 08:10:07 UTC
Thanks Sumit.
Will check your update.

Comment 10 amitkuma 2018-10-29 12:09:43 UTC
||I would suggest checking what and why is touching the card while running ssh (sssd checking known hosts initializes NSS DB?)
I have placed the smart card on my local machine. configured sssd for AD auth. How can I check card being touched while running ssh? There is no different DB i am using.

||After changing /etc/opensc.conf
||disconnect_action = leave;
||card_drivers = coolkey;
# ssh  -I /usr/lib64/pkcs11/opensc-pkcs11.so prasad@localhost
Enter PIN for 'prasad (prasad)': 
C_SignInit failed: 99
sign_and_send_pubkey: signing failed: error in libcrypto
no such identity: /root/.ssh/id_ecdsa: No such file or directory
no such identity: /root/.ssh/id_ed25519: No such file or directory
Password: 

# ssh -I /usr/lib64/pkcs11/libcoolkeypk11.so prasad@localhost
Enter PIN for 'prasad': 
<worked>
[prasad@amitkuma ~]$ 

||So I would first suggest to try ssh from a different host. If that works than the certificate validation done by SSSD and the login process of the ssh client step on each others toes as Jakub assumes.
:( I don't have different machine But I will try to look around, will take some time.

Comment 11 Sumit Bose 2018-10-29 13:20:57 UTC
You can create a new NSSDB with the CA certificate only and add the 'ca_db' option to the [ssh] section of sssd.conf with the path to the new NSSDB.

Comment 12 Jakub Hrozek 2018-11-12 09:03:00 UTC
Does it work now? Can the bug be closed?

Comment 13 amitkuma 2018-11-12 10:13:13 UTC
Dear jakub,
opensc.conf
 card_drivers=coolkey
# ssh -I /usr/lib64/pkcs11/libcoolkeypk11.so prasad@localhost -> Works

I would like to get it work with opensc(Driver).
# ssh  -I /usr/lib64/pkcs11/opensc-pkcs11.so prasad@localhost
Enter PIN for 'prasad (prasad)': 
C_SignInit failed: 99
sign_and_send_pubkey: signing failed: error in libcrypto
no such identity: /root/.ssh/id_ecdsa: No such file or directory
no such identity: /root/.ssh/id_ed25519: No such file or directory
Password: 

Considering Since RHEL-7.4 coolkey deprecated & opensc taken up, 
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/7.4_release_notes/chap-red_hat_enterprise_linux-7.4_release_notes-deprecated_functionality
Getting it work with opensc becomes more weighty.

Sumit:
||You can create a new NSSDB with the CA certificate only and add the 'ca_db' option to the [ssh] section of sssd.conf with the path to the new NSSDB.
IF i am not wrong, this is best habit to be followed. Will is make ssh work with opensc?

Comment 14 Sumit Bose 2018-11-12 16:54:27 UTC
(In reply to amitkuma from comment #13)
...
> Sumit:
> ||You can create a new NSSDB with the CA certificate only and add the
> 'ca_db' option to the [ssh] section of sssd.conf with the path to the new
> NSSDB.
> IF i am not wrong, this is best habit to be followed. Will is make ssh work
> with opensc?

I hope, but you have to try.

Comment 15 amitkuma 2018-12-21 04:23:21 UTC
Hello sumit,

About creating a new NSSDB with the CA certificate only.
- Presently /etc/pki/nssdb only has CA cert.

'ca_db' option to the [ssh] section of sssd.conf
- ca_db defaults to nssdb only.

I think CA certificate present in nssdb is correct and validates user certificate correctly.

]# certutil -L -d /etc/pki/nssdb

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

RHCS CA Cert                                                 CT,C,C <<<<<<<<<<<<<<
# 

# certutil -L -d /etc/pki/nssdb -h all

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

Enter Password or Pin for "prasad (prasad)":
Enter Password or Pin for "prasad":
RHCS CA Cert                                                 CT,C,C     <<<<<<<<<<<
prasad:signing key for prasad                                u,u,u
prasad (prasad):signing key for prasad                       u,u,u
prasad:encryption key for prasad                             u,u,u
prasad (prasad):encryption key for prasad                    u,u,u


I tested using pam_pkcs11. Later I removed pam_pkcs11.
Here "user certificate" present on smart card named "prasad" is validated using CA certificate present in nssdb.


# pklogin_finder debug
DEBUG:pam_config.c:238: Using config file /etc/pam_pkcs11/pam_pkcs11.conf
DEBUG:pkcs11_lib.c:182: Initializing NSS ...
DEBUG:pkcs11_lib.c:192: Initializing NSS ... database=/etc/pki/nssdb <<<<<<<<<<<
DEBUG:pkcs11_lib.c:210: ...  NSS Complete
DEBUG:pklogin_finder.c:71: loading pkcs #11 module...
DEBUG:pkcs11_lib.c:235: Looking up module in list
DEBUG:pkcs11_lib.c:238: modList = 0x1aa3d10 next = 0x1ab2960

DEBUG:pkcs11_lib.c:239: dllName= <null> 

DEBUG:pkcs11_lib.c:238: modList = 0x1ada130 next = 0x0

DEBUG:pkcs11_lib.c:239: dllName= opensc-pkcs11.so            <<<<<<<<<<<<<<<<<<<

DEBUG:pklogin_finder.c:79: initialising pkcs #11 module...
PIN for token: 
DEBUG:pkcs11_lib.c:48: PIN = [prasad]
DEBUG:pkcs11_lib.c:759: cert 0: found (prasad (prasad):signing key for prasad), "UID=prasad,O=Token Key User,CN=prasad"
DEBUG:mapper_mgr.c:172: Retrieveing mapper module list
DEBUG:mapper_mgr.c:73: Loading static module for mapper 'cn'
DEBUG:mapper_mgr.c:197: Inserting mapper [cn] into list
DEBUG:mapper_mgr.c:73: Loading static module for mapper 'uid'
DEBUG:mapper_mgr.c:197: Inserting mapper [uid] into list
DEBUG:mapper_mgr.c:73: Loading static module for mapper 'pwent'
DEBUG:mapper_mgr.c:197: Inserting mapper [pwent] into list
DEBUG:mapper_mgr.c:73: Loading static module for mapper 'null'
DEBUG:mapper_mgr.c:197: Inserting mapper [null] into list
DEBUG:pklogin_finder.c:127: Found '1' certificate(s)
DEBUG:pklogin_finder.c:131: verifing the certificate #1
DEBUG:cert_vfy.c:34: Verifying Cert: prasad (prasad):signing key for prasad (UID=prasad,O=Token Key User,CN=prasad)
DEBUG:pklogin_finder.c:145: Trying to deduce login from certificate
DEBUG:pklogin_finder.c:151: Certificate is valid and maps to user test2   <<<<<<<<<<<<<<<<<<<<<
test2
DEBUG:mapper_mgr.c:214: unloading mapper module list
DEBUG:mapper_mgr.c:137: calling mapper_module_end() cn
DEBUG:mapper_mgr.c:148: Module cn is static: don't remove
DEBUG:mapper_mgr.c:137: calling mapper_module_end() uid
DEBUG:mapper_mgr.c:148: Module uid is static: don't remove
DEBUG:mapper_mgr.c:137: calling mapper_module_end() pwent
DEBUG:mapper_mgr.c:148: Module pwent is static: don't remove
DEBUG:mapper_mgr.c:137: calling mapper_module_end() null
DEBUG:mapper_mgr.c:148: Module null is static: don't remove
DEBUG:pklogin_finder.c:169: releasing pkcs #11 module...
DEBUG:pklogin_finder.c:172: Process completed
[root@amitkuma ~]#

# vim /etc/opensc.conf
card_drivers = opensc,internal;
# ssh -I /usr/lib64/pkcs11/opensc-pkcs11.so prasad@localhost
Enter PIN for 'prasad (prasad)': 
C_SignInit failed: 99
sign_and_send_pubkey: signing failed: error in libcrypto
no such identity: /root/.ssh/id_ecdsa: No such file or directory
no such identity: /root/.ssh/id_ed25519: No such file or directory
prasad@localhost's password:

# vim /etc/opensc.conf
card_drivers = coolkey,internal;
# ssh -I /usr/lib64/pkcs11/libcoolkeypk11.so prasad@localhost
WORKS


So I feel this has to do with coolkey and opensc, not on certificate validation.
Since coolkey has verified certificate successfully and given login prompt.
I will dive more into it.

Comment 16 Sumit Bose 2018-12-21 07:29:07 UTC
The reason I asked to use a separate NSSDB is that Jakub found in the OpenSC logs that it looks like two processes stepping on each others toes. So it might help to let SSSD's ssh responder to use a different NSSDB which does not has OpenSC attached. This way ssh would be the only process using OpenSC at this time.

bye,
Sumit

Comment 17 amitkuma 2018-12-21 07:50:18 UTC
# mkdir new_nssdb
[root@amitkuma ~]# certutil -N -d ./new_nssdb/
Enter a password which will be used to encrypt your keys.
The password should be at least 8 characters long,
and should contain at least one non-alphabetic character.

Enter new password: 
Re-enter password: 

[root@amitkuma ~]# ls -ltr new_nssdb/
total 60
-rw------- 1 root root 16384 Dec 21 13:13 secmod.db
-rw------- 1 root root 16384 Dec 21 13:13 key3.db
-rw------- 1 root root 65536 Dec 21 13:13 cert8.db
[root@amitkuma ~]# certutil -L -d ./new_nssdb/

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

[root@amitkuma ~]# certutil -d ./new_nssdb/ -A -n "RHCS Root CA Cert" -t CT,CT,CT -a -i ./rhcs-ca-latest.crt 

[root@amitkuma ~]# certutil -L -d ./new_nssdb/ -h all

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

RHCS Root CA Cert                                            CT,C,C
[root@amitkuma ~]

sssd.conf
[domain/amitserver.com]
cache_credentials = True
krb5_store_password_if_offline = True
ipa_domain = amitserver.com
id_provider = ipa
auth_provider = ipa
access_provider = ipa
ldap_tls_cacert = /etc/ipa/ca.crt
ipa_hostname = amitkuma.pnq.csb
chpass_provider = ipa
ipa_server = _srv_, rhel7-ipa-2.amitserver.com
dns_discovery_domain = amitserver.com
[sssd]
services = nss, sudo, pam, ssh
certificate_verification = no_verification

domains = amitserver.com
[nss]
homedir_substring = /home

[pam]
pam_cert_auth = True
debug_level = 5

[sudo]
[autofs]

[ssh]
ca_db = /root/new_nssdb

[pac]
[ifp]
[secrets]
[session_recording]

# service sssd restart
# ssh  -I /usr/lib64/pkcs11/opensc-pkcs11.so prasad@localhost
Enter PIN for 'prasad (prasad)': 
C_SignInit failed: 99
sign_and_send_pubkey: signing failed: error in libcrypto
no such identity: /root/.ssh/id_ecdsa: No such file or directory
no such identity: /root/.ssh/id_ed25519: No such file or directory
prasad@localhost's password:

Comment 18 amitkuma 2019-04-10 10:00:42 UTC
any updates here?

Comment 19 Sumit Bose 2020-03-23 16:30:14 UTC
Hi,

iirc Jakub mentioned some time ago in some other context that the key size might cause issue here and that when using a different size e.g. 2048bits for the user key it might work.

I close the ticket here because it is quite old. If you are still seeing similar issues on RHEL8 please open a new one.

bye,
Sumit