2084639 – ipa cert-request ssl error

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 2084639 - ipa cert-request ssl error

Summary: ipa cert-request ssl error

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 9
Classification:	Red Hat
Component:	pki-core
Sub Component:
Version:	9.1
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	high
Target Milestone:	rc
Target Release:	---
Assignee:	Chris Kelley
QA Contact:	PKI QE
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:	2184520
TreeView+	depends on / blocked

Reported:	2022-05-12 15:01 UTC by Scott Poore
Modified:	2023-04-04 21:30 UTC (History)
CC List:	15 users (show)
Fixed In Version:	pki-core-11.2.0-1.el9
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Clones:	2184520 (view as bug list)
Environment:
Last Closed:	2022-11-15 10:13:02 UTC
Type:	Bug
Target Upstream Version:
Embargoed:
Dependent Products:

Attachments	(Terms of Use)

Links
System	ID	Priority	Status	Summary	Last Updated
Red Hat Issue Tracker	RHCS-3058	None	None	None	2022-06-29 09:44:34 UTC
Red Hat Issue Tracker	RHELPLAN-121898	None	None	None	2022-05-12 17:40:09 UTC
Red Hat Product Errata	RHEA-2022:8053	None	None	None	2022-11-15 10:13:25 UTC

Description Scott Poore 2022-05-12 15:01:36 UTC

Description of problem:

On an IPA server with updated openssl, I see SSL errors running ipa cert-request:

# ipa cert-request ipauser1.csr --principal=ipauser1 --certificate-out=ipauser1.crt
ipa: ERROR: cannot connect to 'https://<hostname>:443/ca/rest/account/login': [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:2633)


Version-Release number of selected component (if applicable):
ipa-server-4.9.8-8.el9.x86_64
openssl-3.0.1-27.el9_0.x86_64
389-ds-base-2.0.14-1.el9.x86_64


How reproducible:
seems consistent

Steps to Reproduce:
1.  Install IPA server:

dnf -y install ipa-server-dns

ipa-server-install \
  --setup-dns \
  --auto-forwarders \
  --allow-zone-overlap \
  --reverse-zone 122.168.192.in-addr.arpa. \
  --no-dnssec-validation \
  --forward-policy first \
  --domain example.test \
  --realm EXAMPLE.TEST \
  --admin-password Secret123 \
  --ds-password Secret123 \
  --unattended

2.  Configure for smart card authentication and issuing certificates from IPA:

echo "Secret123"|kinit admin

ipa-advise config-server-for-smart-card-auth > /tmp/sc_server.sh

sh -x /tmp/sc_server.sh /etc/ipa/ca.crt

3.  Add user: 

ipa user-add ipauser1 --first=f --last=l

4,  Request certificate:

openssl req -new -newkey rsa:2048 -keyout ipauser1.key -nodes -out ipauser1.csr -subj '/CN=ipauser1'

ipa cert-request ipauser1.csr --principal=ipauser1 --certificate-out=ipauser1.crt


Actual results:

returns SSL error shown above.


Expected results:

No error and certificate issued for ipauser1


Additional info:

/var/log/httpd/error_log:

[Thu May 12 10:35:39.517081 2022] [:warn] [pid 27364:tid 27532] [client <IP_ADDRESS>:35574] failed to set perms (3140) on file (/run/ipa/ccaches/admin)!, referer: https://<HOSTNAME>/ipa/xml
[Thu May 12 10:35:39.550945 2022] [ssl:error] [pid 27366:tid 27559] [client <IP_ADDRESS>:35576] AH02039: Certificate Verification: Error (50): application verification failure
[Thu May 12 10:35:39.551526 2022] [wsgi:error] [pid 27360:tid 27649] [remote <IP_ADDRESS>:35574] ipa: INFO: [jsonserver_session] admin: cert_request/1('-----BEGIN CERTIFICATE REQUEST-----\\nMIICWDCCAUACAQAwEzERMA8GA1UEAwwIaXBhdXNlcjEwggEiMA0GCSqGSIb3DQEB\\nAQUAA4IBDwAwggEKAoIBAQCtW+wpIWos7yLfwKrGdVE/sF94LieHeiSUoIUfTD40\\n6i1b2YE6LDsr8MxQWDoPlDtZNLy1KaaNUaMSY9ZBLwSv8VRVTLoD7CAip/FHXkYG\\nVvM+qMnb1AJVlEHIDbQ7HJXXQjSm5I23NtoFShmt4RZYP6/AsYKuhFlagAeYYw2Q\\ntnz1HC6zuNuhpnREs59H//BlyGjg2jXrFAOzQOisUPMBnT+fyG3wj7fTXGcfcYFT\\nkfLs9s0f3jLHCKXMYOY3MBH0aL2Uh4eg/RMmh5moc4LpVS6sI1FloZTpA9siGwTF\\nWT/HLOK/XtQP9cSnFE6i8YyWIyeo3hd1rUVuLrogMGXDAgMBAAGgADANBgkqhkiG\\n9w0BAQsFAAOCAQEAbkJGQszgx09ltyhoLGGsqTT8hXXmbSyHg82iKMc5wvQLfdTT\\njdRrBPHkqsdGvTtJIaA6YKNzN/4J1SzbIROtyriMh+mLglK6H7zFv88ZgaUBDpzR\\nP3sg6aCkr/qhx6BHB+bUT9RXD8+RTmXRhVRoA2loPUF0VlX9SmWQ7zAJF180UaoV\\nZb64QrxIkKREkiqbNIgypiCoNfazMkVXkagfXMF4k6PT0dKgpWZhR8VHoEiDZZ5E\\nmU58ljq86jPbR3cLaIr0cjpSIBx0WLoFKwp/eKKAmbOZt8SC126FG16+ZyX/toa+\\nkU5sRBDiO/ZyH+uC6JNwO+jsoPiuv0XMAPjwpA==\\n-----END CERTIFICATE REQUEST-----\\n', principal='ipauser1', version='2.246'): NetworkError

Comment 3 Alexander Bokovoy 2022-05-12 15:13:58 UTC

Endi, could you please look at the logs?

Comment 4 Endi Sukma Dewata 2022-05-12 16:53:15 UTC

The SSL error seems to be happening when the client tries to connect to port 443 which is owned by IPA, so I'm not sure if this is related to PKI. Or are you saying that the new OpenSSL is having a problem using the SSL cert generated by PKI?

Comment 5 Scott Poore 2022-05-12 17:25:29 UTC

I wasn't able to reproduce the issue yet without running the ipa-advise smart card script from step 2.  So I suspect that did something that broke access.  Maybe the SSL option settings for the WebUI?

Comment 7 Alexander Bokovoy 2022-05-12 18:33:02 UTC

I extracted the actual content from the httpd's error_log and just replaced actual hostname with 'hostname.domain':

 ipa: DEBUG: MISS: Hits 0 Misses 1 Size 1
 ipa: DEBUG: request GET https://hostname.domain:443/ca/rest/account/login
 ipa: DEBUG: request body ''
AH02039: Certificate Verification: Error (50): application verification failure
 ipa: DEBUG: httplib request failed:
 Traceback (most recent call last):
   File "/usr/lib/python3.9/site-packages/ipapython/dogtag.py", line 273, in _httplib_request
     res = conn.getresponse()
   File "/usr/lib64/python3.9/http/client.py", line 1377, in getresponse
     response.begin()
   File "/usr/lib64/python3.9/http/client.py", line 320, in begin
     version, status, reason = self._read_status()
   File "/usr/lib64/python3.9/http/client.py", line 281, in _read_status
     line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
   File "/usr/lib64/python3.9/socket.py", line 704, in readinto
     return self._sock.recv_into(b)
   File "/usr/lib64/python3.9/ssl.py", line 1241, in recv_into
     return self.read(nbytes, buffer)
   File "/usr/lib64/python3.9/ssl.py", line 1099, in read
     return self._sslobj.read(len, buffer)
 ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:2633)
 ipa: DEBUG: WSGI wsgi_execute PublicError: Traceback (most recent call last):
   File "/usr/lib/python3.9/site-packages/ipapython/dogtag.py", line 273, in _httplib_request
     res = conn.getresponse()
   File "/usr/lib64/python3.9/http/client.py", line 1377, in getresponse
     response.begin()
   File "/usr/lib64/python3.9/http/client.py", line 320, in begin
     version, status, reason = self._read_status()
   File "/usr/lib64/python3.9/http/client.py", line 281, in _read_status
     line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
   File "/usr/lib64/python3.9/socket.py", line 704, in readinto
     return self._sock.recv_into(b)
   File "/usr/lib64/python3.9/ssl.py", line 1241, in recv_into
     return self.read(nbytes, buffer)
   File "/usr/lib64/python3.9/ssl.py", line 1099, in read
     return self._sslobj.read(len, buffer)
 ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:2633)

 During handling of the above exception, another exception occurred:

 Traceback (most recent call last):
   File "/usr/lib/python3.9/site-packages/ipaserver/rpcserver.py", line 407, in wsgi_execute
     result = command(*args, **options)
   File "/usr/lib/python3.9/site-packages/ipalib/frontend.py", line 471, in __call__
     return self.__do_call(*args, **options)
   File "/usr/lib/python3.9/site-packages/ipalib/frontend.py", line 499, in __do_call
     ret = self.run(*args, **options)
   File "/usr/lib/python3.9/site-packages/ipalib/frontend.py", line 821, in run
     return self.execute(*args, **options)
   File "/usr/lib/python3.9/site-packages/ipaserver/plugins/cert.py", line 657, in execute
     ca_obj = api.Command.ca_show(ca, all=all, chain=chain)['result']
   File "/usr/lib/python3.9/site-packages/ipalib/frontend.py", line 471, in __call__
     return self.__do_call(*args, **options)
   File "/usr/lib/python3.9/site-packages/ipalib/frontend.py", line 499, in __do_call
     ret = self.run(*args, **options)
   File "/usr/lib/python3.9/site-packages/ipalib/frontend.py", line 821, in run
     return self.execute(*args, **options)
   File "/usr/lib/python3.9/site-packages/ipaserver/plugins/ca.py", line 252, in execute
     msg = set_certificate_attrs(result['result'], options)
   File "/usr/lib/python3.9/site-packages/ipaserver/plugins/ca.py", line 189, in set_certificate_attrs
     with api.Backend.ra_lightweight_ca as ca_api:
   File "/usr/lib/python3.9/site-packages/ipaserver/plugins/dogtag.py", line 1201, in __enter__
     status, resp_headers, _resp_body = dogtag.https_request(
   File "/usr/lib/python3.9/site-packages/ipapython/dogtag.py", line 217, in https_request
     return _httplib_request(
   File "/usr/lib/python3.9/site-packages/ipapython/dogtag.py", line 281, in _httplib_request
     raise NetworkError(uri=uri, error=str(e))
 ipalib.errors.NetworkError: cannot connect to 'https://hostname.domain:443/ca/rest/account/login': [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:2633)

path /ca/rest/account/login is passed through to Tomcat/Dogtag.

According to ssl_request_log, it is TLS 1.3 request:

[12/May/2022:10:46:15 -0400] ip.ad.dr.es - - "GET /ca/rest/account/login HTTP/1.1" 669

In PKI's localhost_access_log.2022-05-12.txt

ip.ad.dr.es - - [12/May/2022:10:46:15 -0400] "GET /ca/rest/account/login HTTP/1.1" 401 669

so I think it is not Apache itself, it is use of RA cert to login to Dogtag. Dogtag logs, though, do not have anything on 10:46 or around, the closes after is 10:57, the closest before is 10:45.

Comment 8 Alexander Bokovoy 2022-05-12 18:46:34 UTC

The TLS 1.3 request part is from my initial investigation, ignore that. TLS 1.3 is for connections terminated by Apache's mod_ssl. /ca/* part is passed through to tomcat and tomcat/pki logs have no mentioning of the actual TLS version.

Looking at IPA RA cert:
2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: CAProcessor: saving authentication token into request:
2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: - userid:
2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: - user:
2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: - sslClientCert:
2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: - uid:
2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: - userdn:
2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: - authTime:
2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: - authMgrInstName:
2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: KeyConstraint: Key algorithnm: RSA
2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: KeyConstraint: Key type: RSA
2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: CAEnrollProfile: Processing enrollment request 7
2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: CAService: Signing cert 0x7
2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: CASigningUnit: Getting algorithm context for SHA256withRSA RSASignatureWithSHA256Digest
2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: CASigningUnit: Signing Certificate
2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: CAService: Storing cert 0x7
2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: LDAPSession: Adding cn=7,ou=certificateRepository, ou=ca,o=ipaca
2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: enrollment reqID 7 fromAgent userID: admin authenticated by certUserDBAuthMgr is completed DN requested: CN=IPA RA,O=EXAMPLE.TEST cert issued serial number: 0x7 time: 79

So it is SHA256withRSA RSASignatureWithSHA256Digest, should be just fine?

Scott, I think we can skip IPA part here and test directly from shell using RA cert:
/var/lib/ipa/ra-agent.pem and /var/lib/ipa/ra-agent.key


What would be show by curl for

  # curl --cert-type PEM --cert /var/lib/ipa/ra-agent.pem --key /var/lib/ipa/ra-agent.key -v https://`hostname`:443/ca/rest/account/login

?

This is what I get with Fedora 35:

# curl --cert-type PEM --cert /var/lib/ipa/ra-agent.pem --key /var/lib/ipa/ra-agent.key -v https://`hostname`:443/ca/rest/account/login
*   Trying 192.168.122.141:443...
* Connected to dc.ipa.test (192.168.122.141) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/pki/tls/certs/ca-bundle.crt
*  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: O=IPA.TEST; CN=dc.ipa.test
*  start date: Apr 25 16:59:11 2022 GMT
*  expire date: Apr 25 16:59:11 2024 GMT
*  subjectAltName: host "dc.ipa.test" matched cert's "dc.ipa.test"
*  issuer: O=IPA.TEST; CN=Certificate Authority
*  SSL certificate verify ok.
> GET /ca/rest/account/login HTTP/1.1
> Host: dc.ipa.test
> User-Agent: curl/7.79.1
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* TLSv1.3 (IN), TLS handshake, Request CERT (13):
* TLSv1.3 (OUT), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS handshake, CERT verify (15):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 200
< Date: Thu, 12 May 2022 18:44:55 GMT
< Server: Apache/2.4.53 (Fedora Linux) OpenSSL/1.1.1n mod_wsgi/4.9.0 Python/3.10 mod_auth_gssapi/1.6.3
< Cache-Control: private
< Set-Cookie: JSESSIONID=65A6936AFF2EB08B1F8D4055CF38BF6A; Path=/ca; Secure; HttpOnly
< Content-Type: application/json
< Content-Length: 198
< Vary: Accept-Encoding
< 
* Connection #0 to host dc.ipa.test left intact
{"id":"ipara","fullName":"ipara","roles":["Certificate Manager Agents","Enterprise ACME Administrators","Registration Manager Agents","Security Domain Administrators"],"Attributes":{"Attribute":[]}}

Comment 9 Alexander Bokovoy 2022-05-12 18:47:41 UTC

Obviously, Fedora 35 is openssl 1.1.1n, not 3.0.1, but the logic stays the same: if we get error here, we need to investigate RA agent cert and its authentication to tomcat's end-point.

Comment 10 Endi Sukma Dewata 2022-05-12 20:35:48 UTC

Jack/Christina, any idea about this?

Alexander/Scott, have any SSL tests been done between OpenSSL and NSS
(without IPA or PKI)?

Comment 11 Scott Poore 2022-05-12 23:32:48 UTC

(In reply to Alexander Bokovoy from comment #8)
> The TLS 1.3 request part is from my initial investigation, ignore that. TLS
> 1.3 is for connections terminated by Apache's mod_ssl. /ca/* part is passed
> through to tomcat and tomcat/pki logs have no mentioning of the actual TLS
> version.
> 
> Looking at IPA RA cert:
> 2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: CAProcessor: saving
> authentication token into request:
> 2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: - userid:
> 2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: - user:
> 2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: - sslClientCert:
> 2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: - uid:
> 2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: - userdn:
> 2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: - authTime:
> 2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: - authMgrInstName:
> 2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: KeyConstraint: Key
> algorithnm: RSA
> 2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: KeyConstraint: Key
> type: RSA
> 2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: CAEnrollProfile:
> Processing enrollment request 7
> 2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: CAService: Signing
> cert 0x7
> 2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: CASigningUnit:
> Getting algorithm context for SHA256withRSA RSASignatureWithSHA256Digest
> 2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: CASigningUnit:
> Signing Certificate
> 2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: CAService: Storing
> cert 0x7
> 2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: LDAPSession: Adding
> cn=7,ou=certificateRepository, ou=ca,o=ipaca
> 2022-05-12 10:28:17 [https-jsse-nio-8443-exec-4] INFO: enrollment reqID 7
> fromAgent userID: admin authenticated by certUserDBAuthMgr is completed DN
> requested: CN=IPA RA,O=EXAMPLE.TEST cert issued serial number: 0x7 time: 79
> 
> So it is SHA256withRSA RSASignatureWithSHA256Digest, should be just fine?
> 
> Scott, I think we can skip IPA part here and test directly from shell using
> RA cert:
> /var/lib/ipa/ra-agent.pem and /var/lib/ipa/ra-agent.key
> 
> 
> What would be show by curl for
> 
>   # curl --cert-type PEM --cert /var/lib/ipa/ra-agent.pem --key
> /var/lib/ipa/ra-agent.key -v https://`hostname`:443/ca/rest/account/login
> 
> ?

I see this:

# curl --cert-type PEM --cert /var/lib/ipa/ra-agent.pem --key /var/lib/ipa/ra-agent.key -v https://`hostname`:443/ca/rest/account/login
*   Trying IP.ADDRESS:443...
* Connected to hostname.domain (IP.ADDRESS) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/pki/tls/certs/ca-bundle.crt
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Unknown (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.2 (IN), TLS header, Unknown (23):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS header, Unknown (23):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.2 (IN), TLS header, Unknown (23):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Unknown (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: O=EXAMPLE.TEST; CN=hostname.domain
*  start date: May 12 17:31:30 2022 GMT
*  expire date: May 12 17:31:30 2024 GMT
*  subjectAltName: host "hostname.domain" matched cert's "hostname.domain"
*  issuer: O=EXAMPLE.TEST; CN=Certificate Authority
*  SSL certificate verify ok.
* TLSv1.2 (OUT), TLS header, Unknown (23):
> GET /ca/rest/account/login HTTP/1.1
> Host: hostname.domain
> User-Agent: curl/7.76.1
> Accept: */*
> 
* TLSv1.2 (IN), TLS header, Unknown (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.2 (IN), TLS header, Unknown (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* TLSv1.2 (IN), TLS header, Unknown (23):
* TLSv1.3 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (OUT), TLS header, Unknown (23):
* TLSv1.3 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS header, Unknown (23):
* TLSv1.3 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS header, Unknown (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS header, Unknown (23):
* TLSv1.3 (IN), TLS alert, handshake failure (552):
* OpenSSL SSL_read: error:0A000410:SSL routines::sslv3 alert handshake failure, errno 0
* Closing connection 0
curl: (56) OpenSSL SSL_read: error:0A000410:SSL routines::sslv3 alert handshake failure, errno 0

Comment 12 Scott Poore 2022-05-16 16:57:28 UTC

Hi Endi, I forgot to answer your question last week.  I do not know offhand if any OpenSSL NSS tests have been run yet without IPA/PKI.  

I did just see today that this is impacting the same smart card tests that ran for gating the sssd update in 9.1.

Jack/Christina,  any ideas?

Comment 19 Florence Blanc-Renaud 2022-06-13 16:45:14 UTC

Hi @spoore 
I followed your reproducer and the issue indeed happens only when the config script config-server-for-smart-card-auth is executed. I could narrow it down to the addition of the directive "SSLOCSPEnable on" in /etc/httpd/conf.d/ssl.conf.
If you comment out this directive, the ipa cert-request command completes successfully.

The client cert used to connect to PKI is the RA cert, stored in /var/lib/ipa/ra-agent.pem. It seems to contain an invalid OCSP URI:

# openssl x509 -noout -text -in /var/lib/ipa/ra-agent.pem
[...]
        X509v3 extensions:
            Authority Information Access: 
                OCSP - URI:http

On a RHEL 9.0 system, the RA cert is generated with the following extension:
       X509v3 extensions:
            Authority Information Access: 
                OCSP - URI:http://ipa-ca.ipa.test/ca/ocsp

We need to understand why PKI doesn't properly generate the cert. IPA make a request to PKI to generate it using the caSubsystemCert profile.

Moving to pki component so that they can help troubleshoot why the cert is generated with an incomplete OCSP extension.

Comment 20 Endi Sukma Dewata 2022-06-13 20:45:20 UTC

It looks like the problem was caused by this change:
https://github.com/dogtagpki/pki/commit/901ba9ca74de0036518a91cedb5742e656398589#diff-fd8e55e8a86873c8bf6074e109d6950ebafd35f5a71001056385c540ca2bd418R757-R758

The code is supposed to split this string into name-value pair:

 Location:http://ipa-ca.example.com/ca/ocsp

In the original code the string was split by the first colon into 2 parts.
In the new code the token was split by all colons into multiple parts,
causing the value to become truncated.

This problem seems to be affecting master, v11.2, and v11.1 branches.

Comment 22 Scott Poore 2022-06-14 21:37:25 UTC

Thanks @frenaud 

I was able to use that and workaround the issue in the tests that were failing.

Comment 33 Scott Poore 2022-07-08 15:45:36 UTC

Verified

Version ::

idm-pki-server-11.2.0-1.el9.noarch

Results ::

[root bug_test]# openssl req -new -newkey rsa:2048 -keyout ipauser1.key -nodes -out ipauser1.csr -subj '/CN=ipauser1'
..+..+.......+...+..+.+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+........+.............+..+.+.....+.+..............+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.............+.....+...+.+...........+.+..............+.+.....+...+.+...+...+...+.....+......+.+..+..........+......+..+...+.+........+...+...+.........+......+......+.+..+.+......+...............+...+..+.....................+....+.....+...+.+..+....+...+...........+.....................+.........+.+..............+....+...+........+....+..+.+.........+...........+...+.......+...+..+.........+.+...+..............+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
.......+...+......+............+...+....+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+..+....+......+........+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....................+.+...+........+...+......+...+.+..............+......+....+......+.....+......+.+..+.......+.....+....+......+......+..+...+.........+.+........+.......................................+.+..+....+.....+.........+.+...............+...+...+..+.......+...+...........+......+...+...+.............+......+...+...+.....+.........+.........+.+........+.+.....+.......+.........+.....+.............+.....+...+.........+......+....+...+........+....+.........+.....+.......+..+.+..+...+...............+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----

[root bug_test]# ipa cert-request ipauser1.csr --principal=ipauser1 --certificate-out=ipauser1.crt
  Issuing CA: ipa
  Certificate: MIIE...osaAPbmabg==
  Subject: CN=ipauser1,O=SMARTCARD.TEST
  Issuer: CN=Certificate Authority,O=SMARTCARD.TEST
  Not Before: Fri Jul 08 15:43:14 2022 UTC
  Not After: Mon Jul 08 15:43:14 2024 UTC
  Serial number: 37
  Serial number (hex): 0x25


Removed the workarounds for the tests and got a good run:

============================= test session starts ==============================
platform linux -- Python 3.9.7, pytest-3.10.1, py-1.11.0, pluggy-1.0.0 -- /usr/bin/python3
cachedir: .pytest_cache
metadata: {'Python': '3.9.7', 'Platform': 'Linux-5.8.18-300.fc33.x86_64-x86_64-with-glibc2.33', 'Packages': {'pytest': '3.10.1', 'py': '1.11.0', 'pluggy': '1.0.0'}, 'Plugins': {'html': '1.22.1', 'metadata': '1.11.0', 'multihost': '3.4'}}
rootdir: /home/jenkins/tews/smartcard, inifile: ipa/automated/pytest.ini
plugins: html-1.22.1, metadata-1.11.0, multihost-3.4
collecting ... collected 170 items / 90 deselected

ipa/automated/test_0001_su.py::TestSuSingleUser::test_0001 PASSED        [  1%]
ipa/automated/test_0001_su.py::TestSuSingleUser::test_0002 PASSED        [  2%]
ipa/automated/test_0001_su.py::TestSuSingleUser::test_0003 PASSED        [  3%]
ipa/automated/test_0001_su.py::TestSuSingleUser::test_0004 PASSED        [  5%]
ipa/automated/test_0001_su.py::TestSuSingleUser::test_0005 PASSED        [  6%]
ipa/automated/test_0001_su.py::TestSuSingleUser::test_0006 PASSED        [  7%]
ipa/automated/test_0001_su.py::TestSuSingleUser::test_0007 PASSED        [  8%]
ipa/automated/test_0001_su.py::TestSuSingleUser::test_0008 PASSED        [ 10%]
ipa/automated/test_0001_su.py::TestSuSingleUser::test_0010 PASSED        [ 11%]
ipa/automated/test_0001_su.py::TestSuSingleUser::test_0011 PASSED        [ 12%]
ipa/automated/test_0001_su.py::TestSuSingleUser::test_0012 PASSED        [ 13%]
ipa/automated/test_0001_su.py::TestSuSingleUser::test_0013 PASSED        [ 15%]
ipa/automated/test_0001_su.py::TestSuMultiUser::test_0001 PASSED         [ 16%]
ipa/automated/test_0001_su.py::TestSuMultiUser::test_0002 PASSED         [ 17%]
ipa/automated/test_0001_su.py::TestSuMultiUser::test_0003 PASSED         [ 18%]
ipa/automated/test_0001_su.py::TestSuMultiUser::test_0004 PASSED         [ 20%]
ipa/automated/test_0001_su.py::TestSuMultiUser::test_0005 PASSED         [ 21%]
ipa/automated/test_0001_su.py::TestSuMultiUser::test_0006 PASSED         [ 22%]
ipa/automated/test_0001_su.py::TestSuMultiUser::test_0007 PASSED         [ 23%]
ipa/automated/test_0001_su.py::TestSuMultiUser::test_0008 PASSED         [ 25%]
ipa/automated/test_0001_su.py::TestSuMultiUser::test_0009 PASSED         [ 26%]
ipa/automated/test_0001_su.py::TestSuMultiUser::test_0010 PASSED         [ 27%]
ipa/automated/test_0001_su.py::TestSuMultiUser::test_0011 PASSED         [ 28%]
ipa/automated/test_0001_su.py::TestSuMultiUser::test_0012 PASSED         [ 30%]
ipa/automated/test_0001_su.py::TestSuMultiUser::test_0013 PASSED         [ 31%]
ipa/automated/test_0001_su.py::TestSuMultiUser::test_0014 PASSED         [ 32%]
ipa/automated/test_0001_su.py::TestSuMultiUser::test_0017 PASSED         [ 33%]
ipa/automated/test_0001_su.py::TestSuMultiUser::test_0018 PASSED         [ 35%]
ipa/automated/test_0001_su.py::TestSuMultiUser::test_0019 PASSED         [ 36%]
ipa/automated/test_0001_su.py::TestSuMultiUser::test_0020 PASSED         [ 37%]
ipa/automated/test_0003_webui.py::TestWebUISingleUser::test_0001 PASSED  [ 38%]
ipa/automated/test_0003_webui.py::TestWebUISingleUser::test_0002 PASSED  [ 40%]
ipa/automated/test_0003_webui.py::TestWebUISingleUser::test_0003 PASSED  [ 41%]
ipa/automated/test_0003_webui.py::TestWebUISingleUser::test_0004 PASSED  [ 42%]
ipa/automated/test_0003_webui.py::TestWebUISingleUser::test_0005 PASSED  [ 43%]
ipa/automated/test_0003_webui.py::TestWebUISingleUser::test_0006 PASSED  [ 45%]
ipa/automated/test_0003_webui.py::TestWebUISingleUser::test_0011 PASSED  [ 46%]
ipa/automated/test_0003_webui.py::TestWebUISingleUser::test_0012 PASSED  [ 47%]
ipa/automated/test_0003_webui.py::TestWebUISingleUser::test_0013 PASSED  [ 48%]
ipa/automated/test_0003_webui.py::TestWebUIMultiUser::test_0001 PASSED   [ 50%]
ipa/automated/test_0003_webui.py::TestWebUIMultiUser::test_0002 PASSED   [ 51%]
ipa/automated/test_0003_webui.py::TestWebUIMultiUser::test_0003 PASSED   [ 52%]
ipa/automated/test_0003_webui.py::TestWebUIMultiUser::test_0004 PASSED   [ 53%]
ipa/automated/test_0003_webui.py::TestWebUIMultiUser::test_0005 PASSED   [ 55%]
ipa/automated/test_0003_webui.py::TestWebUIMultiUser::test_0006 PASSED   [ 56%]
ipa/automated/test_0003_webui.py::TestWebUIMultiUser::test_0011 PASSED   [ 57%]
ipa/automated/test_0003_webui.py::TestWebUIMultiUser::test_0012 PASSED   [ 58%]
ipa/automated/test_0003_webui.py::TestWebUIMultiUser::test_0014 PASSED   [ 60%]
ipa/automated/test_0003_webui.py::TestWebUIMultiUser::test_0015 PASSED   [ 61%]
ipa/automated/test_0003_webui.py::TestWebUIMultiUser::test_0018 PASSED   [ 62%]
ipa/automated/test_0003_webui.py::TestWebUIMultiUser::test_0019 PASSED   [ 63%]
ipa/automated/test_0003_webui.py::TestWebUIMultiUser::test_0020 PASSED   [ 65%]
ipa/automated/test_0003_webui.py::TestWebUIMultiUser::test_0021 PASSED   [ 66%]
ipa/automated/test_0003_webui.py::TestWebUIMultiUser::test_0022 PASSED   [ 67%]
ipa/automated/test_0003_webui.py::TestWebUIMultiUser::test_0023 PASSED   [ 68%]
ipa/automated/test_0005_ssh.py::TestSshSingleUser::test_0001 PASSED      [ 70%]
ipa/automated/test_0005_ssh.py::TestSshSingleUser::test_0002 PASSED      [ 71%]
ipa/automated/test_0005_ssh.py::TestSshSingleUser::test_0003 PASSED      [ 72%]
ipa/automated/test_0005_ssh.py::TestSshSingleUser::test_0004 PASSED      [ 73%]
ipa/automated/test_0005_ssh.py::TestSshSingleUser::test_0005 PASSED      [ 75%]
ipa/automated/test_0005_ssh.py::TestSshSingleUser::test_0006 PASSED      [ 76%]
ipa/automated/test_0005_ssh.py::TestSshSingleUser::test_0007 PASSED      [ 77%]
ipa/automated/test_0005_ssh.py::TestSshSingleUser::test_0008 PASSED      [ 78%]
ipa/automated/test_0005_ssh.py::TestSshSingleUser::test_0009 PASSED      [ 80%]
ipa/automated/test_0005_ssh.py::TestSshSingleUser::test_0010 PASSED      [ 81%]
ipa/automated/test_0005_ssh.py::TestSshMultiUser::test_0001 PASSED       [ 82%]
ipa/automated/test_0005_ssh.py::TestSshMultiUser::test_0002 PASSED       [ 83%]
ipa/automated/test_0005_ssh.py::TestSshMultiUser::test_0003 PASSED       [ 85%]
ipa/automated/test_0005_ssh.py::TestSshMultiUser::test_0004 PASSED       [ 86%]
ipa/automated/test_0005_ssh.py::TestSshMultiUser::test_0005 PASSED       [ 87%]
ipa/automated/test_0005_ssh.py::TestSshMultiUser::test_0006 PASSED       [ 88%]
ipa/automated/test_0005_ssh.py::TestSshMultiUser::test_0007 PASSED       [ 90%]
ipa/automated/test_0005_ssh.py::TestSshMultiUser::test_0008 PASSED       [ 91%]
ipa/automated/test_0005_ssh.py::TestSshMultiUser::test_0009 PASSED       [ 92%]
ipa/automated/test_0005_ssh.py::TestSshMultiUser::test_0010 PASSED       [ 93%]
ipa/automated/test_0005_ssh.py::TestSshMultiUser::test_0011 PASSED       [ 95%]
ipa/automated/test_0005_ssh.py::TestSshMultiUser::test_0012 PASSED       [ 96%]
ipa/automated/test_0005_ssh.py::TestSshMultiUser::test_0013 PASSED       [ 97%]
ipa/automated/test_0005_ssh.py::TestSshMultiUser::test_0014 PASSED       [ 98%]
ipa/automated/test_0005_ssh.py::TestSshMultiUser::test_0015 PASSED       [100%]

---------- generated xml file: /home/jenkins/tews/scbz91pki/junit.xml ----------
----- generated html file: file:///home/jenkins/tews/scbz91pki/report.html -----
================= 80 passed, 90 deselected in 2878.30 seconds ==================

Comment 40 errata-xmlrpc 2022-11-15 10:13:02 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (pki-core bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2022:8053

Note You need to log in before you can comment on or make changes to this bug.