RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1905613 - mod_ssl does not like valid certificate chain
Summary: mod_ssl does not like valid certificate chain
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: httpd
Version: 8.3
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: rc
: 8.5
Assignee: Luboš Uhliarik
QA Contact: Branislav Náter
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-12-08 16:13 UTC by Joe Madden
Modified: 2021-11-10 00:52 UTC (History)
4 users (show)

Fixed In Version: httpd-2.4-8050020210712211742.b4937e53
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-11-09 18:43:08 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Server Cert (2.07 KB, text/plain)
2020-12-08 16:13 UTC, Joe Madden
no flags Details
Intermediate (1.57 KB, text/plain)
2020-12-08 16:13 UTC, Joe Madden
no flags Details
root ca (1.20 KB, text/plain)
2020-12-08 16:13 UTC, Joe Madden
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2021:4257 0 None None None 2021-11-09 18:43:28 UTC

Description Joe Madden 2020-12-08 16:13:01 UTC
Created attachment 1737668 [details]
Server Cert

Created attachment 1737668 [details]
Server Cert

Description of problem:
Rebuilding a webserver from redhat 6 to redhat 8, we came across an issue where mod_ssl/httpd does not like a valid certificate chain and will refuse to start.

Version-Release number of selected component (if applicable):
httpd-tools-2.4.37-30.module+el8.3.0+7001+0766b9e7.x86_64
httpd-2.4.37-30.module+el8.3.0+7001+0766b9e7.x86_64
httpd-manual-2.4.37-30.module+el8.3.0+7001+0766b9e7.noarch
redhat-logos-httpd-81.1-1.el8.noarch
rphttpd-filesystem-2.4.37-30.module+el8.3.0+7001+0766b9e7.noarch
mhttpd-devel-2.4.37-30.module+el8.3.0+7001+0766b9e7.x86_64
mod_ssl-2.4.37-30.module+el8.3.0+7001+0766b9e7.x86_64


How reproducible:
Always

Steps to Reproduce:
1. Install httpd
2. Configure SSL like so:
        # SSL config
        SSLEngine On
        SSLOptions +StrictRequire
        SSLVerifyClient none
        SSLProxyEngine Off
        SSLInsecureRenegotiation Off
        SSLHonorCipherOrder On
        SSLProtocol TLSv1.2
        SSLCipherSuite !ADH:!RC4-SHA:+HIGH:!MEDIUM:!LOW:!SSLv2:!EXPORT:ALL
        SSLCertificateFile /etc/httpd/certs/rccstatus.crt
        SSLCertificateKeyFile /etc/httpd/certs/rccstatus.key
        SSLCertificateChainFile /etc/httpd/certs/rfd-intermediate.crt
3. restart httpd - Fails,

Actual results:
error_log:
[Tue Dec 08 15:58:43.641806 2020] [ssl:emerg] [pid 4574:tid 139812245698880] AH02311: Fatal error initialising mod_ssl, exiting. See /var/log/httpd/rfd/ssl_error_log for more information
AH00016: Configuration Failed

virtual host error log:
[Tue Dec 08 15:58:43.641412 2020] [ssl:info] [pid 4574:tid 139812245698880] AH01914: Configuring server www.rccstatus.org.uk:443 for SSL protocol
[Tue Dec 08 15:58:43.641800 2020] [ssl:emerg] [pid 4574:tid 139812245698880] AH01903: Failed to configure CA certificate chain!


Expected results:
apache to restart normaly

Additional info:
I've checked the certificates with an online certitifcate chain checker, openssl and they come back as okay:

certs] openssl verify -CAfile <(cat rfd-ca.crt rfd-intermediate.crt) rccstatus.crt
rccstatus.crt: OK

Attached the certificates - I've tried adding the root ca into the chain, makes no difference. I have other chain certificates working as expected but httpd just doesn't appear to like this one for some reason.

If i comment out the SSLCertificateChainFile the service will start, but ovbisouly does not provide the intemediate to clients which is an issue.

Comment 1 Joe Madden 2020-12-08 16:13:22 UTC
Created attachment 1737669 [details]
Intermediate

Comment 2 Joe Madden 2020-12-08 16:13:43 UTC
Created attachment 1737670 [details]
root ca

Comment 3 Joe Madden 2020-12-08 16:32:23 UTC
Further update to this - I've checked other virtual hosts i had to put on and the follow intermediates fail:


        Issuer: C = BE, O = GlobalSign nv-sa, OU = Root CA, CN = GlobalSign Root CA
        Validity
            Not Before: Feb 20 10:00:00 2014 GMT
            Not After : Feb 20 10:00:00 2024 GMT
        Subject: C = BE, O = GlobalSign nv-sa, CN = GlobalSign Domain Validation CA - SHA256 - G2


         Signature Algorithm: sha256WithRSAEncryption
        Issuer: OU = GlobalSign Root CA - R3, O = GlobalSign, CN = GlobalSign
        Validity
            Not Before: Nov 21 00:00:00 2018 GMT
            Not After : Nov 21 00:00:00 2028 GMT
        Subject: C = BE, O = GlobalSign nv-sa, CN = GlobalSign RSA OV SSL CA 2018
        Subject Public Key Info:

        Issuer: C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
        Validity
            Not Before: Mar  8 12:00:00 2013 GMT
            Not After : Mar  8 12:00:00 2023 GMT
        Subject: C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA

I though maybe it was an issue with root ca, but i think this rules it out.


This one works fine - I can't for the life of me figure out why 

        Issuer: C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
        Validity
            Not Before: Nov  6 12:23:52 2017 GMT
            Not After : Nov  6 12:23:52 2027 GMT
        Subject: C = US, O = DigiCert Inc, OU = www.digicert.com, CN = Thawte RSA CA 2018

Comment 4 Joe Madden 2020-12-09 13:06:36 UTC
I've managed to fix this issue.

At build time we run this to apply the pci-dss profile:

oscap xccdf eval --remediate --fetch-remote-resources --profile pci-dss --report /root/oscap_usgcb_remediation_report.html /usr/share/xml/scap/ssg/content/ssg-rhel8-xccdf.xml

This performs the following remediation which adds a new include into /etc/pki/tls/openssl.cnf:

[ default_modules ]

ssl_conf = ssl_module

[ ssl_module ]

system_default = crypto_policy

[ crypto_policy ]

.include /etc/crypto-policies/back-ends/openssl.config


.include /etc/crypto-policies/back-ends/opensslcnf.config

This is the added line:

.include /etc/crypto-policies/back-ends/openssl.config

which links to :

/etc/crypto-policies/back-ends/openssl.config -> /usr/share/crypto-policies/DEFAULT/openssl.txt


The other include link links to:

/etc/crypto-policies/back-ends/opensslcnf.config -> /usr/share/crypto-policies/DEFAULT/opensslcnf.txt



The Broken addition:

 cat /usr/share/crypto-policies/DEFAULT/openssl.txt
@SECLEVEL=2:kEECDH:kRSA:kEDH:kPSK:kDHEPSK:kECDHEPSK:-aDSS:-3DES:!DES:!RC4:!RC2:!IDEA:-SEED:!eNULL:!aNULL:!MD5:-SHA384:-CAMELLIA:-ARIA:-AESCCM8

the other file:

 cat /usr/share/crypto-policies/DEFAULT/opensslcnf.txt
CipherString = @SECLEVEL=2:kEECDH:kRSA:kEDH:kPSK:kDHEPSK:kECDHEPSK:-aDSS:-3DES:!DES:!RC4:!RC2:!IDEA:-SEED:!eNULL:!aNULL:!MD5:-SHA384:-CAMELLIA:-ARIA:-AESCCM8
Ciphersuites = TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:TLS_AES_128_CCM_SHA256
MinProtocol = TLSv1.2
MaxProtocol = TLSv1.3
SignatureAlgorithms = ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:ed25519:ed448:rsa_pss_pss_sha256:rsa_pss_rsae_sha256:rsa_pss_pss_sha384:rsa_pss_rsae_sha384:rsa_pss_pss_sha512:rsa_pss_rsae_sha512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224:ECDSA+SHA1:RSA+SHA1


I'm guessing either openssl doesn't like the two includes or the broke addition breaks openssl, but in turn that breaks mod_ssl.

Not sure where to report this if i'm honest.

Thanks,

Comment 5 Joe Orton 2020-12-14 09:19:08 UTC
Adding OpenSSL and scap-security-guide owners, not immediately obvious how to triage this.

I'm assuming that the OpenSSL profile applied excludes the signature algorithm used by one of the CAs, but mod_ssl is not reporting enough information to be sure, which itself is a bug we can look at.  Not sure if that configuration is intentional or not, so maybe scap or OpenSSL folks can help?

Comment 6 Joe Madden 2020-12-14 09:22:04 UTC
As the end user, a log entry would have been useful from mod_ssl to say it was rejected because of reasons.

That said, i haven't looked at what the scap configuration has done yet and figured out what it didn't like about the certificate.

Comment 7 Joe Orton 2020-12-14 09:28:47 UTC
Yeah, agreed.  Could you file a support ticket for this please so we can investigate further?

Comment 8 Joe Madden 2020-12-14 09:46:51 UTC
I would but I can't unfortunately where on a self support tier :(

Happy to provide anything you need though if you get someone to contact me.

Thanks,

Comment 9 Joe Orton 2020-12-15 09:54:03 UTC
Fixed upstream: http://svn.apache.org/viewvc?view=revision&revision=1884452

Comment 10 Tomas Mraz 2020-12-15 10:14:46 UTC
@vpolasek The addition of .include /etc/crypto-policies/back-ends/openssl.config is totally wrong. The openssl.config is not supposed to be included, only the opensslcnf.config is.

The weak signature algorithms in certificates can be blocked by both the @SECLEVEL=2 or the signaturealgorithms setting.

Comment 11 Tomas Mraz 2020-12-15 10:19:53 UTC
However unless the broken include of the openssl.config somehow messes things up I do not see any reason why this certificate chain would not be acceptable for the settings in the opensslcnf.config.

Comment 12 Tomas Mraz 2020-12-15 10:23:01 UTC
Can you please try
openssl verify -auth_level 2 -CAfile <(cat rfd-ca.crt rfd-intermediate.crt) rccstatus.crt

Comment 13 Joe Orton 2020-12-15 10:24:43 UTC
There is also a possible spurious failure in this code path due to mod_ssl not clearing the OpenSSL error stack, if mod_md is also in use here.

https://bz.apache.org/bugzilla/show_bug.cgi?id=62880

Comment 14 Tomas Mraz 2020-12-15 11:23:31 UTC
Yeah, that is the most probable culprit - and the incorrect include triggers this.

Comment 16 Joe Madden 2020-12-16 10:59:03 UTC
I checked the certificates for you - They came back as okay on Auth 2

 openssl verify -auth_level 2 -CAfile <(cat rfd-ca.crt rfd-intermediate.crt) rccstatus.crt
rccstatus.crt: OK

Comment 17 Tomas Mraz 2020-12-16 12:48:32 UTC
So yeah, the broken include (which would be otherwise just ignored) triggers the issue from the comment #13.

Comment 37 errata-xmlrpc 2021-11-09 18:43:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: httpd:2.4 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:4257


Note You need to log in before you can comment on or make changes to this bug.