Bug 1905613 - mod_ssl does not like valid certificate chain
Summary: mod_ssl does not like valid certificate chain
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: httpd
Version: 8.3
Hardware: x86_64
OS: Linux
Target Milestone: rc
: 8.5
Assignee: Luboš Uhliarik
QA Contact: Branislav Náter
Depends On:
TreeView+ depends on / blocked
Reported: 2020-12-08 16:13 UTC by Joe Madden
Modified: 2021-11-10 00:52 UTC (History)
4 users (show)

Fixed In Version: httpd-2.4-8050020210712211742.b4937e53
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2021-11-09 18:43:08 UTC
Type: Bug
Target Upstream Version:

Attachments (Terms of Use)
Server Cert (2.07 KB, text/plain)
2020-12-08 16:13 UTC, Joe Madden
no flags Details
Intermediate (1.57 KB, text/plain)
2020-12-08 16:13 UTC, Joe Madden
no flags Details
root ca (1.20 KB, text/plain)
2020-12-08 16:13 UTC, Joe Madden
no flags Details

System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2021:4257 0 None None None 2021-11-09 18:43:28 UTC

Description Joe Madden 2020-12-08 16:13:01 UTC
Created attachment 1737668 [details]
Server Cert

Created attachment 1737668 [details]
Server Cert

Description of problem:
Rebuilding a webserver from redhat 6 to redhat 8, we came across an issue where mod_ssl/httpd does not like a valid certificate chain and will refuse to start.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Install httpd
2. Configure SSL like so:
        # SSL config
        SSLEngine On
        SSLOptions +StrictRequire
        SSLVerifyClient none
        SSLProxyEngine Off
        SSLInsecureRenegotiation Off
        SSLHonorCipherOrder On
        SSLProtocol TLSv1.2
        SSLCertificateFile /etc/httpd/certs/rccstatus.crt
        SSLCertificateKeyFile /etc/httpd/certs/rccstatus.key
        SSLCertificateChainFile /etc/httpd/certs/rfd-intermediate.crt
3. restart httpd - Fails,

Actual results:
[Tue Dec 08 15:58:43.641806 2020] [ssl:emerg] [pid 4574:tid 139812245698880] AH02311: Fatal error initialising mod_ssl, exiting. See /var/log/httpd/rfd/ssl_error_log for more information
AH00016: Configuration Failed

virtual host error log:
[Tue Dec 08 15:58:43.641412 2020] [ssl:info] [pid 4574:tid 139812245698880] AH01914: Configuring server www.rccstatus.org.uk:443 for SSL protocol
[Tue Dec 08 15:58:43.641800 2020] [ssl:emerg] [pid 4574:tid 139812245698880] AH01903: Failed to configure CA certificate chain!

Expected results:
apache to restart normaly

Additional info:
I've checked the certificates with an online certitifcate chain checker, openssl and they come back as okay:

certs] openssl verify -CAfile <(cat rfd-ca.crt rfd-intermediate.crt) rccstatus.crt
rccstatus.crt: OK

Attached the certificates - I've tried adding the root ca into the chain, makes no difference. I have other chain certificates working as expected but httpd just doesn't appear to like this one for some reason.

If i comment out the SSLCertificateChainFile the service will start, but ovbisouly does not provide the intemediate to clients which is an issue.

Comment 1 Joe Madden 2020-12-08 16:13:22 UTC
Created attachment 1737669 [details]

Comment 2 Joe Madden 2020-12-08 16:13:43 UTC
Created attachment 1737670 [details]
root ca

Comment 3 Joe Madden 2020-12-08 16:32:23 UTC
Further update to this - I've checked other virtual hosts i had to put on and the follow intermediates fail:

        Issuer: C = BE, O = GlobalSign nv-sa, OU = Root CA, CN = GlobalSign Root CA
            Not Before: Feb 20 10:00:00 2014 GMT
            Not After : Feb 20 10:00:00 2024 GMT
        Subject: C = BE, O = GlobalSign nv-sa, CN = GlobalSign Domain Validation CA - SHA256 - G2

         Signature Algorithm: sha256WithRSAEncryption
        Issuer: OU = GlobalSign Root CA - R3, O = GlobalSign, CN = GlobalSign
            Not Before: Nov 21 00:00:00 2018 GMT
            Not After : Nov 21 00:00:00 2028 GMT
        Subject: C = BE, O = GlobalSign nv-sa, CN = GlobalSign RSA OV SSL CA 2018
        Subject Public Key Info:

        Issuer: C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
            Not Before: Mar  8 12:00:00 2013 GMT
            Not After : Mar  8 12:00:00 2023 GMT
        Subject: C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA

I though maybe it was an issue with root ca, but i think this rules it out.

This one works fine - I can't for the life of me figure out why 

        Issuer: C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
            Not Before: Nov  6 12:23:52 2017 GMT
            Not After : Nov  6 12:23:52 2027 GMT
        Subject: C = US, O = DigiCert Inc, OU = www.digicert.com, CN = Thawte RSA CA 2018

Comment 4 Joe Madden 2020-12-09 13:06:36 UTC
I've managed to fix this issue.

At build time we run this to apply the pci-dss profile:

oscap xccdf eval --remediate --fetch-remote-resources --profile pci-dss --report /root/oscap_usgcb_remediation_report.html /usr/share/xml/scap/ssg/content/ssg-rhel8-xccdf.xml

This performs the following remediation which adds a new include into /etc/pki/tls/openssl.cnf:

[ default_modules ]

ssl_conf = ssl_module

[ ssl_module ]

system_default = crypto_policy

[ crypto_policy ]

.include /etc/crypto-policies/back-ends/openssl.config

.include /etc/crypto-policies/back-ends/opensslcnf.config

This is the added line:

.include /etc/crypto-policies/back-ends/openssl.config

which links to :

/etc/crypto-policies/back-ends/openssl.config -> /usr/share/crypto-policies/DEFAULT/openssl.txt

The other include link links to:

/etc/crypto-policies/back-ends/opensslcnf.config -> /usr/share/crypto-policies/DEFAULT/opensslcnf.txt

The Broken addition:

 cat /usr/share/crypto-policies/DEFAULT/openssl.txt

the other file:

 cat /usr/share/crypto-policies/DEFAULT/opensslcnf.txt
Ciphersuites = TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:TLS_AES_128_CCM_SHA256
MinProtocol = TLSv1.2
MaxProtocol = TLSv1.3
SignatureAlgorithms = ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:ed25519:ed448:rsa_pss_pss_sha256:rsa_pss_rsae_sha256:rsa_pss_pss_sha384:rsa_pss_rsae_sha384:rsa_pss_pss_sha512:rsa_pss_rsae_sha512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224:ECDSA+SHA1:RSA+SHA1

I'm guessing either openssl doesn't like the two includes or the broke addition breaks openssl, but in turn that breaks mod_ssl.

Not sure where to report this if i'm honest.


Comment 5 Joe Orton 2020-12-14 09:19:08 UTC
Adding OpenSSL and scap-security-guide owners, not immediately obvious how to triage this.

I'm assuming that the OpenSSL profile applied excludes the signature algorithm used by one of the CAs, but mod_ssl is not reporting enough information to be sure, which itself is a bug we can look at.  Not sure if that configuration is intentional or not, so maybe scap or OpenSSL folks can help?

Comment 6 Joe Madden 2020-12-14 09:22:04 UTC
As the end user, a log entry would have been useful from mod_ssl to say it was rejected because of reasons.

That said, i haven't looked at what the scap configuration has done yet and figured out what it didn't like about the certificate.

Comment 7 Joe Orton 2020-12-14 09:28:47 UTC
Yeah, agreed.  Could you file a support ticket for this please so we can investigate further?

Comment 8 Joe Madden 2020-12-14 09:46:51 UTC
I would but I can't unfortunately where on a self support tier :(

Happy to provide anything you need though if you get someone to contact me.


Comment 9 Joe Orton 2020-12-15 09:54:03 UTC
Fixed upstream: http://svn.apache.org/viewvc?view=revision&revision=1884452

Comment 10 Tomas Mraz 2020-12-15 10:14:46 UTC
@vpolasek The addition of .include /etc/crypto-policies/back-ends/openssl.config is totally wrong. The openssl.config is not supposed to be included, only the opensslcnf.config is.

The weak signature algorithms in certificates can be blocked by both the @SECLEVEL=2 or the signaturealgorithms setting.

Comment 11 Tomas Mraz 2020-12-15 10:19:53 UTC
However unless the broken include of the openssl.config somehow messes things up I do not see any reason why this certificate chain would not be acceptable for the settings in the opensslcnf.config.

Comment 12 Tomas Mraz 2020-12-15 10:23:01 UTC
Can you please try
openssl verify -auth_level 2 -CAfile <(cat rfd-ca.crt rfd-intermediate.crt) rccstatus.crt

Comment 13 Joe Orton 2020-12-15 10:24:43 UTC
There is also a possible spurious failure in this code path due to mod_ssl not clearing the OpenSSL error stack, if mod_md is also in use here.


Comment 14 Tomas Mraz 2020-12-15 11:23:31 UTC
Yeah, that is the most probable culprit - and the incorrect include triggers this.

Comment 16 Joe Madden 2020-12-16 10:59:03 UTC
I checked the certificates for you - They came back as okay on Auth 2

 openssl verify -auth_level 2 -CAfile <(cat rfd-ca.crt rfd-intermediate.crt) rccstatus.crt
rccstatus.crt: OK

Comment 17 Tomas Mraz 2020-12-16 12:48:32 UTC
So yeah, the broken include (which would be otherwise just ignored) triggers the issue from the comment #13.

Comment 37 errata-xmlrpc 2021-11-09 18:43:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: httpd:2.4 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.