Bug 2117265

Summary: Clarify FUTURE crypto policy support
Product: Red Hat Satellite Reporter: Jan Senkyrik <jsenkyri>
Component: SecurityAssignee: Eric Helms <ehelms>
Status: CLOSED CURRENTRELEASE QA Contact: Satellite QE Team <sat-qe-bz-list>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.11.0CC: ahumbe, dsinglet, ehelms, ekohlvan, ewoud+redhat, gpayelka, jbreitwe, lzap, marco.verschuur, mhulan
Target Milestone: 6.14.0Keywords: Documentation, FutureFeature
Target Release: UnusedFlags: marco.verschuur: needinfo?
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-08-07 12:15:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2166839    

Description Jan Senkyrik 2022-08-10 13:05:36 UTC
Description of problem:

Red Hat supports FUTURE cryptographic policy on RHEL 8 [0].

Ideally we should also support FUTURE crypto policy on Satellite itself (6.11+ running on RHEL 8).


[0] https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/security_hardening/using-the-system-wide-cryptographic-policies_security-hardening#system-wide-crypto-policies_using-the-system-wide-cryptographic-policies

Comment 1 Marco Verschuur 2022-08-11 07:45:42 UTC
To be able to support the FUTURE crypto policy we only need to extend the default cypher block with the 'stronger' cipher suits.
So this doesn't change any existing behavior, other than it now supports FUTURE crypto policy (which considers SHA1 unsafe).

This is already working in my Satellite 6.11.0 / RHEL 8.6 setup, so we would really appreciate if this change can be pushed through,
so we're running a fully supported setup again and also a more compliant setup i.r.t. hardening.

               ciphers="SSL_RSA_WITH_3DES_EDE_CBC_SHA,
                    TLS_RSA_WITH_AES_256_CBC_SHA,
                    TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA,
                    TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA,
                    TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA,
                    TLS_ECDH_RSA_WITH_AES_128_CBC_SHA,
                    TLS_ECDH_RSA_WITH_AES_256_CBC_SHA,
                    TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
                    TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,      <== non-SHA1 strong cipher
                    TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,    <== non-SHA1 strong cipher
                    TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384,       <== non-SHA1 strong cipher
                    TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384,     <== non-SHA1 strong cipher
                    TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,      <== non-SHA1 strong cipher
                    TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,    <== non-SHA1 strong cipher
                    TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256,       <== non-SHA1 strong cipher
                    TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256" />  <== non-SHA1 strong cipher

Comment 2 Ewoud Kohl van Wijngaarden 2022-10-17 10:30:12 UTC
(In reply to Marco Verschuur from comment #1)
> To be able to support the FUTURE crypto policy we only need to extend the
> default cypher block with the 'stronger' cipher suits.
> So this doesn't change any existing behavior, other than it now supports
> FUTURE crypto policy (which considers SHA1 unsafe).
> 
> This is already working in my Satellite 6.11.0 / RHEL 8.6 setup, so we would
> really appreciate if this change can be pushed through,
> so we're running a fully supported setup again and also a more compliant
> setup i.r.t. hardening.
> 
>                ciphers="SSL_RSA_WITH_3DES_EDE_CBC_SHA,
>                     TLS_RSA_WITH_AES_256_CBC_SHA,
>                     TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA,
>                     TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA,
>                     TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA,
>                     TLS_ECDH_RSA_WITH_AES_128_CBC_SHA,
>                     TLS_ECDH_RSA_WITH_AES_256_CBC_SHA,
>                     TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
>                     TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,      <== non-SHA1
> strong cipher
>                     TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,    <== non-SHA1
> strong cipher
>                     TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384,       <== non-SHA1
> strong cipher
>                     TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384,     <== non-SHA1
> strong cipher
>                     TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,      <== non-SHA1
> strong cipher
>                     TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,    <== non-SHA1
> strong cipher
>                     TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256,       <== non-SHA1
> strong cipher
>                     TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256" />  <== non-SHA1
> strong cipher

From the 'ciphers="' bit and the default values I guess you're talking about Candlepin, correct?

If so, you can modify the ciphers using custom-hiera.yaml:

candlepin::ciphers:
  - cipher1
  - cipher2

However, as an installer maintainer I think this should be done out of the box. There's no good reason we're not listing that. Way back in 2017 the existing values were copied when it was made a parameter (https://github.com/theforeman/puppet-candlepin/commit/c5a36f728cc12443709d0437b205c4a9e32c0fbe) and those ciphers were from the initial commit back in 2013 (https://github.com/theforeman/puppet-candlepin/commit/832bafa66c9fdc8d632908613695691e90f78583). I'm slightly surprised we never caught this earlier since we do run nmap to determine cipher strength in our CI (https://github.com/theforeman/puppet-candlepin/blob/cd10bbb81e82c8da770763abb54edcf30bbc2d3c/spec/acceptance/basic_candlepin_spec.rb#L16-L30).

Using your list I at least opened https://github.com/theforeman/puppet-candlepin/pull/224 but I'd like to investigate if we can use the system profile instead of maintaining our own list.

Comment 3 Marco Verschuur 2022-10-17 15:21:33 UTC
Hi Ewoud,

Yes, this is the Candlepin tomcat instance, that refuses to start while having crypto policy FUTURE active.
But after we fixed the ciphers we did ran into another issue... :-)
The sync was no longer working anymore.

And that was due to the "Entitlement Master CA" of cdn.redhat.com being a "PKCS #1 SHA-1 With RSA Encryption" Certificate Signature Algorithm

Comment 4 Ewoud Kohl van Wijngaarden 2022-10-19 13:32:35 UTC
(In reply to Marco Verschuur from comment #3)
> Yes, this is the Candlepin tomcat instance, that refuses to start while
> having crypto policy FUTURE active.

Thanks! I'll note that https://github.com/theforeman/puppet-candlepin/pull/224 was merged, which should naturally flow into Satellite 6.13. I did end up enhancing the cipher list to replace all SHA1 instead of only adding additional ciphers. Until then you'll need override it in custom-hiera.yaml as noted earlier.

> But after we fixed the ciphers we did ran into another issue... :-)
> The sync was no longer working anymore.
> 
> And that was due to the "Entitlement Master CA" of cdn.redhat.com being a
> "PKCS #1 SHA-1 With RSA Encryption" Certificate Signature Algorithm

I've opened a ticket with the CDN team to inquire about this.

Comment 5 Ewoud Kohl van Wijngaarden 2022-10-31 13:15:02 UTC
(In reply to Marco Verschuur from comment #3)
> And that was due to the "Entitlement Master CA" of cdn.redhat.com being a
> "PKCS #1 SHA-1 With RSA Encryption" Certificate Signature Algorithm

Documenting how to check this. Using openssl on Fedora 36:

$ openssl s_client -connect cdn.redhat.com:443
CONNECTED(00000003)
depth=2 C = US, ST = North Carolina, L = Raleigh, O = "Red Hat, Inc.", OU = Red Hat Network, CN = Entitlement Master CA, emailAddress = ca-support
verify error:num=19:self-signed certificate in certificate chain
verify return:1
depth=2 C = US, ST = North Carolina, L = Raleigh, O = "Red Hat, Inc.", OU = Red Hat Network, CN = Entitlement Master CA, emailAddress = ca-support
verify return:1
depth=1 C = US, ST = North Carolina, O = "Red Hat, Inc.", OU = Red Hat Network, CN = Red Hat Entitlement Operations Authority, emailAddress = ca-support
verify return:1
depth=0 C = US, ST = North Carolina, O = "Red Hat, Inc.", OU = Red Hat Network, CN = cdn.redhat.com
verify return:1
---
Certificate chain
 0 s:C = US, ST = North Carolina, O = "Red Hat, Inc.", OU = Red Hat Network, CN = cdn.redhat.com
   i:C = US, ST = North Carolina, O = "Red Hat, Inc.", OU = Red Hat Network, CN = Red Hat Entitlement Operations Authority, emailAddress = ca-support
   a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA256
   v:NotBefore: Sep 26 12:18:10 2022 GMT; NotAfter: Sep 25 12:18:10 2026 GMT
 1 s:C = US, ST = North Carolina, O = "Red Hat, Inc.", OU = Red Hat Network, CN = Red Hat Entitlement Operations Authority, emailAddress = ca-support
   i:C = US, ST = North Carolina, L = Raleigh, O = "Red Hat, Inc.", OU = Red Hat Network, CN = Entitlement Master CA, emailAddress = ca-support
   a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA384
   v:NotBefore: Sep 12 18:13:21 2018 GMT; NotAfter: Mar 15 18:13:21 2030 GMT
 2 s:C = US, ST = North Carolina, L = Raleigh, O = "Red Hat, Inc.", OU = Red Hat Network, CN = Entitlement Master CA, emailAddress = ca-support
   i:C = US, ST = North Carolina, L = Raleigh, O = "Red Hat, Inc.", OU = Red Hat Network, CN = Entitlement Master CA, emailAddress = ca-support
   a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA1
   v:NotBefore: Mar 17 19:00:44 2010 GMT; NotAfter: Mar 12 19:00:44 2030 GMT
---

Note how the last certificate has sigalg: RSA-SHA1. Quoting https://www.redhat.com/en/blog/consistent-security-crypto-policies-red-hat-enterprise-linux-8 on the FUTURE policy:

> This level does not allow the use of SHA-1 in signature algorithms.

It should be noted that openssl on EL8 isn't as verbose on the Certificate Chain.

Comment 7 Ewoud Kohl van Wijngaarden 2023-01-16 14:10:36 UTC
In some discussion with the CDN team it was pointed out that it's likely the CA that issues the client certificates. They're aware of it, but there's no ETA on replacement.

Comment 8 Ewoud Kohl van Wijngaarden 2023-03-09 14:19:15 UTC
Update from the CDN team:

> A new Candlepin Authority was deployed and the chain should verify up to the root without sha1.

I haven't verified this yet, but it may be the last blocker to support the FUTURE crypto policy.

Comment 9 Marco Verschuur 2023-03-09 14:38:44 UTC
Well, it's not in PR yet, because first the new CA must be deployed in the landscape via a (Satellite/Candlepin) update, otherwise all existing installs will get SSL errors and thus cease to sync...
But apart from that, it's good to hear that progress has been made there.

Thanks Ewoud from chasing it all upwards to the CDN team

Comment 10 Ewoud Kohl van Wijngaarden 2023-03-09 15:21:30 UTC
(In reply to Marco Verschuur from comment #9)
> Well, it's not in PR yet, because first the new CA must be deployed in the
> landscape via a (Satellite/Candlepin) update, otherwise all existing
> installs will get SSL errors and thus cease to sync...
> But apart from that, it's good to hear that progress has been made there.

My mind went to Pull Request, but I suspect you may mean PRoduction here.

Still, you raise another good point: I do see we include the certificate in Katello: https://github.com/Katello/katello/blob/master/ca/redhat-uep.pem

A quick openssl x509 -in ca/redhat-uep.pem -noout -text suggests that also uses sha1 signatures. It's a copy of /etc/rhsm/ca/redhat-uep.pem, whihc is provided by subscription-manager-rhsm-certificates. Makes me wonder why we don't use a dependency in our packaging. If we decide to go that route we should make sure SELinux is OK with that.

Comment 11 Ewoud Kohl van Wijngaarden 2023-04-24 18:22:52 UTC
To keep things linked: I see that redhat-uep.pem was updated, at least in CentOS Stream 8 so I opened https://github.com/Katello/katello/pull/10531 to sync Katello.

We also had some discussion in the platform team about how to tackle this properly. It will probably come down to building some verification pipeline similar to what we have for FIPS.

Comment 12 Eric Helms 2023-07-10 18:22:07 UTC
Some additional context. Per the RHEL docs for FUTURE:

"A stricter forward-looking security level intended for testing a possible future policy."


This is to indicate that FUTURE is intended for testing purposes to help ensure systems are ready for changes coming in the future, but is not intended for production use. Given that is RHEL's stance, and Satellite runs on RHEL, we have no intention of supporting production Satellite workloads running on top of FUTURE crypto policy. To that end, I am inclined to close this BZ. If RHEL ever changes it's policy we can re-open it.

Comment 14 Eric Helms 2023-07-26 23:17:12 UTC
I am going to recommend we turn this into a documentation change and add some clarification that Satellite follows RHEL's policy around FUTURE crypto policy and it should not be enabled for production Satellite deployments.