Hi,
this bug is marked as blocking #1635116 which means the component is implementing cryptogrpahic algorithms it shouldn't, and that is a problem for our RHEL security certifications.

As of recently the FIPS compliance project has been transferred to me (Simo), so I would like to asses the status of all the components that still have bugs open.

As the RHEL 8.1 deadlines are looming it is important that we understand what's the status of these bugs, resolution ETA or additional blockers that need resolving.

This update request comes in now as I would like to avoid last minute escalations or exception requests.

Thank you for your cooperation.
Simo.

Hi Simo,

I discovered in the bug:

- usage of gnutls for hashing can be backported from upstream
- only thing I did not solve with Nikos was whether some usages of MD5 is an encryption or not, so when I should you relax mode of FIPS. CUPS uses MD5 for following cases:

  - creating session cookie - IMO not encryption
  - creating UUID - IMO not encryption
  - digest authentication - usage MD5 or SHA-2 is needed by RFC - it is encryption, can be left out - there are other authentications in CUPS (basic, kerberos)
  - hashing IPP attribute job-password - defined by RFC, but not critical - it is encryption 
  - hashing credentials sent over secure (gnutls-created) session - it is not for encryption purpose, so relaxed mode for it.

Note: I heard hash SHA-256 is FIPS compliant, is it true? If it is, then I can just choose 'sha2-256' for these hash functions (md5 is set just for backward compatibility).

Are my conclusions acceptable?

(In reply to Zdenek Dohnal from comment #11)
> Hi Simo,
> 
> I discovered in the bug:
> 
> - usage of gnutls for hashing can be backported from upstream
> - only thing I did not solve with Nikos was whether some usages of MD5 is an
> encryption or not, so when I should you relax mode of FIPS. CUPS uses MD5
> for following cases:
> 
>   - creating session cookie - IMO not encryption

What are the inputs to the digest?
Is the digest used to validate those inputs later?

>   - creating UUID - IMO not encryption

What are the inputs to the digest?
Is the digest used simply to create a uniform pseudo random string?

>   - digest authentication - usage MD5 or SHA-2 is needed by RFC - it is
> encryption, can be left out - there are other authentications in CUPS
> (basic, kerberos)
>   - hashing IPP attribute job-password - defined by RFC, but not critical -
> it is encryption 
>   - hashing credentials sent over secure (gnutls-created) session - it is
> not for encryption purpose, so relaxed mode for it.
> 
> Note: I heard hash SHA-256 is FIPS compliant, is it true? If it is, then I
> can just choose 'sha2-256' for these hash functions (md5 is set just for
> backward compatibility).

The SHA2 family (SHA-256,SHA-384,SHA-512) is approved in FIPS, so you can definitely use SHA-2 in FIPS mode.

> Are my conclusions acceptable?

Sounds reasonable,
thank you.

(In reply to Simo Sorce from comment #12)
> (In reply to Zdenek Dohnal from comment #11)
> > Hi Simo,
> > 
> > I discovered in the bug:
> > 
> > - usage of gnutls for hashing can be backported from upstream
> > - only thing I did not solve with Nikos was whether some usages of MD5 is an
> > encryption or not, so when I should you relax mode of FIPS. CUPS uses MD5
> > for following cases:
> > 
> >   - creating session cookie - IMO not encryption
> 
> What are the inputs to the digest?

The input is string created by concatenation of remote address of remote client, server name, server port and eight random numbers, each with seed based on output gettimeofday() output, 'logically AND'ed with 255.

> Is the digest used to validate those inputs later?

No, it is just compared to other cookie stored before.

> 
> >   - creating UUID - IMO not encryption
> 
> What are the inputs to the digest?

String created by concatenation of server name, remote port, random data and optionally object name and/or object number. 

> Is the digest used simply to create a uniform pseudo random string?

Yes.

>

This bug is still in NEWS and doesn't seem to have all necessary ACKs, RHEL 8.1 is fast approaching, what is the status of this bug ?

I'm currently working on rooting out qpdf crypto and creating proper gating for vim, so I can add patch for 'beware vim is not fips compliant', so this one is still in the queue. It is about backporting current upstream into el8, set SHA2 as default and add gnutls weakening calls for things which are not crypto, so I hope it will not be so time consuming like qpdf.

Would you mind evaluating if those crypt usages are really not crypto as I thought?

>   - creating session cookie - IMO not encryption
>   - creating UUID - IMO not encryption

We can argue these two are not used for protection of user data as pulling random numbers would work the same.

>   - digest authentication - usage MD5 or SHA-2 is needed by RFC - it is
> encryption, can be left out - there are other authentications in CUPS
> (basic, kerberos)

authentication, therefore protection of user data, FIPS relevant

>   - hashing IPP attribute job-password - defined by RFC, but not critical -
> it is encryption 

protection of user data, FIPS relevant

>   - hashing credentials sent over secure (gnutls-created) session - it is
> not for encryption purpose, so relaxed mode for it.

If I understand this correctly this is for authentication, therefore protection of user data, FIPS relevant

> Note: I heard hash SHA-256 is FIPS compliant, is it true? If it is, then I
> can just choose 'sha2-256' for these hash functions (md5 is set just for
> backward compatibility).

Yes SHA2 is perfectly fine for FIPS.

> Are my conclusions acceptable?

I think we disagree on one only.

(In reply to Simo Sorce from comment #16)
> >   - hashing credentials sent over secure (gnutls-created) session - it is
> > not for encryption purpose, so relaxed mode for it.
> 
> If I understand this correctly this is for authentication, therefore
> protection of user data, FIPS relevant
> 

But it is already sent over gnutls-created secure connection, so relaxed mode can be used (at least Nikos told me) for it, correct?

Just use the crypto library without any relax, as you say it is configuration whether to use an agorithm or not, an admin setting the system in FIPS mode makes a choices of blocking all unapproved algorithms, so those algorithms should simply stop working in CUPS as well.
If the admin is useing FIPS all their machines will use the same configuration so they will all use only approved crypto.
So I would not relax anything. Use the cryptolibrary functions and let FIPS mode/crypto-policies do their job.

My understanding is that other alogirhms are use by cups as needed and blocking md5 would not be a breaking change ?

Blocking MD5 would break non-cryptographic use cases of MD5, so relaxing mode is needed. Is that correct?

I would leave non-relaxing mode of digest auth and job-password attribute.

Ok, sounds good

Sent upstream pull request https://github.com/apple/cups/pull/5620 , opened issue https://github.com/apple/cups/issues/5601 .

Hi Simo,

upstream rejected the patch with reason they did not want to break someone's printing only due internet security.

Citation:
"Ok so we won’t accept a patch like that.

Local or printing is different from usage over the internet. For this reason, all of the cert and cipher suite policies are managed separately by CUPS so that appropriate policies can be assigned for local, “IoT” devices that are different from what you’d use for cloud services and web browsers. Moreover, CUPS always prefers the most secure hash algorithms and cipher suites when communicating with a printer. If a printer supports something better than MD5 or SHA1 we will use it.

If there is an API we can call to obtain specific policy choices for local printing, we will be happy to use it to set the defaults vs the current client.conf file stuff, but we will not break printing to satisfy the whims of someone that is focused solely on internet security."

I'm a bit worried those are valid points from POV of upstream and our change can be only downstream... do you have any other arguments which can be presented to upstream? Would you mind writing a comment at upstream issue and trying to convince upstream to reconsider?

We have crypto-policies on RHEL8 which will be relaxable by an admin in a future version via  custom crypto policy.
So if you always use one of our crypto libraries we could have a configuration or policy for CUPS too as upstream likes.
CCing Tomas for more advice around crypto policies.
However that work will have to be for 8.2 or later, for 8.1 I guess we'll hav to go with downstream only.

Please note that crypto-policies do not affect general use of MD5. (FIPS mode might be a different thing.) The crypto-policies affect only use of such algorithm in higher-level implementations such as IPSec or TLS or DNSSEC.

So if CUPS just calls MD5 hash from GnuTLS it won't be broken by crypto-policies disabling MD5 hash in TLS (which is already done in RHEL-8).

Created attachment 1603362 [details]
Proposed patch backported from upstream

Hi,

I backported updated cupsHashData() function from upstream, which uses MD5 for digest authentication, because cups-2.2.6 still implements digest authentication based on RFC 2617.

https://www.ietf.org/rfc/rfc2617.txt

Based on https://bugzilla.redhat.com/show_bug.cgi?id=1650233#c7 I put relaxed mode for gnutls_hash_fast() call and I left internal MD5 for use cases which are not cryptographic such as:

- cookie
- UUID
- informational digest of certchain

as I agreed with Simo.

Anderson, I'm not sure who is backup for Simo for reviewing FIPS related issues, but he asked you for looking into my qpdf FIPS related bug, so I'll try you (feel free to reassign NEEDINFO to a correct person).

Would you mind reviewing the patch?

Sure, I can review the patch.

Hello,

Sorry, but this patch does not look right. If I understood it correctly, you replaced the calls to the internal MD5 implementation with cupsHashData() for the FIPS relevant use cases. But in cupsHashData() you are relaxing the restrictions enforcement for all algorithms, which means you will never execute with the FIPS restrictions, even for the approved algorithms (SHA-2).

In my opinion you should not relax at all. From the comment 19, the algorithm chooses SHA-2 if it is supported, but it would choose MD5 otherwise, and this should cause an error in FIPS mode. Running in relaxed mode would make the crypto library to not error in this case, which is not FIPS compliant. Moreover, if MD5 is chosen to be used, it means the device do not support FIPS approved algorithms and shouldn't be used in an environment which requires FIPS mode (meaning the user should replace his device). If he wants to use a device that do not support FIPS approved algorithms, he is basically breaking the law, as in the majority of cases he is using FIPS mode because he is obligated to do so by law.

And I don't see a reason to keep the internal implementation of MD5 in the cases that are not crypto. In these cases you could use the cupsHashData() in relaxed mode.

My suggestion is to keep the backport of cupsHashData(), but to remove the relaxation.

Hi Sasaki,

(In reply to Anderson Sasaki from comment #36)
> Hello,
> 
> Sorry, but this patch does not look right. If I understood it correctly, you
> replaced the calls to the internal MD5 implementation with cupsHashData()
> for the FIPS relevant use cases. But in cupsHashData() you are relaxing the
> restrictions enforcement for all algorithms, which means you will never
> execute with the FIPS restrictions, even for the approved algorithms (SHA-2).
> 
> In my opinion you should not relax at all. From the comment 19, the
> algorithm chooses SHA-2 if it is supported, but it would choose MD5
> otherwise, and this should cause an error in FIPS mode.

I'm sorry for not keeping this updated - it was after the talk with upstream who meant we are talking about current upstream master, not about our version in RHEL 8.

The algorithm which enables such behavior (choosing better crypto/hash when it is available) it is in upstream CUPS since end of 2017 (in 2.3 alphas and betas) and got into Fedora 29, so IMHO it is not well proven by community to put it into RHEL. 

Plus the change was big (it is completely new implementation of the newer WWW authentication RFC), so backporting would be difficult and it is rather material for CUPS rebase in (approx.) 8.2/8.3..., but it is not question for RHEL 8.1 to do rebase now.

In our RHEL 8 version - cups-2.2.6 - CUPS still implements older RFC for WWW authentication, where MD5 is mandatory. And according Nikos in https://bugzilla.redhat.com/show_bug.cgi?id=1650233#c7 :

"When the use of MD5 is for non cryptographic purposes or mandated by the protocol you can make it functional even at fips mode. See "Relaxing FIPS140-2 requirements" in the gnutls documentation:
https://gnutls.org/manual/html_node/FIPS140_002d2-mode.html
"

So relaxing the MD5 for just this use case falls into the category.

> And I don't see a reason to keep the internal implementation of MD5 in the
> cases that are not crypto. In these cases you could use the cupsHashData()
> in relaxed mode.
> 

Because that's what we agreed on with Simo about non-crypto use cases (follows snippet of irc chat from #crypto channel):

<font color="#16569E"><font size="2">(04:07:49 PM)</font> <b>zdohnal:</b></font> ueno: I have internal implementation of MD5 in CUPS, which is used in several case in non-crypto way (informal digest of part of cert chain, cookie and UUID creation - I talked about them with simo and we agreed it is not crypto use case ) - do I need to port those use cases to a core crypto library usage?<br/>
<font color="#16569E"><font size="2">(04:08:34 PM)</font> <b>zdohnal:</b></font> ueno: or can I stay with internal implementation for those use cases?<br/>
<font color="#A82F2F"><font size="2">(04:13:12 PM)</font> <b>simo:</b></font> zdohnal: md5 used for cookies and uuid as we discussed is fine, as long as it is not an HMAC<br/>
<font color="#16569E"><font size="2">(04:14:40 PM)</font> <b>zdohnal:</b></font> simo: so should I use core crypto lib for those cases or can I stay with internal one?<br/>
<font color="#A82F2F"><font size="2">(04:14:52 PM)</font> <b>simo:</b></font> zdohnal: so to your q. if it is not a cryptographic usage you do not have to port them<br/>
<font color="#16569E"><font size="2">(04:20:17 PM)</font> <b>zdohnal:</b></font> simo: thank you!<br/>

If we apply those what Nikos and Simo said, is the patch okay?

Hello,

@t8m I'm adding you to the discussion as it seems impossible to make cups FIPS compliant, which will hurt our compliance story. As the interim PO while Simo is on PTO, I think the decision for accepting the patch (and exception for having a non compliant component in RHEL) is on you.

(In reply to Zdenek Dohnal from comment #37)
> I'm sorry for not keeping this updated - it was after the talk with upstream
> who meant we are talking about current upstream master, not about our
> version in RHEL 8.
> 
> The algorithm which enables such behavior (choosing better crypto/hash when
> it is available) it is in upstream CUPS since end of 2017 (in 2.3 alphas and
> betas) and got into Fedora 29, so IMHO it is not well proven by community to
> put it into RHEL. 
> 
> Plus the change was big (it is completely new implementation of the newer
> WWW authentication RFC), so backporting would be difficult and it is rather
> material for CUPS rebase in (approx.) 8.2/8.3..., but it is not question for
> RHEL 8.1 to do rebase now.
> 
> In our RHEL 8 version - cups-2.2.6 - CUPS still implements older RFC for WWW
> authentication, where MD5 is mandatory. 

Then, in my understanding, it is not possible to claim FIPS compliance for the current cups we have in RHEL 8, which hurts our compliance story. This will have to be escalated and documented as it will be an exception.

> And according Nikos in
> https://bugzilla.redhat.com/show_bug.cgi?id=1650233#c7 :
> 
> "When the use of MD5 is for non cryptographic purposes or mandated by the
> protocol you can make it functional even at fips mode. See "Relaxing
> FIPS140-2 requirements" in the gnutls documentation:
> https://gnutls.org/manual/html_node/FIPS140_002d2-mode.html
> "
> 
> So relaxing the MD5 for just this use case falls into the category.

The problem is that in your patch the relaxation is done for all algorithms, and not only for MD5. Please change it to relax only for MD5. This does not make it FIPS compliant, but at least make it "less wrong".

Relaxing the execution makes the library to not perform the enforcement of the FIPS restrictions, it is not something that makes the use of non-approved algorithms automatically acceptable.

> Because that's what we agreed on with Simo about non-crypto use cases (follows snippet of irc chat from #crypto channel):

> <font color="#16569E"><font size="2">(04:07:49 PM)</font> <b>zdohnal:</b></font> ueno: I have internal implementation of MD5 in CUPS, which is used in several case in non-crypto way (informal digest of part of cert chain, cookie and UUID creation - I talked about them with simo and we agreed it is not crypto use case ) - do I need to port those use cases to a core crypto library usage?<br/>
> <font color="#16569E"><font size="2">(04:08:34 PM)</font> <b>zdohnal:</b></font> ueno: or can I stay with internal implementation for those use cases?<br/>
> <font color="#A82F2F"><font size="2">(04:13:12 PM)</font> <b>simo:</b></font> zdohnal: md5 used for cookies and uuid as we discussed is fine, as long as it is not an HMAC<br/>
> <font color="#16569E"><font size="2">(04:14:40 PM)</font> <b>zdohnal:</b></font> simo: so should I use core crypto lib for those cases or can I stay with internal one?<br/>
> <font color="#A82F2F"><font size="2">(04:14:52 PM)</font> <b>simo:</b></font> zdohnal: so to your q. if it is not a cryptographic usage you do not have to port them<br/>

I'm fine with keeping the internal implementation for non-crypto usage, but it would look better to use all crypto from the libraries instead of carrying an internal implementation for specific cases.

> If we apply those what Nikos and Simo said, is the patch okay?

I'll leave the approval to Tomaš who is the interim PO while Simo is on PTO.

It is OK to use MD5 for cookies and uuid. It is not OK to use it where its use really protects the user data. Is there any such use with the patch applied? If so, this must be documented and cups has to be given a temporary exception. Or if this use is non-mandatory or some special use-case it can be blocked in the FIPS mode and just documented in release notes that this is blocked for FIPS compliance in FIPS mode so no exception would be needed.

(In reply to Tomas Mraz from comment #39)
> It is OK to use MD5 for cookies and uuid. It is not OK to use it where its
> use really protects the user data. Is there any such use with the patch
> applied?

CUPS-2.2.6 still implements older RFC for HTTP authentication - https://www.ietf.org/rfc/rfc2617.txt - where MD5 is mandatory for digest authentication. 

I looked into the code more deeply again:

I found that CUPS is deciding usage of digest authentication based on the value of 'WWW-Authenticate' field in HTTP packet - if there is string 'digest', it performs digest authentication. But CUPS itself does not set 'digest' string into 'WWW-Authenticate' field in its code.
So other CUPS servers should be able to communicate between each other, the problem happens when outside app/device sends HTTP packet with 'digest' in WWW-Authenticate field.


I really don't like it since it is divergence from upstream, who said (part of citation from https://bugzilla.redhat.com/show_bug.cgi?id=1650233#c26):

"Local or printing is different from usage over the internet. For this reason, all of the cert and cipher suite policies are managed separately by CUPS so that appropriate policies can be assigned for local, “IoT” devices that are different from what you’d use for cloud services and web browsers."

and relaxed whole gnutls call in cupsHashData(). Plus the change will bring more issues to CEE people and to us, maintainers of printing stack. But I'm willing to compromise on the matter and remove the relaxing calls for now...

But it would be certainly great if the tool Simo mentioned in https://bugzilla.redhat.com/show_bug.cgi?id=1650233#c27 would be applicable to FIPS mode as well...

Anderson, Tomas, will be the patch okay when I remove GNU TLS relaxation for MD5?

Created attachment 1605789 [details]
Patch without relaxing

Zdenek,
if you can confirm these are not used in RHEL we can ignore them.
If you want you can add ifdefs downstream to completely disable them too, but not necessary if we have a pretty good guarantee PAM is always used.

There is no exception process in FIPS anymore, but always remember that admins can decide to relax crypto-policies or simply not put the machine in fIPS mode in the first place.
It is and has to be the admin's decision not ours. We just need to document, if it is really likely, what may break in FIPS mode, and then the admins will decide what to deploy.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3617