RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1872689 - Radius Authentication does not work in FIPS mode
Summary: Radius Authentication does not work in FIPS mode
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: krb5
Version: 8.3
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: 8.0
Assignee: Robbie Harwood
QA Contact: Filip Dvorak
Josip Vilicic
URL:
Whiteboard:
Depends On: 1884741
Blocks: 1894575 1958979
TreeView+ depends on / blocked
 
Reported: 2020-08-26 12:53 UTC by Sergey Orlov
Modified: 2021-05-18 14:42 UTC (History)
8 users (show)

Fixed In Version: krb5-1.18.2-6.el8
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-05-18 14:42:18 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2021:1593 0 None None None 2021-05-18 14:42:32 UTC

Description Sergey Orlov 2020-08-26 12:53:34 UTC 
Description of problem:
IPA installed on RHEL8.3 with FIPS mode enabled fails to authenticate user using RADIUS. In the same setup without FIPS RADIUS authentication works.

Version-Release number of selected component (if applicable):
ipa-server-4.8.7-10.module+el8.3.0+7702+ced5f219.x86_64
freeradius-3.0.20-3.module+el8.3.0+7597+67902674.x86_64

How reproducible:
Always

Steps to Reproduce:

Environment setup
=================
fips-mode-setup --enable
reboot
hostnamectl set-hostname master.test.ipa
dnf module reset idm -y
dnf module enable -y idm:DL1/dns
dnf install -y ipa-server-dns
systemctl stop firewalld
ipa-server-install -a Secret123 -p Secret123 --setup-dns --auto-forwarders -n test.ipa -U -r TEST.IPA
dnf install -y freeradius freeradius-ldap freeradius-utils

# Enable MD5 according to https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html-single/linux_domain_identity_authentication_and_policy_guide/index#required-settings-for-configuring-radius-proxy-on-an-idm-server-running-in-fips-mode
mkdir -p /etc/systemd/system/radiusd.service.d/
printf "[Service]\nEnvironment=OPENSSL_FIPS_NON_APPROVED_MD5_ALLOW=1\n" > /etc/systemd/system/radiusd.service.d/ipa-otp.conf
systemctl daemon-reload
systemctl restart radiusd

Environment check:
==================
# cat /proc/`pidof radiusd`/environ
LANG=en_US.UTF-8PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/binINVOCATION_ID=ac466ff29ad645aca3de53e6310bfcecJOURNAL_STREAM=9:98852OPENSSL_FIPS_NON_APPROVED_MD5_ALLOW=1
# cat /proc/sys/crypto/fips_enabled
1
# rpm -q ipa-server
ipa-server-4.8.7-10.module+el8.3.0+7702+ced5f219.x86_64
# rpm -q freeradius
freeradius-3.0.20-3.module+el8.3.0+7597+67902674.x86_64

Test setup
==========
echo Secret123 | kinit admin
echo SecretPrePassword1 | ipa user-add --first tuser --last tuser tuser --password
printf "SecretPrePassword1\nSecretUser1\nSecretUser1\n" | kinit tuser
echo Secret123 | kinit admin
printf "testing123\ntesting123\n" | ipa radiusproxy-add tproxy --server=127.0.0.1
ipa user-mod tuser --user-auth-type=radius --radius=tproxy
echo "tuser Cleartext-Password := "Secret123456"" > /etc/raddb/users
systemctl restart radiusd

Test
====
# kdestroy -A
# echo Secret123 | kinit admin
# echo Secret123456 | kinit -T KCM:0 tuser
Enter OTP Token Value:
kinit: Preauthentication failed while getting initial credentials

Actual results:
User not authenticated

Expected results:
User is authenticated

Additional info:

Kerberos debug
==============
# export KRB5_TRACE=/dev/stdout
# echo Secret123456 | kinit -T KCM:0 tuser
[12609] 1598446199.478084: Resolving unique ccache of type KCM
[12609] 1598446199.478085: Getting initial credentials for tuser
[12609] 1598446199.478086: FAST armor ccache: KCM:0
[12609] 1598446199.478087: Retrieving admin -> krb5_ccache_conf_data/fast_avail/krbtgt\/TEST.IPA\@TEST.IPA@X-CACHECONF: from KCM:0 with result: 0/Success
[12609] 1598446199.478088: Read config in KCM:0 for krbtgt/TEST.IPA: fast_avail: yes
[12609] 1598446199.478089: Using FAST due to armor ccache negotiation result
[12609] 1598446199.478090: Getting credentials admin -> krbtgt/TEST.IPA using ccache KCM:0
[12609] 1598446199.478091: Retrieving admin -> krbtgt/TEST.IPA from KCM:0 with result: 0/Success
[12609] 1598446199.478092: Armor ccache sesion key: aes256-cts/094B
[12609] 1598446199.478094: Creating authenticator for admin -> krbtgt/TEST.IPA, seqnum 0, subkey aes256-cts/FA77, session key aes256-cts/094B
[12609] 1598446199.478096: FAST armor key: aes256-cts/13D8
[12609] 1598446199.478098: Sending unauthenticated request
[12609] 1598446199.478099: Encoding request body and padata into FAST request
[12609] 1598446199.478100: Sending request (916 bytes) to TEST.IPA
[12609] 1598446199.478101: Initiating TCP connection to stream 192.168.121.83:88
[12609] 1598446199.478102: Sending TCP request to stream 192.168.121.83:88
[12609] 1598446199.478103: Received answer (549 bytes) from stream 192.168.121.83:88
[12609] 1598446199.478104: Terminating TCP connection to stream 192.168.121.83:88
[12609] 1598446199.478105: Response was from master KDC
[12609] 1598446199.478106: Received error from KDC: -1765328359/Additional pre-authentication required
[12609] 1598446199.478107: Decoding FAST response
[12609] 1598446199.478110: Preauthenticating using KDC method data
[12609] 1598446199.478111: Processing preauth types: PA-PK-AS-REQ (16), PA-FX-FAST (136), PA-PKINIT-KX (147), PA-OTP-CHALLENGE (141), PA_AS_FRESHNESS (150), PA-FX-COOKIE (133), PA-FX-ERROR (137)
[12609] 1598446199.478112: Received cookie: MIT
[12609] 1598446199.478113: PKINIT client has no configured identity; giving up
[12609] 1598446199.478114: Preauth module pkinit (147) (info) returned: 0/Success
[12609] 1598446199.478115: PKINIT client received freshness token from KDC
[12609] 1598446199.478116: Preauth module pkinit (150) (info) returned: 0/Success
[12609] 1598446199.478117: PKINIT client has no configured identity; giving up
[12609] 1598446199.478118: Preauth module pkinit (16) (real) returned: 22/Invalid argument
Enter OTP Token Value: 
[12609] 1598446199.478119: Preauth module otp (141) (real) returned: 0/Success
[12609] 1598446199.478120: Produced preauth for next request: PA-FX-COOKIE (133), PA-OTP-REQUEST (142)
[12609] 1598446199.478121: Encoding request body and padata into FAST request
[12609] 1598446199.478122: Sending request (1055 bytes) to TEST.IPA
[12609] 1598446199.478123: Initiating TCP connection to stream 192.168.121.83:88
[12609] 1598446199.478124: Sending TCP request to stream 192.168.121.83:88
[12609] 1598446199.478125: Received answer (549 bytes) from stream 192.168.121.83:88
[12609] 1598446199.478126: Terminating TCP connection to stream 192.168.121.83:88
[12609] 1598446199.478127: Response was from master KDC
[12609] 1598446199.478128: Received error from KDC: -1765328360/Preauthentication failed
[12609] 1598446199.478129: Decoding FAST response
kinit: Preauthentication failed while getting initial credentials


radiusd restart log
===================
Wed Aug 26 14:44:38 2020 : Info: Signalled to terminate
Wed Aug 26 14:44:38 2020 : Info: Exiting normally
Wed Aug 26 14:44:38 2020 : Info: Debugger not attached
Wed Aug 26 14:44:38 2020 : Warning: Please use tls_min_version and tls_max_version instead of disable_tlsv1
Wed Aug 26 14:44:38 2020 : Warning: Please use tls_min_version and tls_max_version instead of disable_tlsv1_2
Wed Aug 26 14:44:38 2020 : Warning: tls: Ignoring user-selected DH parameters in FIPS mode. Using defaults.
Wed Aug 26 14:44:38 2020 : Info: Loaded virtual server <default>
Wed Aug 26 14:44:38 2020 : Warning: Ignoring "sql" (see raddb/mods-available/README.rst)
Wed Aug 26 14:44:38 2020 : Warning: Ignoring "ldap" (see raddb/mods-available/README.rst)
Wed Aug 26 14:44:38 2020 : Info: Loaded virtual server default
Wed Aug 26 14:44:38 2020 : Info:  # Skipping contents of 'if' as it is always 'false' -- /etc/raddb/sites-enabled/inner-tunnel:336
Wed Aug 26 14:44:38 2020 : Info: Loaded virtual server inner-tunnel
Wed Aug 26 14:44:38 2020 : Info: Debugger not attached
Wed Aug 26 14:44:38 2020 : Warning: Please use tls_min_version and tls_max_version instead of disable_tlsv1
Wed Aug 26 14:44:38 2020 : Warning: Please use tls_min_version and tls_max_version instead of disable_tlsv1_2
Wed Aug 26 14:44:38 2020 : Warning: tls: Ignoring user-selected DH parameters in FIPS mode. Using defaults.
Wed Aug 26 14:44:38 2020 : Info: Loaded virtual server <default>
Wed Aug 26 14:44:38 2020 : Warning: Ignoring "sql" (see raddb/mods-available/README.rst)
Wed Aug 26 14:44:38 2020 : Warning: Ignoring "ldap" (see raddb/mods-available/README.rst)
Wed Aug 26 14:44:38 2020 : Info: Loaded virtual server default
Wed Aug 26 14:44:38 2020 : Info:  # Skipping contents of 'if' as it is always 'false' -- /etc/raddb/sites-enabled/inner-tunnel:336
Wed Aug 26 14:44:38 2020 : Info: Loaded virtual server inner-tunnel
Wed Aug 26 14:44:38 2020 : Info: Ready to process requests

kerberos log
============
Aug 26 14:49:59 master.test.ipa krb5kdc[10880](info): AS_REQ (4 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha1-96(17), aes128-cts-hmac-sha256-128(19)}) 192.168.121.83: NEEDED_PREAUTH: tuser for krbtgt/TEST.IPA, Additional pre-authentication required
Aug 26 14:49:59 master.test.ipa krb5kdc[10880](info): closing down fd 12
Aug 26 14:49:59 master.test.ipa krb5kdc[10880](info): preauth (otp) verify failure: Generic preauthentication failure
Aug 26 14:49:59 master.test.ipa krb5kdc[10880](info): AS_REQ (4 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha1-96(17), aes128-cts-hmac-sha256-128(19)}) 192.168.121.83: PREAUTH_FAILED: tuser for krbtgt/TEST.IPA, Preauthentication failed
Aug 26 14:49:59 master.test.ipa krb5kdc[10880](info): closing down fd 12
Comment 1 Alex Scheel 2020-08-26 13:44:44 UTC 
Could you run radiusd with the -X flag? 

$ sudo systemctl edit --full radiusd.service
(add -X flag to radiusd)

This should give you additional debug output in the radiusd logs. As it is, there's no indication that kerberos is even hitting radiusd.
Comment 2 Sergey Orlov 2020-08-26 14:00:13 UTC 
After adding -X flag:
"systemctl restart radiusd" does not return, can be exited with Ctrl-C, radiusd is running:

# ps aux | grep radiusd
radiusd    12914  0.0  0.6 166816 12648 ?        Ss   15:55   0:00 /usr/sbin/radiusd -d /etc/raddb -X


I see nothing new in /var/log/radius/radius.log
Wed Aug 26 15:57:03 2020 : Info: Debugger not attached
Wed Aug 26 15:57:03 2020 : Warning: Please use tls_min_version and tls_max_version instead of disable_tlsv1
Wed Aug 26 15:57:03 2020 : Warning: Please use tls_min_version and tls_max_version instead of disable_tlsv1_2
Wed Aug 26 15:57:03 2020 : Warning: tls: Ignoring user-selected DH parameters in FIPS mode. Using defaults.
Wed Aug 26 15:57:03 2020 : Info: Loaded virtual server <default>
Wed Aug 26 15:57:03 2020 : Warning: Ignoring "sql" (see raddb/mods-available/README.rst)
Wed Aug 26 15:57:03 2020 : Warning: Ignoring "ldap" (see raddb/mods-available/README.rst)
Wed Aug 26 15:57:03 2020 : Info: Loaded virtual server default
Wed Aug 26 15:57:03 2020 : Info:  # Skipping contents of 'if' as it is always 'false' -- /etc/raddb/sites-enabled/inner-tunnel:336
Wed Aug 26 15:57:03 2020 : Info: Loaded virtual server inner-tunnel

I have also tried to add -xx instead of -X - stil nothing is written to log file during kinit attempt:
Comment 3 Alex Scheel 2020-08-26 14:13:49 UTC 
Well, yuck. I guess, I'd ask if there's any useful journalctl output and a packet capture if not... Otherwise, a reproducer VM and I can grab those and take a look.
Comment 6 Robbie Harwood 2020-08-26 15:30:22 UTC 
Hi, krb5 maint here.

First, my sympathies to everyone involved for the lack of logging here.  The RADIUS bits that ipa-otpd uses are from libkrad, which, being a library, doesn't log anywhere.

Second, I don't believe this is a supported use case in FIPS mode.  My understanding is that we don't support running anything other than IPA on an IPA server.  Furthermore, RADIUS uses cryptographic MD5 for user-password stuff; this means we can't let it leave the machine.  Because krad can't tell where the packets are being sent, it blocks packet construction entirely in FIPS mode.  This limitation is intentional and known.

Did this ever work?  My feeling is that it can't have, given the above.
Comment 9 Sergey Orlov 2020-09-09 08:41:36 UTC 
Robbie,

Could you please explain your comment in more details?
In  #7 you say that RADIUS authentication is "usable with FreeIPA" when both servers are on the same machine.
I do not have a good knowledge of how IPA works with RADIUS so I do not fully understand why it should not work given the fact that RADIUS and IPA servers are installed on the same machine.
Comment 10 Alexander Bokovoy 2020-09-09 09:00:48 UTC 
Sergey, let me explain it.

https://www.freeipa.org/page/V4/OTP#High-Level_Architecture_and_Workflow explains a high level architecture. In short, when Kerberos client uses OTP pre-authentication mechanism, KDC will talk to its own 'RADIUS' end-point which is by default a UNIX domain socket. The other end of this socket is listened by systemd which triggers ipa-otpd run if something comes in to that socket. ipa-otpd will perform a lookup in IPA LDAP of the details of the user trying to authenticate. If this user has been allowed to authenticate with OTP, ipa-otpd will do an LDAP bind using the user's DN and the password+otp value passed in by KDC through the UNIX domain socket.

If user entry has associated RADIUS proxy configuration, ipa-otpd instead will use the proxy information to connect to the RADIUS proxy. To do so, ipa-otpd uses the same library that KDC uses to talk to RADIUS (libkrad). libkrad library code was tightened in FIPS mode to only allow to communicate over UNIX domain socket because RADIUS communication protocol uses crypto primitives incompatible with FIPS requirements. However, RADIUS proxy definition in FreeIPA cannot specify UNIX domain socket as a target for the proxy. It means, even a localhost connection to RADIUS server would be done using TCP/IP protocol. This connection will be rejected by libkrad in FIPS mode.

To solve this problem we could add an infrastructure to allow specifying RADIUS proxy in FreeIPA by UNIX domain socket in addition to using a server name and its port. In FIPS mode, then libkrad would allow to connect to that UNIX domain socket. This is currently not implemented.
Comment 11 Sergey Orlov 2020-09-09 09:59:37 UTC 
Alexander,

Thank you for your explanation, now it is clear to me.

Alex, 

I agree that this it is not a bug but a documentation issue. Should we close it now or should we wait for documentation to be updated?
Comment 44 errata-xmlrpc 2021-05-18 14:42:18 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: krb5 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:1593
Note You need to log in before you can comment on or make changes to this bug.