On Fedora 42, the sssd has been upgraded to 2.11.0-1.fc42 and with that, the caching of the auth broke. Without a connection to the Kerberos server, the user can't authenticate/log in to the machine. Downgrading to 2.10.2-3.fc42 solves the issue. Lots of logs (hundreds of megabytes) like the following (from /var/log/sssd/sssd_ipa.redhat.com.log): (2025-06-23 19:06:39): [be[ipa.redhat.com]] [sbus_issue_request_done] (0x0040): sssd.dataprovider.getAccountInfo: Error [1432158212]: SSSD is offline and also from /var/log/sssd/sssd_nss.log: (2025-06-23 19:06:39): [nss] [cache_req_common_process_dp_reply] (0x3f7c0): [CID#7] CR #769: Could not get account info [1432158212]: SSSD is offline While authentication without VPN I also see (in /var/log/sssd/sssd_pam.log): (2025-06-23 19:05:52): [pam] [get_password_for_cache_auth] (0x0010): [CID#3] Unsupported auth token type [17]. * ... skipping repetitive backtrace ... (2025-06-23 19:05:52): [pam] [get_password_for_cache_auth] (0x0010): [CID#3] Failed to get password. * ... skipping repetitive backtrace ... (2025-06-23 19:05:52): [pam] [pam_reply] (0x0010): [CID#3] get_password_and_type_for_cache_auth failed. * ... skipping repetitive backtrace ... (2025-06-23 19:05:52): [pam] [client_send] (0x0010): [CID#3] Failed to send data, aborting client! Reproducible: Always Steps to Reproduce: 1. Have a Kerberos auth setup and working both with connection and without it, running on sssd < 2.11.0 2. Upgrade to 2.11.x Actual Results: Without the network connection, the auth does not work Expected Results: The auth should work regardless of the network connection. Credentials should be cached. Additional Information: Possible upstream issue: https://github.com/SSSD/sssd/issues/7968 More info: https://redhat-internal.slack.com/archives/C05AWLALLP6/p1749486720894889
Hi, thank you for your report. I think it is the same as https://github.com/SSSD/sssd/issues/7968 which was reported upstream. There is already a patch under review https://github.com/SSSD/sssd/pull/7995. bye, Sumit
Hi, can you share your PAM configuration? bye, Sumit
(In reply to Sumit Bose from comment #2) > can you share your PAM configuration? Sure. But, can you state what I should share? A output of a command or specific file? Here's a listing of my PAM dir: # tree /etc/pam.d /etc/pam.d ├── chfn ├── chsh ├── config-util ├── crond ├── cups ├── fingerprint-auth -> /etc/authselect/fingerprint-auth ├── gdm-autologin ├── gdm-fingerprint ├── gdm-launch-environment ├── gdm-password ├── gdm-smartcard ├── hddtemp ├── ksu ├── login ├── mock ├── other ├── passwd ├── password-auth -> /etc/authselect/password-auth ├── postlogin -> /etc/authselect/postlogin ├── ppp ├── remote ├── runuser ├── runuser-l ├── smartcard-auth -> /etc/authselect/smartcard-auth ├── sshd ├── sssd-shadowutils ├── su ├── sudo ├── sudo-i ├── su-l ├── system-auth -> /etc/authselect/system-auth ├── vlock ├── vmtoolsd ├── vpnc ├── vpnc-disconnect └── xserver 1 directory, 36 files
Here's the `/etc/pam.d/system-auth` file: ``` # Generated by authselect # Do not modify this file manually, use authselect instead. Any user changes will be overwritten. # You can stop authselect from managing your configuration by calling 'authselect opt-out'. # See authselect(8) for more details. auth required pam_env.so auth required pam_faildelay.so delay=2000000 auth sufficient pam_fprintd.so auth [default=1 ignore=ignore success=ok] pam_usertype.so isregular auth [default=1 ignore=ignore success=ok] pam_localuser.so auth sufficient pam_unix.so nullok auth [default=1 ignore=ignore success=ok] pam_usertype.so isregular auth sufficient pam_sss.so forward_pass auth required pam_deny.so account required pam_unix.so account sufficient pam_localuser.so account sufficient pam_usertype.so issystem account [default=bad success=ok user_unknown=ignore] pam_sss.so account required pam_permit.so password requisite pam_pwquality.so local_users_only password sufficient pam_unix.so yescrypt shadow nullok use_authtok password [success=1 default=ignore] pam_localuser.so password sufficient pam_sss.so use_authtok password required pam_deny.so session optional pam_keyinit.so revoke session required pam_limits.so -session optional pam_systemd.so session [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid session required pam_unix.so session optional pam_sss.so ``` Here's the output of `authselect current`: ``` Profile ID: sssd Enabled features: - with-mdns4 - with-fingerprint - with-silent-lastlog ``` Hope this is useful. Just tell me, if I could help with anything else.
Hi, thank you for the PAM configuration. I'm still a bit puzzled why you see this issue at all, because there should be no other PAM module than pam_sss.so asking for the password. Can you send the output of the journal covering a failed login attempt, especially the PAM related messages? bye, Sumit
Which log file? Or command? I already posted the logs in the original message. Please be more specific - I'm not a sssd/pam expert!
Hi, sorry, I meant the output of `journalctl` or the content /var/log/secure covering the time of a failed login. bye, Sumit
Created attachment 2096011 [details] Logs of failed auth attempt (without VPN, sssd 2.11.0) Here are the logs from /var/log/sssd/*.log from the time of the auth failure, plus an output of journalctl --facility=auth command.
Hi, thank you for the new logs. Unfortunately the journalctl output does not contain the messages I was looking for. Since it looks like you are using ssh for login, can you send the output of `journalctl -u sshd` and the content of `/etc/pam.d/sshd`? bye, Sumit
The `/etc/pam.d/sshd` contents: ``` #%PAM-1.0 auth substack password-auth auth include postlogin account required pam_sepermit.so account required pam_nologin.so account include password-auth password include password-auth # pam_selinux.so close should be the first session rule session required pam_selinux.so close session required pam_loginuid.so # pam_selinux.so open should only be followed by sessions to be executed in the user context session required pam_selinux.so open env_params session required pam_namespace.so session optional pam_keyinit.so force revoke session optional pam_motd.so session include password-auth session include postlogin ``` Not much within the `journalctl --unit=sshd.service`: ``` Jul 09 20:51:24 ksuszyns-thinkpadp1gen4i.remote.csb sshd-session[1724964]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=::1 user=ksuszyns Jul 09 20:51:27 ksuszyns-thinkpadp1gen4i.remote.csb sshd-session[1724964]: Failed password for ksuszyns from ::1 port 40184 ssh2 ``` I can also log in using the second terminal, as I've been using ssh just for ease of use. Here's the `journalctl --facility=auth` for login try on tty3: ``` Jul 09 20:55:34 ksuszyns-thinkpadp1gen4i.remote.csb audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=getty@tty3 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Jul 09 20:55:40 ksuszyns-thinkpadp1gen4i.remote.csb audit: BPF prog-id=2058 op=LOAD Jul 09 20:55:40 ksuszyns-thinkpadp1gen4i.remote.csb audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=fprintd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Jul 09 20:55:50 ksuszyns-thinkpadp1gen4i.remote.csb audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=flatpak-system-helper comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Jul 09 20:55:50 ksuszyns-thinkpadp1gen4i.remote.csb audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=packagekit comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Jul 09 20:56:04 ksuszyns-thinkpadp1gen4i.remote.csb audit[1726459]: USER_AUTH pid=1726459 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:local_login_t:s0-s0:c0.c1023 msg='op=PAM:authentication grantors=? acct="ksuszyns" exe="/usr/bin/login" hostname=ksuszyns-thinkpadp1gen4i.remote.csb addr=? terminal=/dev/tty3 res=failed' Jul 09 20:56:06 ksuszyns-thinkpadp1gen4i.remote.csb audit[1726459]: USER_LOGIN pid=1726459 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:local_login_t:s0-s0:c0.c1023 msg='op=login id=107275 exe="/usr/bin/login" hostname=ksuszyns-thinkpadp1gen4i.remote.csb addr=? terminal=tty3 res=failed' ```
Hi, thank you for the new logs, especially the ones form the sshd.service. This shows that pam_unix.so is actually called, so pam_unix.so will ask for the password and not pam_sss.so. Is there a chance that the user 'ksuszyns' is not only handled on the IPA server but present in /etc/passwd as well? bye, Sumit
Yeah! You're totally right! I got this entry in `/etc/passwd`: ``` ksuszyns:*:107275:107275:Chris Suszynski:/home/ksuszyns:/bin/zsh ``` I have it, just to force the `/bin/zsh` as a shell.
Hi, that most probably explains the issue because the local user will make pam_unix to ask for the password first. I would expect that everything works as expected if you remove the entry. The fix for https://github.com/SSSD/sssd/issues/7968 will fix the issue as well. If you do not want to open a ticket with your IT department you can get `/bin/zsh` for all IPA users if you add override_shell = /bin/zsh to the [nss] section in sssd.conf and restart SSSD. HTH bye, Sumit
I removed the `/etc/passwd` entry and used the following command (from KB0023756): ``` # sss_override user-add ksuszyns --shell /bin/zsh # sss_override user-show ksuszyns ksuszyns.com::::::/bin/zsh: ^D $ getent passwd ksuszyns ksuszyns:*:107275:107275:Chris Suszynski:/home/ksuszyns:/bin/zsh ``` Also, I did upgrade to sssd 2.11.0-1.fc42, and the auth is working without VPN once again! Thanks for the help. BTW, I can help verify the fix for https://github.com/SSSD/sssd/issues/7968 when it lands. If it solves the issue in my original setup.
> The fix for https://github.com/SSSD/sssd/issues/7968 will fix the issue as well. Pushed PR: https://github.com/SSSD/sssd/pull/7995 * `master` * 3b106f1888b6430b8bab75e1c0fe0f054eafce48 - krb5: offline with SSS_AUTHTOK_TYPE_PAM_STACKED * 297ecc467efb6035e370f62e62ffa668bb1d0050 - authtok: add IS_PW_OR_ST_AUTHTOK() * `sssd-2-11` * 856d20a3790080248b78f3c06c043f141d9d9585 - krb5: offline with SSS_AUTHTOK_TYPE_PAM_STACKED * 706a673aef24742f1f8ffa13b6b2257a274ee26a - authtok: add IS_PW_OR_ST_AUTHTOK() * `sssd-2-9` * 6d3e61523698bc0ec17287de01a2dbe1a2d0acab - krb5: offline with SSS_AUTHTOK_TYPE_PAM_STACKED * be42436c2070e1dc9b2e5d3e03700624f4cc20bf - authtok: add IS_PW_OR_ST_AUTHTOK()
Fixed in sssd-2.11.1