Bug 1962611

Summary: sssd-kcm crashes, disabling password authentication
Product: [Fedora] Fedora Reporter: Thomas Boroske <boroske>
Component: sssdAssignee: sssd-maintainers <sssd-maintainers>
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 33CC: abokovoy, atikhono, jhrozek, lslebodn, luk.claes, mzidek, pbrezina, sbose, ssorce, sssd-maintainers
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-05-20 11:32:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Thomas Boroske 2021-05-20 11:23:58 UTC
Hello, 


we are running an application server on fedora 33. 

We use FreeIPA for authentication, which builds on kerberos.

After the update to sssd packages this morning (20. May 2021) login or su using password authentication no longer works. 

I tracked down the problem to a crash in sssd-kcm I think: 


A simple "klist" as a logged in user (using pubkey auth) gets you this: 


[thomasb@localhost ~]$ klist
klist: Internal credentials cache error while resolving ccache


In the logs:

[root@localhost log]# tail -f  sssd/sssd_kcm.log 
...
(2021-05-20 12:26:27): [kcm] [local_decrypt] (0x0040): sss_decrypt failed [14]: Bad address
********************** PREVIOUS MESSAGE WAS TRIGGERED BY THE FOLLOWING BACKTRACE:
   *  (2021-05-20 12:23:55): [kcm] [kcm_failbuf_construct] (0x1000): Sent reply with error -1765328188
   *  (2021-05-20 12:23:55): [kcm] [queue_removal_cb] (0x0200): Removed queue for 1449 
   *  (2021-05-20 12:23:55): [kcm] [kcm_send] (0x2000): All data sent!
   *  (2021-05-20 12:23:55): [kcm] [kcm_recv] (0x4000): Client closed connection.
   *  (2021-05-20 12:23:55): [kcm] [client_close_fn] (0x2000): Terminated client [0x55cd82ddcd90][14]
   *  (2021-05-20 12:26:09): [kcm] [responder_idle_handler] (0x2000): Re-scheduling the idle timeout [responder_idle_timeout] for the responder [0x55cd819c8ae0]
   *  (2021-05-20 12:26:09): [kcm] [schedule_responder_idle_timer] (0x2000): Re-scheduling the idle timeout [responder_idle_timeout] for the responder [0x55cd819c8ae0]
   *  (2021-05-20 12:26:27): [kcm] [get_client_cred] (0x4000): Client [0x55cd82dd3040][14] creds: euid[1449] egid[100] pid[29259] cmd_line['klist'].
   *  (2021-05-20 12:26:27): [kcm] [setup_client_idle_timer] (0x4000): Idle timer re-set for client [0x55cd82dd3040][14]
   *  (2021-05-20 12:26:27): [kcm] [accept_fd_handler] (0x0400): Client [0x55cd82dd3040][14] connected!
   *  (2021-05-20 12:26:27): [kcm] [kcm_input_parse] (0x1000): Received message with length 4
   *  (2021-05-20 12:26:27): [kcm] [kcm_get_opt] (0x2000): The client requested operation 20
   *  (2021-05-20 12:26:27): [kcm] [kcm_cmd_send] (0x0400): KCM operation GET_DEFAULT_CACHE
   *  (2021-05-20 12:26:27): [kcm] [kcm_cmd_send] (0x1000): 0 bytes on KCM input
   *  (2021-05-20 12:26:27): [kcm] [kcm_op_queue_send] (0x0200): Adding request by 1449 to the wait queue
   *  (2021-05-20 12:26:27): [kcm] [kcm_op_queue_get] (0x1000): No existing queue for this ID
   *  (2021-05-20 12:26:27): [kcm] [kcm_op_queue_send] (0x1000): Queue was empty, running the request immediately
   *  (2021-05-20 12:26:27): [kcm] [kcm_op_get_default_ccache_send] (0x1000): Getting client's default ccache
   *  (2021-05-20 12:26:27): [kcm] [ccdb_secdb_get_default_send] (0x2000): Getting the default ccache
   *  (2021-05-20 12:26:27): [kcm] [sss_sec_map_path] (0x1000): Mapping prefix /kcm/
   *  (2021-05-20 12:26:27): [kcm] [kcm_map_url_to_path] (0x1000): User-specific KCM path is [/kcm/persistent/1449/default]
   *  (2021-05-20 12:26:27): [kcm] [local_db_dn] (0x2000): Local path for [persistent/1449/default] is [cn=default,cn=1449,cn=persistent,cn=kcm]
   *  (2021-05-20 12:26:27): [kcm] [sss_sec_new_req] (0x1000): Local DB path is persistent/1449/default
   *  (2021-05-20 12:26:27): [kcm] [secdb_dfl_url_req] (0x2000): Created request for URL /kcm/persistent/1449/default
   *  (2021-05-20 12:26:27): [kcm] [sss_sec_get] (0x0400): Retrieving a secret from [persistent/1449/default]
   *  (2021-05-20 12:26:27): [kcm] [sss_sec_get] (0x2000): Searching for [(|(type=simple)(type=binary))] at [cn=default,cn=1449,cn=persistent,cn=kcm] with scope=base
   *  (2021-05-20 12:26:27): [kcm] [local_decrypt] (0x2000): Decrypting with masterkey
   *  (2021-05-20 12:26:27): [kcm] [local_decrypt] (0x0040): sss_decrypt failed [14]: Bad address
********************** BACKTRACE DUMP ENDS HERE *********************************

(2021-05-20 12:26:27): [kcm] [sec_get] (0x0040): Cannot retrieve the secret [14]: Bad address
(2021-05-20 12:26:27): [kcm] [kcm_ccdb_get_default_done] (0x0040): Failed to get the default ccache [14]: Bad address
********************** PREVIOUS MESSAGE WAS TRIGGERED BY THE FOLLOWING BACKTRACE:
   *  (2021-05-20 12:26:27): [kcm] [sec_get] (0x0040): Cannot retrieve the secret [14]: Bad address
   *  (2021-05-20 12:26:27): [kcm] [kcm_ccdb_get_default_done] (0x0040): Failed to get the default ccache [14]: Bad address
********************** BACKTRACE DUMP ENDS HERE *********************************

(2021-05-20 12:26:27): [kcm] [kcm_op_get_get_default_done] (0x0040): Cannot get default ccache [14]: Bad address
(2021-05-20 12:26:27): [kcm] [kcm_cmd_done] (0x0040): op receive function failed [14]: Bad address
********************** PREVIOUS MESSAGE WAS TRIGGERED BY THE FOLLOWING BACKTRACE:
   *  (2021-05-20 12:26:27): [kcm] [kcm_op_get_get_default_done] (0x0040): Cannot get default ccache [14]: Bad address
   *  (2021-05-20 12:26:27): [kcm] [kcm_cmd_done] (0x0040): op receive function failed [14]: Bad address
********************** BACKTRACE DUMP ENDS HERE *********************************

(2021-05-20 12:26:27): [kcm] [kcm_cmd_request_done] (0x0040): KCM operation failed [14]: Bad address
(2021-05-20 12:26:27): [kcm] [kcm_reply_error] (0x0040): KCM operation returns failure [14]: Bad address
********************** PREVIOUS MESSAGE WAS TRIGGERED BY THE FOLLOWING BACKTRACE:
   *  (2021-05-20 12:26:27): [kcm] [kcm_cmd_request_done] (0x0040): KCM operation failed [14]: Bad address
   *  (2021-05-20 12:26:27): [kcm] [kcm_reply_error] (0x0040): KCM operation returns failure [14]: Bad address
********************** BACKTRACE DUMP ENDS HERE *********************************

Comment 1 Thomas Boroske 2021-05-20 11:25:23 UTC
I "fixed" the problem temporarily by reverting to sssd-kcm-2.3.1-4.fc33

Comment 2 Thomas Boroske 2021-05-20 11:32:03 UTC

*** This bug has been marked as a duplicate of bug 1962006 ***

Comment 3 Alexey Tikhonov 2021-05-20 11:39:04 UTC
(In reply to Thomas Boroske from comment #2)
> 
> *** This bug has been marked as a duplicate of bug 1962006 ***

Right, that's duplicate.

Fixed build is available: https://bugzilla.redhat.com/show_bug.cgi?id=1962006#c7

Comment 4 Thomas Boroske 2021-05-20 11:48:03 UTC
Thx very much. I will try the fixed build in testing tonight when current users are gone.