Bug 2227831

+++ This bug was initially created as a clone of Bug #2214638 +++

Description of problem:
krb5kdc is crashing intermittently on the IPA server even after updating the packages to latest version.

Version-Release number of selected component (if applicable):
ipa-server-4.9.11-5.module+el8.8.0+18146+a1d8660b.x86_64
krb5-server-1.18.2-22.el8_7.x86_64


How reproducible:
Intermittently

--- Additional comment from Rob Crittenden on 2023-06-13 17:53:31 UTC ---

The crash is happening in the ipa kdb code. Changing components.

The ld connection is NULL, hence the failure.

Did this fail in a particular context? e.g. under load, a specific Kerberos request?

--- Additional comment from Rob Crittenden on 2023-06-13 19:43:10 UTC ---

Alexander thinks this may be upstream issue https://pagure.io/freeipa/issue/9251 . We lost traction with an upstream user who had a reproducing system.

--- Additional comment from Têko Mihinto on 2023-07-03 17:33:35 UTC ---

Hi Rob,

(In reply to Rob Crittenden from comment #1)
> The crash is happening in the ipa kdb code. Changing components.
> 
> The ld connection is NULL, hence the failure.
> 
> Did this fail in a particular context? e.g. under load, a specific Kerberos
> request?

Customer mentioned that this typically happens when the LDAP server is under load
( addition / deletion of new hosts triggering multiple and replicated changes ).

Regards,
Têko.

--- Additional comment from Julien Rische on 2023-07-26 10:03:25 UTC ---

This crash is due to the is_master_host() function[1], which is called in ipadb_fill_info3()[2] as part of the process to populate the PAC.

is_master_host() searches for a master host entry using ipadb_simple_search()[3], and interprets an error as the absence of master entry, and the absence of error as a the presence of a master entry. Hence, a connection error would be misinterpreted.

The crash occurs because in case ipadb_simple_search() faces a connection issue during the query, it will try to reset the connection to run the query again. But if it fails to reset the connection, it returns with an error code right away, leaving the LDAP context uninitialized. But, as mentioned above, is_master_host() won't handle this as an actual error. So the process in ipadb_fill_info3() will continue until the ldap_get_entry_controls() call, where the process crashes because of the uninitialized LDAP context.

[1] https://github.com/freeipa/freeipa/blob/release-4-9-12/daemons/ipa-kdb/ipa_kdb_mspac.c#L404
[2] https://github.com/freeipa/freeipa/blob/release-4-9-12/daemons/ipa-kdb/ipa_kdb_mspac.c#L695-L697
[3] https://github.com/freeipa/freeipa/blob/release-4-9-12/daemons/ipa-kdb/ipa_kdb_common.c#L169

--- Additional comment from Julien Rische on 2023-07-31 11:43:36 UTC ---

Upstream ticket:
https://pagure.io/freeipa/issue/9422

--- Additional comment from Julien Rische on 2023-07-31 11:48:31 UTC ---

Upstream pull request:
https://github.com/freeipa/freeipa/pull/6929