Description of problem:krb5kdc fails to start when pkinit and otp auth type is enabled in ipa


Version-Release number of selected component (if applicable):
OS Version: rhel8.5
IPA Version: ipa-server-4.9.6-6.module+el8.5.0+12660+88e16a2c.x86_64
Kerberos: krb5-server-1.18.2-14.el8.x86_64  


How reproducible:
[root@example ~]# ipa config-mod --user-auth-type=pkinit

ipa cannot be started

[root@example ~]# ipactl restart
Restarting Directory Service
Restarting krb5kdc Service
Failed to restart krb5kdc Service
Shutting down

---------------------------
Steps to Reproduce:

[root@example ~]# ipa config-mod --user-auth-type={otp,pkinit}
  Maximum username length: 32
  Maximum hostname length: 64
  Home directory base: /home
  Default shell: /bin/sh
  Default users group: ipausers
  Default e-mail domain: test.realm
  Search time limit: 2
  Search size limit: 100
  User search fields: uid,givenname,sn,telephonenumber,ou,title
  Group search fields: cn,description
  Enable migration mode: FALSE
  Certificate Subject base: O=TEST.REALM
  Password Expiration Notification (days): 4
  Password plugin features: AllowNThash, KDC:Disable Last Success
  SELinux user map order: guest_u:s0$xguest_u:s0$user_u:s0$staff_u:s0-sx0:c0.c1023$sysadm_u:s0-s0:c0.c1023$unconfined_u:sx0-s0:c0.c1023
  Default SELinux user: unconfined_u:s0-s0:c0.c1023
  Default PAC types: MS-PAC, nfs:NONE
  Default user authentication types: otp, pkinit
  IPA masters: example.test.realm
  IPA master capable of PKINIT: example.test.realm
  IPA CA servers: example.test.realm
  IPA CA renewal master: example.test.realm
  IPA DNS servers: example.test.realm

[root@example ~]# ipactl restart
Restarting Directory Service
Restarting krb5kdc Service
Failed to restart krb5kdc Service
Shutting down

[root@example ~]# ipactl status
Directory Service: RUNNING
krb5kdc Service: STOPPED
kadmin Service: STOPPED
named Service: STOPPED
httpd Service: STOPPED
ipa-custodia Service: STOPPED
^CCancelled.
------

And following logs can be seen in the krb5kdc.log during this time.

------
May 02 22:40:41 example.test.realm krb5kdc[103666](Error): Cannot find master key record in database - while fetching master keys list for realm TEST.REALM
May 02 22:43:53 example.test.realm krb5kdc[103761](Error): Cannot find master key record in database - while fetching master keys list for realm TEST.REALM
May 02 22:44:14 example.test.realm krb5kdc[103837](Error): Cannot find master key record in database - while fetching master keys list for realm TEST.REALM
.

Actual results:
We must enforce 2FA authentication upon all users in IdM as per our customers' requirements. Some of them use OTP tokens and others use Smartcards. 

Thus we need to set user auth types to allow either OTP tokens or Smartcards:

[root@example ~]# ipa config-mod --user-auth-type={otp,pkinit}

The feature already exists in rhel8 but setting user-auth-type to pkinit causes krb5kdc to malfunction 

Expected results:

krb5kdc should start along with IPA


Additional info: