I'm filing this bug against RHEL8, but it also affects RHEL7.

Description of problem:

The system default /etc/krb5.conf file as provided by Red Hat's krb5-libs package contains a syntax error that silently breaks PKINIT.

Specifically, this line in the [libdefaults] section:

    pkinit_anchors = /etc/pki/tls/certs/ca-bundle.crt

…is syntactically incorrect. As per the "Specifying PKINIT identity information" section of krb5.conf(5), the "FILE:" tag is *not* optional. If a file location is being specified for the value of the pkinit_anchors option, then the line *must* be:

    pkinit_anchors = FILE:/etc/pki/tls/certs/ca-bundle.crt

This is a devastating syntax error, because the only way to get any useful error messages out of the krb5 pkinit.so module is to *recompile* it with various DEBUG options #define'd in the source code—which I was forced to do in order to debug why we couldn't get PKINIT working.

The debugging output from pkinit.so demonstrates the problem:

pkinit_cert_matching: Selecting the matching cert!
process_option_ca_crl: processing catype ANCHORS, value '/etc/pki/tls/certs/ca-bundle.crt'
No type given for '/etc/pki/tls/certs/ca-bundle.crt'
pkinit_identity_prompt returned 22 (Invalid argument)

Looking at process_option_ca_crl, we clearly see that the tag is required:

static krb5_error_code
process_option_ca_crl(krb5_context context,
                      pkinit_plg_crypto_context plg_cryptoctx,
                      pkinit_req_crypto_context req_cryptoctx,
                      pkinit_identity_opts *idopts,
                      pkinit_identity_crypto_context id_cryptoctx,
                      const char *value,
                      int catype)
{
    char *residual;
    unsigned int typelen;
    int idtype;

    pkiDebug("%s: processing catype %s, value '%s'\n",
             __FUNCTION__, catype2string(catype), value);
    residual = strchr(value, ':');
    if (residual == NULL) {
        pkiDebug("No type given for '%s'\n", value);
        return EINVAL;
    }
    residual++; /* skip past colon */
    typelen = residual - value;
    if (strncmp(value, "FILE:", typelen) == 0) {
        idtype = IDTYPE_FILE;
    } else if (strncmp(value, "DIR:", typelen) == 0) {
        idtype = IDTYPE_DIR;
#ifdef PKINIT_CRYPTO_IMPL_NSS
    } else if (strncmp(value, "NSS:", typelen) == 0) {
        idtype = IDTYPE_NSS;
#endif
    } else {
        return ENOTSUP;
    }
    return crypto_load_cas_and_crls(context,
                                    plg_cryptoctx,
                                    req_cryptoctx,
                                    idopts, id_cryptoctx,
                                    idtype, catype, residual);
}

Even if a syntactically correct pkinit_anchors option is provided in the [libdefaults] section of an included /etc/krb5.conf.d/* file, due to how /etc/krb5.conf.d/* include file precedence works, this will not prevent the syntactically incorrect pkinit_anchors option in /etc/krb5.conf from silently killing pkinit:

pkinit_cert_matching: Selecting the matching cert!
process_option_ca_crl: processing catype ANCHORS, value 'FILE:/etc/pki/tls/certs/ca-bundle.crt'
process_option_ca_crl: processing catype ANCHORS, value '/etc/pki/tls/certs/ca-bundle.crt'
No type given for '/etc/pki/tls/certs/ca-bundle.crt'

There are only two ways to prevent the syntactically incorrect pkinit_anchors setting in the /etc/krb5.conf file from silently killing pkinit:

1. Remove or correct the syntactically incorrect pkinit_anchors setting.
2. Correctly set pkinit_anchors in the [realms] section of either the /etc/krb5.conf file, or an included /etc/krb5.conf.d/* file, which will entirely override any [libdefaults] pkinit_anchors settings.

That all being said, the best solution for resolving the syntactically incorrect pkinit_anchors setting is to *remove* this setting entirely.

The reason for this is that pkinit.so is currently very, very dumb about how it builds the certificate chain. Specifically, it includes *every* CA certificate it discovers via the pkinit_anchors setting in the initial AS-REQ request it sends to the KDC. Since /etc/pki/tls/certs/ca-bundle.crt currently has about 138 CA certificates in it, this results in an initial AS-REQ to the KDC that has ~138 certificates included, which requires >256KiB of data to be exchanged just for the AS-REQ/AS-REP sequence.

Based on this behavior, the krb5 authors clearly intended that pkinit_anchors should be set so that it includes just the CA certificates required for PKINIT authentication, and no more. Since that CA certificate list will depend on how individual sites have implemented smart card login (i.e. the vendors they have elected to use will determine the necessary CA certificates), pkinit_anchors should be set by individual site administrators on a site-by-site basis. Having a default value that results in every AS-REQ pelting the KDC with 138 CA certificates is harmful, not helpful.

So, in summary:

1) The syntactically incorrect pkinit_anchors option in the system default /etc/krb5.conf file silently breaks PKINIT authentication in a way that is undetectable unless someone compiles a custom version of krb5 with debugging enabled.
2) The preferred solution to resolving this problem is to remove the pkinit_anchors setting from the system default /etc/krb5.conf entirely, instead of correctly the syntax error.

Due to how nefariously difficult it is to detect the syntax error, and that the syntax error breaks PKINIT entirely, I would argue that this bug qualifies for an Urgent Priority Bug Fix Advisory (RHBA) for RHEL7 Maintenance Support 2 Phase.

Version-Release number of selected component (if applicable):

krb5-libs-1.15.1-37.el7_7.2
krb5-libs-1.16.1-22.el8

(RHEL6 and earlier do not set pkinit_anchors in the system default /etc/krb5.conf file.)