Bug 1750242

This is only reproducible in locales that represent date and time in a single element without spaces. Internally klist is used to list keys in a keytab and output is parsed. klist outputs date/time as a field that typically has space between date and time but in some locales they are not separate by space. Parsing then fails.

[root@master ~]# klist -l
Principal name                 Cache name
--------------                 ----------
admin              KCM:0
[root@master ~]# klist -etK -k /etc/krb5.keytab
Keytab name: FILE:/etc/krb5.keytab
KVNO Timestamp           Principal
---- ------------------- ------------------------------------------------------
   2 2019-09-08T20:21:35 host/master.rhel81.test (aes256-cts-hmac-sha1-96)  (0x67dc5c8e8ad449270ff00b16219e2ac7b4e370a2e6ff1d5edf4a3a3d55994757)
   2 2019-09-08T20:21:35 host/master.rhel81.test (aes128-cts-hmac-sha1-96)  (0x665c3be69062b863be36057e0c81a4f8)
   2 2019-09-08T20:21:35 host/master.rhel81.test (camellia128-cts-cmac)  (0x21e979cd25e52b53930e47cbcc66929b)
   2 2019-09-08T20:21:35 host/master.rhel81.test (camellia256-cts-cmac)  (0x56aa0302bf7247240ae91e49f1428222176fc968b7849ecb0a7efe0ff12938d6)

[root@master ~]# LC_TIME=C  klist -etK -k /etc/krb5.keytab
Keytab name: FILE:/etc/krb5.keytab
KVNO Timestamp         Principal
---- ----------------- --------------------------------------------------------
   2 09/08/19 20:21:35 host/master.rhel81.test (aes256-cts-hmac-sha1-96)  (0x67dc5c8e8ad449270ff00b16219e2ac7b4e370a2e6ff1d5edf4a3a3d55994757)
   2 09/08/19 20:21:35 host/master.rhel81.test (aes128-cts-hmac-sha1-96)  (0x665c3be69062b863be36057e0c81a4f8)
   2 09/08/19 20:21:35 host/master.rhel81.test (camellia128-cts-cmac)  (0x21e979cd25e52b53930e47cbcc66929b)
   2 09/08/19 20:21:35 host/master.rhel81.test (camellia256-cts-cmac)  (0x56aa0302bf7247240ae91e49f1428222176fc968b7849ecb0a7efe0ff12938d6)

A fix needs to add explicit locale to the environment where klist is executed so that stable locale is used.

Created attachment 1613002 [details]
ipa adtrust install log

Locale on the test system

[root@master ~]# locale date_fmt
%a %b %e %H:%M:%S %Z %Y

[root@master ~]# locale
LANG=en_IN.UTF-8
LC_CTYPE="en_IN.UTF-8"
LC_NUMERIC="en_IN.UTF-8"
LC_TIME="en_IN.UTF-8"
LC_COLLATE="en_IN.UTF-8"
LC_MONETARY="en_IN.UTF-8"
LC_MESSAGES="en_IN.UTF-8"
LC_PAPER="en_IN.UTF-8"
LC_NAME="en_IN.UTF-8"
LC_ADDRESS="en_IN.UTF-8"
LC_TELEPHONE="en_IN.UTF-8"
LC_MEASUREMENT="en_IN.UTF-8"
LC_IDENTIFICATION="en_IN.UTF-8"
LC_ALL=

I checked the code and we don't really need the date/time stamp in the klist output, so it can be dropped and therefore the problem will not be there at all. This is an easier and more bullet-proof fix.

Cloned to upstream: https://pagure.io/freeipa/issue/8066

Upstream pull request: https://github.com/freeipa/freeipa/pull/3635

This is the 4.8 backport PR: https://github.com/freeipa/freeipa/pull/3639

1. Run ipa-adtrust-install
2. Then run ipa-server-upgrade.

[root@master ~]# ipa-server-upgrade 
Upgrading IPA:. Estimated time: 1 minute 30 seconds
  [1/11]: stopping directory server
  [2/11]: saving configuration
  [3/11]: disabling listeners
  [4/11]: enabling DS global lock
  [5/11]: disabling Schema Compat
  [6/11]: starting directory server
  [7/11]: updating schema
  [8/11]: upgrading server
Upgrade failed with list index out of range
  [error] RuntimeError: list index out of range
  [cleanup]: stopping directory server
  [cleanup]: restoring configuration
IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
('IPA upgrade failed.', 1)
The ipa-server-upgrade command failed. See /var/log/ipaupgrade.log for more information

Created attachment 1615844 [details]
ipa-upgrade log

Sudhir, the attachment seems to be from notpatched version.

Fix is seen. Verified on RHEL8.1

[root@master ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux release 8.1 Beta (Ootpa)

ipa-server-4.8.0-11.module+el8.1.0+4247+9f3fd721.x86_64
ipa-server-trust-ad-4.8.0-11.module+el8.1.0+4247+9f3fd721.x86_64
389-ds-base-1.4.1.3-7.module+el8.1.0+4150+5b8c2c1f.x86_64
krb5-server-1.17-9.el8.x86_64
selinux-policy-3.14.3-20.el8.noarch


[root@master ~]# locale
LANG=en_IN.UTF-8

[root@master ~]# ipa-adtrust-install 
Configuring CIFS
  [1/24]: validate server hostname
  [2/24]: stopping smbd
  [3/24]: creating samba domain object
  [4/24]: retrieve local idmap range
  [5/24]: creating samba config registry
  [6/24]: writing samba config file
  [7/24]: adding cifs Kerberos principal
  [8/24]: adding cifs and host Kerberos principals to the adtrust agents group
  [9/24]: check for cifs services defined on other replicas
  [10/24]: adding cifs principal to S4U2Proxy targets
  [11/24]: adding admin(group) SIDs
  [12/24]: adding RID bases
  [13/24]: updating Kerberos config
'dns_lookup_kdc' already set to 'true', nothing to do.
  [14/24]: activating CLDAP plugin
  [15/24]: activating sidgen task
  [16/24]: map BUILTIN\Guests to nobody group
  [17/24]: configuring smbd to start on boot
  [18/24]: adding special DNS service records
  [19/24]: restarting Directory Server to take MS PAC and LDAP plugins changes into account
  [20/24]: adding fallback group
  [21/24]: adding Default Trust View
  [22/24]: setting SELinux booleans
  [23/24]: starting CIFS services
  [24/24]: restarting smbd
Done configuring CIFS.

[root@master ~]# ipa-server-upgrade 
Upgrading IPA:. Estimated time: 1 minute 30 seconds
  [1/11]: stopping directory server
  [2/11]: saving configuration
  [3/11]: disabling listeners
  [4/11]: enabling DS global lock
  [5/11]: disabling Schema Compat
  [6/11]: starting directory server
  [7/11]: updating schema
  [8/11]: upgrading server
  [9/11]: stopping directory server
  [10/11]: restoring configuration
  [11/11]: starting directory server
Done.
Update complete
Upgrading IPA services
Upgrading the configuration of the IPA services
Disabled p11-kit-proxy
[Verifying that root certificate is published]
[Migrate CRL publish directory]
Publish directory already set to new location
[Verifying that KDC configuration is using ipa-kdb backend]
[Fix DS schema file syntax]
[Removing RA cert from DS NSS database]
[Enable sidgen and extdom plugins by default]
[Updating HTTPD service IPA configuration]
[Updating HTTPD service IPA WSGI configuration]
Nothing to do for configure_httpd_wsgi_conf
[Migrating from mod_nss to mod_ssl]
Already migrated to mod_ssl
[Moving HTTPD service keytab to gssproxy]
[Removing self-signed CA]
[Removing Dogtag 9 CA]
[Checking for deprecated KDC configuration files]
[Checking for deprecated backups of Samba configuration files]
[Remove FILE: prefix from 'dedicated keytab file' in Samba configuration]
[Update 'max smbd processes' in Samba configuration to prevent unlimited SMBLoris attack amplification]
[Add missing CA DNS records]
Updating DNS system records
[Removing deprecated DNS configuration options]
[Ensuring minimal number of connections]
[Updating GSSAPI configuration in DNS]
[Updating pid-file configuration in DNS]
[Enabling "dnssec-enable" configuration in DNS]
[Setting "bindkeys-file" option in named.conf]
[Including named root key in named.conf]
[Checking global forwarding policy in named.conf to avoid conflicts with automatic empty zones]
[Masking named]
[Fix bind-dyndb-ldap IPA working directory]
[Adding server_id to named.conf]
Changes to named.conf have been made, restart named
[Upgrading CA schema]
CA schema update complete (no changes)
[Verifying that CA audit signing cert has 2 year validity]
[Update certmonger certificate renewal configuration]
Certmonger certificate renewal configuration updated
[Enable PKIX certificate path discovery and validation]
[Authorizing RA Agent to modify profiles]
[Authorizing RA Agent to manage lightweight CAs]
[Ensuring Lightweight CAs container exists in Dogtag database]
[Adding default OCSP URI configuration]
pki-tomcat configuration changed, restart pki-tomcat
[Ensuring CA is using LDAPProfileSubsystem]
[Migrating certificate profiles to LDAP]
[Ensuring presence of included profiles]
[Add default CA ACL]
[Migrating to authselect profile]
[Create systemd-user hbac service and rule]
hbac service systemd-user already exists
[Setup SPAKE]
[Setup PKINIT]
[Enable certauth]
The IPA services were upgraded
The ipa-server-upgrade command was successful

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3348

Test case provided upstream in  ipatests/test_integration/test_commands.py::TestIPACommand::test_ipa_adtrust_install_with_locale_issue8066

master:
https://pagure.io/freeipa/c/555f8a038dae139ad161c5dab51e2378f8894b81

Test upstream:
ipa-4-8:
https://pagure.io/freeipa/c/c59106f005d520f1c84f12a15902cf005162d15c