Bug 1404750

Summary: ipa-client-install fails to get CA cert via LDAP when non-FQDN name of IPA server is first in /etc/hosts
Product: Red Hat Enterprise Linux 7 Reporter: Jan Pazdziora (Red Hat) <jpazdziora>
Component: ipaAssignee: IPA Maintainers <ipa-maint>
Status: CLOSED ERRATA QA Contact: Mohammad Rizwan <myusuf>
Severity: unspecified Docs Contact: Aneta Šteflová Petrová <apetrova>
Priority: unspecified    
Version: 7.3CC: jpazdziora, mbabinsk, myusuf, pvoborni, rcritten
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ipa-4.5.0-1.el7 Doc Type: Enhancement
Doc Text:
Improved security of DNS lookups and robustness of service principal lookups in Identity Management The Kerberos client library no longer attempts to canonicalize host names when issuing ticket-granting server (TGS) requests. This feature improves: * Security because DNS lookups, which were previously required during canonicalization, are no longer performed * Robustness of service principal lookups in more complex DNS environments, such as clouds or containerized applications Make sure you specify the correct fully qualified domain name (FQDN) in host and service principals. Due to this change in behavior, Kerberos does not attempt to resolve any other form of names in principals, such as short names.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-01 09:44:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jan Pazdziora (Red Hat) 2016-12-14 14:44:56 UTC 
Description of problem:

When record for IPA server is present on IPA-client-to-be in /etc/hosts and it starts with non-FQDN name, and principal + password is used to enroll, the command fails with

In unattended mode without a One Time Password (OTP) or without --ca-cert-file
You must specify --force to retrieve the CA cert using HTTP
Cannot obtain CA certificate
HTTP certificate download requires --force
Installation failed. Rolling back changes.
IPA client is not configured on this system.

This situation occurs for example when the IPA-client is run in container under docker 1.12 because

# docker run --link freeipa-server-container:ipa --rm -ti centos:centos7 cat /etc/hosts
127.0.0.1	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters
172.17.0.2	ipa ipa.example.test freeipa-server-container
172.17.0.5	be237a05c86a

Version-Release number of selected component (if applicable):

ipa-client-4.4.0-14.el7_3.x86_64

How reproducible:

Deterministic.

Steps to Reproduce:
1. Have IPA server ipa.example.test, with IPA-managed DNS server on the same machine.
2. On the client machine, point the nameserver record in /etc/resolv.conf to the IP address of the IPA server.
3. On the client machine, put the IP address + short IPA server name + FQDN of IPA server record to /etc/hosts, for example

10.11.12.13 ipa ipa.example.test

4. Run ipa-client-install -U -p admin -w Secret123

Actual results:

WARNING: ntpd time&date synchronization service will not be configured as
conflicting service (chronyd) is enabled
Use --force-ntpd option to disable it and force configuration of ntpd

Discovery was successful!
Client hostname: client.example.test
Realm: EXAMPLE.TEST
DNS Domain: example.test
IPA Server: ipa.example.test
BaseDN: dc=example,dc=test

Skipping synchronizing time with NTP server.
In unattended mode without a One Time Password (OTP) or without --ca-cert-file
You must specify --force to retrieve the CA cert using HTTP
Cannot obtain CA certificate
HTTP certificate download requires --force
Installation failed. Rolling back changes.
IPA client is not configured on this system.

Expected results:

No error, the client IPA-enrolled.

Additional info:

The /var/log/ipaclient-install.log says

2016-12-14T14:34:52Z DEBUG Initializing principal admin using password
2016-12-14T14:34:52Z DEBUG Starting external process
2016-12-14T14:34:52Z DEBUG args=/usr/bin/kinit admin -c /tmp/krbccxKR8MT/ccache
2016-12-14T14:34:52Z DEBUG Process finished, return code=0
2016-12-14T14:34:52Z DEBUG stdout=Password for admin: 

2016-12-14T14:34:52Z DEBUG stderr=
2016-12-14T14:34:52Z DEBUG trying to retrieve CA cert via LDAP from ipa.example.test
2016-12-14T14:34:53Z DEBUG get_ca_certs_from_ldap() error: Insufficient access: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure.  Minor code may provide more information (Server ldap/ipa not found in Kerberos database)
2016-12-14T14:34:53Z DEBUG Insufficient access: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure.  Minor code may provide more information (Server ldap/ipa not found in Kerberos database)
2016-12-14T14:34:53Z ERROR In unattended mode without a One Time Password (OTP) or without --ca-cert-file
You must specify --force to retrieve the CA cert using HTTP
2016-12-14T14:34:53Z ERROR Cannot obtain CA certificate
HTTP certificate download requires --force
2016-12-14T14:34:53Z ERROR Installation failed. Rolling back changes.
2016-12-14T14:34:53Z ERROR IPA client is not configured on this system.

Note that the non-FQDN name of the IPA server machine is used as principal (ldap/ipa).
Comment 1 Jan Pazdziora (Red Hat) 2016-12-14 14:47:02 UTC 
When the /etc/hosts line for the IPA server on the client starts with yet another string, like

10.11.12.13 abc ipa ipa.example.test

that string is used for principal name:

2016-12-14T14:44:32Z DEBUG trying to retrieve CA cert via LDAP from ipa.example.test
2016-12-14T14:44:32Z DEBUG get_ca_certs_from_ldap() error: Insufficient access: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure.  Minor code may provide more information (Server ldap/abc not found in Kerberos database)
2016-12-14T14:44:32Z DEBUG Insufficient access: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure.  Minor code may provide more information (Server ldap/abc not found in Kerberos database)
2016-12-14T14:44:32Z ERROR In unattended mode without a One Time Password (OTP) or without --ca-cert-file

So the client knows it is retrieving from ipa.example.test but uses some sort of reverse lookup to arrive at wrong hostname and thus wrong principal name.
Comment 4 Petr Vobornik 2017-01-02 11:00:37 UTC 
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/6584
Comment 5 Martin Babinsky 2017-01-11 15:27:01 UTC 
Fixed upstream
master:
https://fedorahosted.org/freeipa/changeset/566c86a782bfd7d50938866e9f89faf56cea773f
Comment 8 Martin Babinsky 2017-05-18 14:28:18 UTC 
LGTM.
Comment 9 Mohammad Rizwan 2017-05-30 10:40:34 UTC 
Version:
ipa-server-4.5.0-13.el7.x86_64

IPA Server:

[root@bkr-hv03-guest19 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.19.41.49 bkr-hv03-guest19.testrelm.test
10.19.41.53 bkr-hv03-guest23.testrelm.test

[root@bkr-hv03-guest19 ~]# hostname
bkr-hv03-guest19.testrelm.test

IPA Client Machine:

[root@bkr-hv03-guest23 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.19.41.49 ipa bkr-hv03-guest19.testrelm.test
10.19.41.53 bkr-hv03-guest23.testrelm.test

[root@bkr-hv03-guest23 ~]# hostname
bkr-hv03-guest23.testrelm.test

[root@bkr-hv03-guest23 ~]# ipa-client-install -U -p admin  -w Secret123
WARNING: ntpd time&date synchronization service will not be configured as
conflicting service (chronyd) is enabled
Use --force-ntpd option to disable it and force configuration of ntpd

Discovery was successful!
Client hostname: bkr-hv03-guest23.testrelm.test
Realm: TESTRELM.TEST
DNS Domain: testrelm.test
IPA Server: bkr-hv03-guest19.testrelm.test
BaseDN: dc=testrelm,dc=test

Skipping synchronizing time with NTP server.
Successfully retrieved CA cert
    Subject:     CN=Certificate Authority,O=TESTRELM.TEST
    Issuer:      CN=Certificate Authority,O=TESTRELM.TEST
    Valid From:  2017-05-30 09:04:50
    Valid Until: 2037-05-30 09:04:50

Enrolled in IPA realm TESTRELM.TEST
Created /etc/ipa/default.conf
New SSSD config will be created
Configured sudoers in /etc/nsswitch.conf
Configured /etc/sssd/sssd.conf
Configured /etc/krb5.conf for IPA realm TESTRELM.TEST
trying https://bkr-hv03-guest19.testrelm.test/ipa/json
Forwarding 'ping' to json server 'https://bkr-hv03-guest19.testrelm.test/ipa/json'
Forwarding 'ca_is_enabled' to json server 'https://bkr-hv03-guest19.testrelm.test/ipa/json'
Systemwide CA database updated.
Adding SSH public key from /etc/ssh/ssh_host_rsa_key.pub
Adding SSH public key from /etc/ssh/ssh_host_ecdsa_key.pub
Adding SSH public key from /etc/ssh/ssh_host_ed25519_key.pub
Forwarding 'host_mod' to json server 'https://bkr-hv03-guest19.testrelm.test/ipa/json'
SSSD enabled
Configured /etc/openldap/ldap.conf
Configured /etc/ssh/ssh_config
Configured /etc/ssh/sshd_config
Configuring testrelm.test as NIS domain.
Client configuration complete.
The ipa-client-install command was successful
[root@bkr-hv03-guest23 ~]# 


[root@bkr-hv03-guest23 ~]# ipa host-show
Host name: bkr-hv03-guest23.testrelm.test      
  Host name: bkr-hv03-guest23.testrelm.test
  Principal name: host/bkr-hv03-guest23.testrelm.test
  Principal alias: host/bkr-hv03-guest23.testrelm.test
  SSH public key fingerprint: SHA256:6lRIus6lvLUrdHBcYVvls6TR37PVkySu0WPf2rww0FY (ssh-rsa), SHA256:4SS2veeIxmlSmgljGk2d2qo24JEPe/ECqLfxgQvah3s (ecdsa-sha2-nistp256),
                              SHA256:FmpMEIAfuSdi0jLPYes5ss2v+H6c+tNxq8ztpPgnMxM (ssh-ed25519)
  Password: False
  Keytab: True
  Managed by: bkr-hv03-guest23.testrelm.test
[root@bkr-hv03-guest23 ~]# 

Making an /etc/hosts as described in https://bugzilla.redhat.com/show_bug.cgi?id=1404750#c1

[root@bkr-hv03-guest23 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.19.41.49 abc ipa bkr-hv03-guest19.testrelm.test
10.19.41.53 bkr-hv03-guest23.testrelm.test

[root@bkr-hv03-guest23 ~]# ipa-client-install -U -p admin  -w Secret123
WARNING: ntpd time&date synchronization service will not be configured as
conflicting service (chronyd) is enabled
Use --force-ntpd option to disable it and force configuration of ntpd

Discovery was successful!
Client hostname: bkr-hv03-guest23.testrelm.test
Realm: TESTRELM.TEST
DNS Domain: testrelm.test
IPA Server: bkr-hv03-guest19.testrelm.test
BaseDN: dc=testrelm,dc=test

Skipping synchronizing time with NTP server.
Successfully retrieved CA cert
    Subject:     CN=Certificate Authority,O=TESTRELM.TEST
    Issuer:      CN=Certificate Authority,O=TESTRELM.TEST
    Valid From:  2017-05-30 09:04:50
    Valid Until: 2037-05-30 09:04:50

Enrolled in IPA realm TESTRELM.TEST
Created /etc/ipa/default.conf
New SSSD config will be created
Configured sudoers in /etc/nsswitch.conf
Configured /etc/sssd/sssd.conf
Configured /etc/krb5.conf for IPA realm TESTRELM.TEST
trying https://bkr-hv03-guest19.testrelm.test/ipa/json
Forwarding 'ping' to json server 'https://bkr-hv03-guest19.testrelm.test/ipa/json'
Forwarding 'ca_is_enabled' to json server 'https://bkr-hv03-guest19.testrelm.test/ipa/json'
Systemwide CA database updated.
Adding SSH public key from /etc/ssh/ssh_host_rsa_key.pub
Adding SSH public key from /etc/ssh/ssh_host_ecdsa_key.pub
Adding SSH public key from /etc/ssh/ssh_host_ed25519_key.pub
Forwarding 'host_mod' to json server 'https://bkr-hv03-guest19.testrelm.test/ipa/json'
SSSD enabled
Configured /etc/openldap/ldap.conf
Configured /etc/ssh/ssh_config
Configured /etc/ssh/sshd_config
Configuring testrelm.test as NIS domain.
Client configuration complete.
The ipa-client-install command was successful

[root@bkr-hv03-guest23 ~]# ipa host-show bkr-hv03-guest23.testrelm.test
  Host name: bkr-hv03-guest23.testrelm.test
  Principal name: host/bkr-hv03-guest23.testrelm.test
  Principal alias: host/bkr-hv03-guest23.testrelm.test
  SSH public key fingerprint: SHA256:6lRIus6lvLUrdHBcYVvls6TR37PVkySu0WPf2rww0FY (ssh-rsa), SHA256:4SS2veeIxmlSmgljGk2d2qo24JEPe/ECqLfxgQvah3s (ecdsa-sha2-nistp256),
                              SHA256:FmpMEIAfuSdi0jLPYes5ss2v+H6c+tNxq8ztpPgnMxM (ssh-ed25519)
  Password: False
  Keytab: True
  Managed by: bkr-hv03-guest23.testrelm.test
[root@bkr-hv03-guest23 ~]# cat /etc/hosts
Comment 10 Mohammad Rizwan 2017-06-01 10:53:09 UTC 
verified using following entry in the /etc/hosts on client machine. Client installation was successful.

101.191.411.561 abc ipa bkr-hv03-guest26
Comment 11 Mohammad Rizwan 2017-06-06 06:05:23 UTC 
As per comment #9 and #10, marking it as verified.
Comment 12 errata-xmlrpc 2017-08-01 09:44:33 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2304