Hide Forgot
Description of problem: Only the very first join from ipa-client-3.2 to ipa-server-3.0 (RHEL-7 > RHEL-6) works. The second join fails unless the machine is deleted explicitly from IPA server. The problem appears only when the client is in the same domain as the server! Also, the issue does not appears with client 3.2 > server 3.2. Version-Release number of selected component (if applicable): client: ipa-client-3.3.1-3.el7 server: ipa-server-3.0.0-25.el6 How reproducible: always Steps to Reproduce: CLIENT: # rpm -qa ipa\* ipa-python-3.3.1-3.el7.x86_64 ipa-client-3.3.1-3.el7.x86_64 # hostname pes-guest-103.ipa.baseos.qe SERVER: # rpm -qa ipa\* ipa-client-3.0.0-25.el6.x86_64 ipa-server-3.0.0-25.el6.x86_64 ipa-pki-ca-theme-9.0.3-7.el6.noarch ipa-admintools-3.0.0-25.el6.x86_64 ipa-server-selinux-3.0.0-25.el6.x86_64 ipa-pki-common-theme-9.0.3-7.el6.noarch ipa-python-3.0.0-25.el6.x86_64 # ipa host-find -------------- 1 host matched -------------- Host name: sec-ipa1.ipa.baseos.qe Principal name: host/sec-ipa1.ipa.baseos.qe.QE Password: False Keytab: True Managed by: sec-ipa1.ipa.baseos.qe SSH public key fingerprint: A6:3E:F8:C8:1C:7D:C5:E9:3D:21:32:52:F2:8B:63:C3 (ssh-dss), 0E:2B:AA:B5:9F:AF:E3:C6:05:AC:FD:28:00:91:02:D5 (ssh-rsa) ---------------------------- Number of entries returned 1 ---------------------------- CLIENT: # ipa-client-install --domain ipa.baseos.qe --realm IPA.BASEOS.QE --mkhomedir --enable-dns-updates --force-join --principal admin -W --force-ntpd Discovery was successful! Hostname: pes-guest-103.ipa.baseos.qe Realm: IPA.BASEOS.QE DNS Domain: ipa.baseos.qe IPA Server: sec-ipa1.ipa.baseos.qe BaseDN: dc=ipa,dc=baseos,dc=qe Continue to configure the system with these values? [no]: yes Synchronizing time with KDC... Password for admin.QE: Successfully retrieved CA cert Subject: CN=Certificate Authority,O=IPA.BASEOS.QE Issuer: CN=Certificate Authority,O=IPA.BASEOS.QE Valid From: Tue Jul 23 12:18:48 2013 UTC Valid Until: Sat Jul 23 12:18:48 2033 UTC Enrolled in IPA realm IPA.BASEOS.QE Created /etc/ipa/default.conf Unable to parse existing SSSD config. As option --preserve-sssd was not specified, new config will override the old one. The old /etc/sssd/sssd.conf is backed up and will be restored during uninstall. New SSSD config will be created Configured /etc/sssd/sssd.conf Configured /etc/krb5.conf for IPA realm IPA.BASEOS.QE Hostname (pes-guest-103.ipa.baseos.qe) not found in DNS DNS server record set to: pes-guest-103.ipa.baseos.qe -> 10.34.59.103 Adding SSH public key from /etc/ssh/ssh_host_rsa_key.pub Adding SSH public key from /etc/ssh/ssh_host_dsa_key.pub host_mod: Unknown option: no_members Failed to upload host SSH public keys. SSSD enabled Configured /etc/openldap/ldap.conf NTP enabled Configured /etc/ssh/ssh_config Configured /etc/ssh/sshd_config Client configuration complete. # # ipa-client-install --uninstall Unenrolling client from IPA server Removing Kerberos service principals from /etc/krb5.keytab Disabling client Kerberos and LDAP configurations Restoring client configuration files nscd daemon is not installed, skip configuration nslcd daemon is not installed, skip configuration Client uninstall complete. The original nsswitch.conf configuration has been restored. You may need to restart services or reboot the machine. Do you want to reboot the machine? [no]: n # SERVER: # ipa host-find --------------- 2 hosts matched --------------- Host name: pes-guest-103.ipa.baseos.qe Principal name: host/pes-guest-103.ipa.baseos.qe.QE Password: False Keytab: False Managed by: pes-guest-103.ipa.baseos.qe Host name: sec-ipa1.ipa.baseos.qe Principal name: host/sec-ipa1.ipa.baseos.qe.QE Password: False Keytab: True Managed by: sec-ipa1.ipa.baseos.qe SSH public key fingerprint: A6:3E:F8:C8:1C:7D:C5:E9:3D:21:32:52:F2:8B:63:C3 (ssh-dss), 0E:2B:AA:B5:9F:AF:E3:C6:05:AC:FD:28:00:91:02:D5 (ssh-rsa) ---------------------------- Number of entries returned 2 ---------------------------- CLIENT: # ipa-client-install --domain ipa.baseos.qe --realm IPA.BASEOS.QE --mkhomedir --enable-dns-updates --force-join --principal admin -W --force-ntpd Discovery was successful! Hostname: pes-guest-103.lab.eng.brq.redhat.com Realm: IPA.BASEOS.QE DNS Domain: ipa.baseos.qe IPA Server: sec-ipa1.ipa.baseos.qe BaseDN: dc=ipa,dc=baseos,dc=qe Continue to configure the system with these values? [no]: yes Synchronizing time with KDC... Password for admin.QE: Successfully retrieved CA cert Subject: CN=Certificate Authority,O=IPA.BASEOS.QE Issuer: CN=Certificate Authority,O=IPA.BASEOS.QE Valid From: Tue Jul 23 12:18:48 2013 UTC Valid Until: Sat Jul 23 12:18:48 2033 UTC Enrolled in IPA realm IPA.BASEOS.QE Failed to obtain host TGT. Installation failed. Rolling back changes. IPA client is not configured on this system. SERVER: # ipa host-del --updatedns pes-guest-103.ipa.baseos.qe ------------------------------------------ Deleted host "pes-guest-103.ipa.baseos.qe" ------------------------------------------ CLIENT: # ipa-client-install --domain ipa.baseos.qe --realm IPA.BASEOS.QE --mkhomedir --enable-dns-updates --force-join --principal admin -W --force-ntpd Discovery was successful! Hostname: pes-guest-103.ipa.baseos.qe Realm: IPA.BASEOS.QE DNS Domain: ipa.baseos.qe IPA Server: sec-ipa1.ipa.baseos.qe BaseDN: dc=ipa,dc=baseos,dc=qe Continue to configure the system with these values? [no]: yes Removed old keys for realm IPA.BASEOS.QE from /etc/krb5.keytab Synchronizing time with KDC... Password for admin.QE: Enrolled in IPA realm IPA.BASEOS.QE Created /etc/ipa/default.conf Unable to parse existing SSSD config. As option --preserve-sssd was not specified, new config will override the old one. The old /etc/sssd/sssd.conf is backed up and will be restored during uninstall. New SSSD config will be created Configured /etc/sssd/sssd.conf Configured /etc/krb5.conf for IPA realm IPA.BASEOS.QE Hostname (pes-guest-103.ipa.baseos.qe) not found in DNS DNS server record set to: pes-guest-103.ipa.baseos.qe -> 10.34.59.103 Adding SSH public key from /etc/ssh/ssh_host_rsa_key.pub Adding SSH public key from /etc/ssh/ssh_host_dsa_key.pub host_mod: Unknown option: no_members Failed to upload host SSH public keys. SSSD enabled Configured /etc/openldap/ldap.conf NTP enabled Configured /etc/ssh/ssh_config Configured /etc/ssh/sshd_config Client configuration complete. BUT!!! CLIENT: # echo 'pes-guest-103.bla.bla.bla' >/etc/hostname # echo 'nameserver 10.34.37.24' > /etc/resolv.conf # hostname pes-guest-103.bla.bla.bla # # ipa-client-install --domain ipa.baseos.qe --realm IPA.BASEOS.QE --mkhomedir --enable-dns-updates --force-join --principal admin -W --force-ntpd Discovery was successful! Hostname: pes-guest-103.bla.bla.bla Realm: IPA.BASEOS.QE DNS Domain: ipa.baseos.qe IPA Server: sec-ipa1.ipa.baseos.qe BaseDN: dc=ipa,dc=baseos,dc=qe Continue to configure the system with these values? [no]: yes Synchronizing time with KDC... Password for admin.QE: Successfully retrieved CA cert Subject: CN=Certificate Authority,O=IPA.BASEOS.QE Issuer: CN=Certificate Authority,O=IPA.BASEOS.QE Valid From: Tue Jul 23 12:18:48 2013 UTC Valid Until: Sat Jul 23 12:18:48 2033 UTC Enrolled in IPA realm IPA.BASEOS.QE Created /etc/ipa/default.conf New SSSD config will be created Configured /etc/sssd/sssd.conf Configured /etc/krb5.conf for IPA realm IPA.BASEOS.QE Hostname (pes-guest-103.bla.bla.bla) not found in DNS Failed to update DNS records. Adding SSH public key from /etc/ssh/ssh_host_rsa_key.pub Adding SSH public key from /etc/ssh/ssh_host_dsa_key.pub host_mod: Unknown option: no_members Failed to upload host SSH public keys. SSSD enabled Configured /etc/openldap/ldap.conf NTP enabled Configured /etc/ssh/ssh_config Configured /etc/ssh/sshd_config Client configuration complete. # # ipa-client-install --uninstall Unenrolling client from IPA server Removing Kerberos service principals from /etc/krb5.keytab Disabling client Kerberos and LDAP configurations Redundant SSSD configuration file /etc/sssd/sssd.conf was moved to /etc/sssd/sssd.conf.deleted Restoring client configuration files nscd daemon is not installed, skip configuration nslcd daemon is not installed, skip configuration Client uninstall complete. The original nsswitch.conf configuration has been restored. You may need to restart services or reboot the machine. Do you want to reboot the machine? [no]: SERVER: # ipa host-find --------------- 3 hosts matched --------------- Host name: pes-guest-103.bla.bla.bla Principal name: host/pes-guest-103.bla.bla.bla.QE Password: False Keytab: False Managed by: pes-guest-103.bla.bla.bla Host name: pes-guest-103.ipa.baseos.qe Principal name: host/pes-guest-103.ipa.baseos.qe.QE Password: False Keytab: False Managed by: pes-guest-103.ipa.baseos.qe Host name: sec-ipa1.ipa.baseos.qe Principal name: host/sec-ipa1.ipa.baseos.qe.QE Password: False Keytab: True Managed by: sec-ipa1.ipa.baseos.qe SSH public key fingerprint: A6:3E:F8:C8:1C:7D:C5:E9:3D:21:32:52:F2:8B:63:C3 (ssh-dss), 0E:2B:AA:B5:9F:AF:E3:C6:05:AC:FD:28:00:91:02:D5 (ssh-rsa) ---------------------------- Number of entries returned 3 ---------------------------- CLIENT: # ipa-client-install --domain ipa.baseos.qe --realm IPA.BASEOS.QE --mkhomedir --enable-dns-updates --force-join --principal admin -W --force-ntpd Discovery was successful! Hostname: pes-guest-103.bla.bla.bla Realm: IPA.BASEOS.QE DNS Domain: ipa.baseos.qe IPA Server: sec-ipa1.ipa.baseos.qe BaseDN: dc=ipa,dc=baseos,dc=qe Continue to configure the system with these values? [no]: yes Synchronizing time with KDC... Unable to sync time with IPA NTP server, assuming the time is in sync. Please check that 123 UDP port is opened. Password for admin.QE: Successfully retrieved CA cert Subject: CN=Certificate Authority,O=IPA.BASEOS.QE Issuer: CN=Certificate Authority,O=IPA.BASEOS.QE Valid From: Tue Jul 23 12:18:48 2013 UTC Valid Until: Sat Jul 23 12:18:48 2033 UTC Enrolled in IPA realm IPA.BASEOS.QE Created /etc/ipa/default.conf New SSSD config will be created Configured /etc/sssd/sssd.conf Configured /etc/krb5.conf for IPA realm IPA.BASEOS.QE Hostname (pes-guest-103.bla.bla.bla) not found in DNS Failed to update DNS records. Adding SSH public key from /etc/ssh/ssh_host_rsa_key.pub Adding SSH public key from /etc/ssh/ssh_host_dsa_key.pub host_mod: Unknown option: no_members Failed to upload host SSH public keys. SSSD enabled Configured /etc/openldap/ldap.conf NTP enabled Configured /etc/ssh/ssh_config Configured /etc/ssh/sshd_config Client configuration complete. Actual results: The second all subsequent join fails until the host machine is removed from the IPA server. Expected results: The same as with ipa-server-3.2, i.e. join pass regardless if it is the very first or not. Additional info: Should you need more logs/debugs or machines with reproducers, feel free to ask.
Note that we don't guarantee a newer client being able to join an older server. That said, please attach a successful and failing /var/log/ipaclient-install.log. Also, it would be helpful to see the contents of /var/log/krb5kdc.log on the IPA server during the second client enrollment.
Created attachment 797336 [details] ipa client/server logs from passing and failing join
(In reply to Rob Crittenden from comment #3) > Note that we don't guarantee a newer client being able to join an older > server. > I thought that this should work. Backward compatibility should be guarantied, shouldn't? > That said, please attach a successful and failing > /var/log/ipaclient-install.log. > > Also, it would be helpful to see the contents of /var/log/krb5kdc.log on the > IPA server during the second client enrollment. See the log attached.
IPA enrollment is forward compatible, not backward compatible. The expectation is that the server is kept current. After the failed second attempt, can you provide from the client: # klist -kt /etc/krb5.keytab The issue is this: kinit: Keytab contains no suitable keys for host/ibm-x3650m4-01-vm-03.lab.eng.bos.redhat.com.QE while getting initial credentials The log suggests that a keytab was created, so we need to see what the contents are.
# klist -kte /etc/krb5.keytab Keytab name: FILE:/etc/krb5.keytab KVNO Timestamp Principal ---- ------------------- ------------------------------------------------------ 1 09/13/2013 09:11:32 host/ibm-x3650m4-01-vm-03.ipa.baseos.qe.QE (aes256-cts-hmac-sha1-96) 1 09/13/2013 09:11:32 host/ibm-x3650m4-01-vm-03.ipa.baseos.qe.QE (aes128-cts-hmac-sha1-96) 1 09/13/2013 09:11:32 host/ibm-x3650m4-01-vm-03.ipa.baseos.qe.QE (des3-cbc-sha1) 1 09/13/2013 09:11:32 host/ibm-x3650m4-01-vm-03.ipa.baseos.qe.QE (arcfour-hmac)
The hostname on the second enrollment is different. In the first enrollment is identifies itself as ibm-x3650m4-01-vm-03.ipa.baseos.qe. In the second as ibm-x3650m4-01-vm-03.lab.eng.bos.redhat.com ipa-join gets the hostname from nodename in uname(2) unless explicitly passed in. It would be interesting to see if you try enrollment with --hostname=ibm-x3650m4-01-vm-03.ipa.baseos.qe. I suspect it will succeed. Still, that wouldn't give us the answer why the machine has this sort of split identity.
(In reply to Rob Crittenden from comment #8) > The hostname on the second enrollment is different. In the first enrollment > is identifies itself as ibm-x3650m4-01-vm-03.ipa.baseos.qe. In the second as > ibm-x3650m4-01-vm-03.lab.eng.bos.redhat.com > > ipa-join gets the hostname from nodename in uname(2) unless explicitly > passed in. > > It would be interesting to see if you try enrollment with > --hostname=ibm-x3650m4-01-vm-03.ipa.baseos.qe. I suspect it will succeed. > Yes, it does. > Still, that wouldn't give us the answer why the machine has this sort of > split identity. The name ibm-x3650m4-01-vm-03.lab.eng.bos.redhat.com is the original hostname of the machine while ibm-x3650m4-01-vm-03.ipa.baseos.qe was changed by purpose for testing just by replacing the original one in /etc/hostname. But # cat /etc/hostname ibm-x3650m4-01-vm-03.ipa.baseos.qe # uname -n ibm-x3650m4-01-vm-03.ipa.baseos.qe so I don't know why ipa-client still uses the old hostname. I can try to play a bit more with it.
ipa-client-install use Python's socket.getfqdn() call to get the hostname: http://docs.python.org/2/library/socket.html#socket.getfqdn I think this may return unexpected answer if you have a broken DNS PTR record resolution. You can try with this call to mimic ipa-client-install logic: # python -c "import socket; print socket.getfqdn()" Anyway, I think correct fix in your case is to either fix your DNS resolution or use --hostname to workaround it.
(In reply to Martin Kosek from comment #10) > ipa-client-install use Python's socket.getfqdn() call to get the hostname: > > http://docs.python.org/2/library/socket.html#socket.getfqdn > > I think this may return unexpected answer if you have a broken DNS PTR > record resolution. You can try with this call to mimic ipa-client-install > logic: > > # python -c "import socket; print socket.getfqdn()" > > Anyway, I think correct fix in your case is to either fix your DNS > resolution or use --hostname to workaround it. Yes, this is it. The problem is in PTR record. What the command python -c "import socket; print socket.getfqdn()" does is that it tries to get the A/AAAA records from DNS (IPA) and then when it receives answer it ask for PTR record. Since PTR records are not configured in IPA by default it forwards the query to DNS servers configured as forwarders. These DNS servers then returns the original hostname of the test machine. The only question that remains is why the DNS on IPA server running on RHEL-7 is not forwarding the PTR record to the configured servers. But this seems to be an issue in bind. Comparing the named.conf on RHEL-6 and RHEL-7 IPA there seems to be no difference.
After talking to the bind maintainer we found out that on RHEL-7 the following line has to be added to named.conf and the reverse resolving starts working. empty-zones-enable no;
Anyhow, this is not a issue of IPA and the case can be solved by proper DNS resolution configuration on IPA server as described in comment 10. The bug can be closed; it is not a bug. Thanks you all for help.
Ok, thank you for info. Closing this Bug.