Bug 905626

Summary: ipa-client-install failed to fall over to replica with master down
Product: Red Hat Enterprise Linux 6 Reporter: Scott Poore <spoore>
Component: ipaAssignee: Martin Kosek <mkosek>
Status: CLOSED ERRATA QA Contact: Namita Soman <nsoman>
Severity: unspecified Docs Contact:
Priority: high    
Version: 6.4CC: dpal, jgalipea, jwest, mkosek, pasteur, tlavigne
Target Milestone: rcKeywords: ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ipa-3.0.0-26.el6 Doc Type: Bug Fix
Doc Text:
The Identity Management client enrollment "ipa-client-install" command would fail to enroll a client if any of the Identity Management masters were unavailable during enrollment. The client installer now tries all servers, either auto-discovered from DNS or passed via the "--server" option on the command line, until it finds one that is available and enrolls it in that one. Now, the Identity Management client enrollment "ipa-client-install" command functions normally.
Story Points: ---
Clone Of:
: 910557 (view as bug list) Environment:
Last Closed: 2013-11-21 20:49:14 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 909161, 910557, 949632    

Description Scott Poore 2013-01-29 19:23:02 UTC
Description of problem:

When IPA Master is down, ipa-client-install failed:

[root@rhel6-3 install-client-cli]# rlRun "ipa-client-install --domain=$DOMAIN --realm=$RELM  -p $ADMINID -w $ADMINPW --unattended "
LDAP Error: Can't contact LDAP server: 
Failed to verify that rhel6-1.testrelm.com is an IPA Server.
This may mean that the remote server is not up or is not reachable due to network or firewall settings.
Please make sure the following ports are opened in the firewall settings:
     TCP: 80, 88, 389
     UDP: 88 (at least one of TCP/UDP ports 88 has to be open)
Also note that following ports are necessary for ipa-client working properly after enrollment:
     TCP: 464
     UDP: 464, 123 (if NTP enabled)
Installation failed. Rolling back changes.
IPA client is not configured on this system.
:: [   FAIL   ] :: Running 'ipa-client-install --domain=testrelm.com --realm=TESTRELM.COM  -p admin -w Secret123 --unattended ' (Expected 0, got 1)

Looking at log:




Version-Release number of selected component (if applicable):
ipa-client-3.0.0-24.el6.x86_64

How reproducible:
very.  seen it in automation and have reproduced it manually with little effort.

Steps to Reproduce:
1.  Install RHEL6.4 IPA Server
2.  Install RHEL6.4 IPA Replica
3.  On Server:  ipactl stop
4.  On Client:  make sure resolv.conf points to Sever first and Replica second
5.  On Client:  ipa-client-install --domain=$DOMAIN --realm=$RELM  -p $ADMINID -w $ADMINPW --unattended
  
Actual results:

fails

Expected results:

installs using replica


Additional info:

/var/log/ipaclient-install.log:

2013-01-29T17:07:27Z DEBUG /usr/sbin/ipa-client-install was invoked with options: {'domain': 'testrelm.com', 'force': False, 'krb5_offline_passwords': True, 'primary': False, 'mkhomedir': False, 'create_sshfp': 
True, 'conf_sshd': True, 'on_master': False, 'conf_ntp': True, 'ca_cert_file': None, 'ntp_server': None, 'principal': 'admin', 'hostname': None, 'no_ac': False, 'unattended': True, 'sssd': True, 'trust_sshfp': F
alse, 'dns_updates': False, 'realm_name': 'TESTRELM.COM', 'conf_ssh': True, 'server': None, 'prompt_password': False, 'permit': False, 'debug': False, 'preserve_sssd': False, 'uninstall': False}
2013-01-29T17:07:27Z DEBUG missing options might be asked for interactively later
2013-01-29T17:07:27Z DEBUG Loading Index file from '/var/lib/ipa-client/sysrestore/sysrestore.index'
2013-01-29T17:07:27Z DEBUG Loading StateFile from '/var/lib/ipa-client/sysrestore/sysrestore.state'
2013-01-29T17:07:27Z DEBUG [IPA Discovery]
2013-01-29T17:07:27Z DEBUG Starting IPA discovery with domain=testrelm.com, server=None, hostname=rhel6-3.testrelm.com
2013-01-29T17:07:27Z DEBUG Search for LDAP SRV record in testrelm.com
2013-01-29T17:07:27Z DEBUG Search DNS for SRV record of _ldap._tcp.testrelm.com.
2013-01-29T17:07:27Z DEBUG DNS record found: DNSResult::name:_ldap._tcp.testrelm.com.,type:33,class:1,rdata={priority:0,port:389,weight:100,server:rhel6-1.testrelm.com.}
2013-01-29T17:07:27Z DEBUG [Kerberos realm search]
2013-01-29T17:07:27Z DEBUG Search DNS for TXT record of _kerberos.testrelm.com.
2013-01-29T17:07:27Z DEBUG DNS record found: DNSResult::name:_kerberos.testrelm.com.,type:16,class:1,rdata={data:TESTRELM.COM}
2013-01-29T17:07:27Z DEBUG Search DNS for SRV record of _kerberos._udp.testrelm.com.
2013-01-29T17:07:27Z DEBUG DNS record found: DNSResult::name:_kerberos._udp.testrelm.com.,type:33,class:1,rdata={priority:0,port:88,weight:100,server:rhel6-1.testrelm.com.}
2013-01-29T17:07:27Z DEBUG DNS record found: DNSResult::name:_kerberos._udp.testrelm.com.,type:33,class:1,rdata={priority:0,port:88,weight:100,server:rhel6-2.testrelm.com.}
2013-01-29T17:07:27Z DEBUG [LDAP server check]
2013-01-29T17:07:27Z DEBUG Verifying that rhel6-1.testrelm.com (realm TESTRELM.COM) is an IPA server
2013-01-29T17:07:27Z DEBUG Init LDAP connection with: ldap://rhel6-1.testrelm.com:389
2013-01-29T17:07:27Z ERROR LDAP Error: Can't contact LDAP server: 
2013-01-29T17:07:27Z DEBUG Discovery result: UNKNOWN_ERROR; server=rhel6-1.testrelm.com, domain=testrelm.com, kdc=rhel6-1.testrelm.com,rhel6-2.testrelm.com, basedn=None
2013-01-29T17:07:27Z DEBUG will use discovered domain: testrelm.com
2013-01-29T17:07:27Z DEBUG Start searching for LDAP SRV record in "testrelm.com" (Validating DNS Discovery) and its sub-domains
2013-01-29T17:07:27Z DEBUG Search DNS for SRV record of _ldap._tcp.testrelm.com.
2013-01-29T17:07:27Z DEBUG DNS record found: DNSResult::name:_ldap._tcp.testrelm.com.,type:33,class:1,rdata={priority:0,port:389,weight:100,server:rhel6-2.testrelm.com.}
2013-01-29T17:07:27Z DEBUG DNS validated, enabling discovery
2013-01-29T17:07:27Z DEBUG will use discovered server: rhel6-1.testrelm.com
2013-01-29T17:07:27Z ERROR Failed to verify that rhel6-1.testrelm.com is an IPA Server.
2013-01-29T17:07:27Z ERROR This may mean that the remote server is not up or is not reachable due to network or firewall settings.
2013-01-29T17:07:27Z INFO Please make sure the following ports are opened in the firewall settings:
     TCP: 80, 88, 389
     UDP: 88 (at least one of TCP/UDP ports 88 has to be open)
Also note that following ports are necessary for ipa-client working properly after enrollment:
     TCP: 464
     UDP: 464, 123 (if NTP enabled)
2013-01-29T17:07:27Z DEBUG (rhel6-1.testrelm.com: Discovered LDAP SRV records from testrelm.com)
2013-01-29T17:07:27Z ERROR Installation failed. Rolling back changes.
2013-01-29T17:07:27Z ERROR IPA client is not configured on this system.

Comment 2 Dmitri Pal 2013-01-31 23:52:52 UTC
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/3388

Comment 6 Rob Crittenden 2013-02-07 21:56:51 UTC
Fixed upstream.

master: cbb262dc07ea0615068a630e6c7136e3200d5a06

ipa-3-1: a5f10e25b27fb860be0f06506d603197c2e5a955

Comment 8 Namita Soman 2013-02-12 19:01:20 UTC
This bug also needs to be cloned for inclusion in RHEL 7.0

Comment 10 Namita Soman 2013-09-05 18:38:03 UTC
Verified using ipa-client-3.0.0-33.el6.x86_64

; generated by /sbin/dhclient-script
search idm.lab.bos.redhat.com
#nameserver 10.16.101.41
#nameserver 10.11.5.19
nameserver 10.16.98.183
nameserver 10.16.98.184
:: [   PASS   ] :: Running 'cat /etc/resolv.conf' (Expected 0, got 0)
:: [ 13:18:51 ] ::  M=10.16.98.183 ; S=10.16.98.184
:: [   PASS   ] :: Running 'ssh -o StrictHostKeyChecking=no root.com "echo 'service iptables stop' >> /tmp/at.1.sh"' (Expected 0, got 0)
job 1 at 2013-09-04 13:20
:: [   PASS   ] :: Running 'ssh -o StrictHostKeyChecking=no root.com "at -f /tmp/at.1.sh now + 2 minutes"' (Expected 0, got 0)
Warning: Permanently added 'ipaqa64vmc.testrelm.com' (RSA) to the list of known hosts.
:: [   PASS   ] :: Running 'ssh -o StrictHostKeyChecking=no root.com "echo 'service iptables stop' >> /tmp/at.1.sh"' (Expected 0, got 0)
job 1 at 2013-09-04 13:20
:: [   PASS   ] :: Running 'ssh -o StrictHostKeyChecking=no root.com "at -f /tmp/at.1.sh now + 2 minutes"' (Expected 0, got 0)
:: [   PASS   ] :: Start Firewall on MASTER IPA server (Expected 0, got 0)
:: [   PASS   ] :: Start Firewall on SLAVE IPA server (Expected 0, got 0)
:: [ 13:18:57 ] ::  EXECUTING: ipa-client-install -U
:: [   PASS   ] :: Running 'ipa-client-install -p admin -w Secret123 -U > /tmp/tmp.Cf1OjRK0T0/ipaclientinstall_server_unreachableserver.out 2>&1' (Expected 1, got 1)
Unable to discover domain, not provided on command line
Installation failed. Rolling back changes.
IPA client is not configured on this system.
:: [   PASS   ] :: Running 'cat /tmp/tmp.Cf1OjRK0T0/ipaclientinstall_server_unreachableserver.out' (Expected 0, got 0)
:: [ 13:19:19 ] ::  Verify expected error message for IPA Install with unreachable server
:: [   PASS   ] :: Expected error seen:  Unable to discover domain, not provided on command line 
:: [   PASS   ] :: File '/var/log/ipaclient-install.log' should not contain 'Can't contact LDAP server' 
:: [   PASS   ] :: File '/var/log/ipaclient-install.log' should not contain 'Failed to verify that.*is an IPA Server' 
:: [   PASS   ] :: BZ 905626 not found 

MARK-LWD-LOOP -- 2013-09-04 13:20:15 --
:: [   PASS   ] :: Running 'sleep 150' (Expected 0, got 0)
:: [   PASS   ] :: Stop Firewall on MASTER IPA server (Expected 0, got 0)
:: [   PASS   ] :: Stop Firewall on SLAVE IPA server (Expected 0, got 0)
'36a69092-6258-4128-8199-926d8b038d5b'
ipa-client-install-10-Negative-Install-with-unreachable-server result: PASS

Comment 12 errata-xmlrpc 2013-11-21 20:49:14 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1651.html