Bug 1379029

Description of problem:

In some test environments we are seeing ipa-replica-install fail occasionally during ipa-replica-conncheck.

2016-09-24T00:43:55Z DEBUG args=/usr/sbin/ipa-replica-conncheck --master master.testrelm.test --auto-master-check --realm TESTRELM.TEST --hostname replica.testrelm.test --principal admin --password XXXXXXXX --ca-cert-file /etc/ipa/ca.crt
2016-09-24T00:44:00Z DEBUG Process finished, return code=1
2016-09-24T00:44:00Z DEBUG stdout=Check connection from replica to remote master 'master.testrelm.test':
   Directory Service: Unsecure port (389): OK
   Directory Service: Secure port (636): OK
   Kerberos KDC: TCP (88): OK
   Kerberos Kpasswd: TCP (464): OK
   HTTP Server: Unsecure port (80): OK
   HTTP Server: Secure port (443): OK

The following list of ports use UDP protocol and would need to be
checked manually:
   Kerberos KDC: UDP (88): SKIPPED
   Kerberos Kpasswd: UDP (464): SKIPPED

Connection from replica to master is OK.
Start listening on required ports for remote master check
Get credentials to log in to remote master
Check RPC connection to remote master
Execute check on remote master
Check connection from master to remote replica 'replica.testrelm.test':
   Directory Service: Unsecure port (389): FAILED
   Directory Service: Secure port (636): FAILED
   Kerberos KDC: TCP (88): FAILED
   Kerberos KDC: UDP (88): WARNING
   Kerberos Kpasswd: TCP (464): FAILED
   Kerberos Kpasswd: UDP (464): WARNING
   HTTP Server: Unsecure port (80): FAILED
   HTTP Server: Secure port (443): FAILED
The following UDP ports could not be verified as open: 88, 464
This can happen if they are already bound to an application
and ipa-replica-conncheck cannot attach own UDP responder.
Port check failed! Inaccessible port(s): 389 (TCP), 636 (TCP), 88 (TCP), 464 (TCP), 80 (TCP), 443 (TCP)

2016-09-24T00:44:00Z DEBUG stderr=Remote master check failed with following error message(s):
ipa-replica-conncheck returned non-zero exit code


Version-Release number of selected component (if applicable):
ipa-server-4.4.0-12.el7.x86_64


How reproducible:
unknown but seen multiple times in test environments.  Mostly in OSP test environments but, I believe we may have seen this earlier in testing in local VM type tests as well.  That we thought had gone away after some release.

Steps to Reproduce:
1.  Install IPA master with DNS
2.  Point resolv.conf on replica to IPA master
3.  Set hostname on replica to match domain of master
4.  on master add DNS record for replica (with a-rec and a-create-reverse)
5.  Install replica with DNS and CA in one step process

Exact command failing:
ipa-replica-install -U --setup-dns --forwarder <DNSFORWARDERIP> --setup-ca --ip-address <REPLICAIP> --server master.testrelm.test --domain testrelm.test --admin-password Secret123 --principal admin


Actual results:

Fails during conncheck


Expected results:

Installs cleanly.

Additional info: