Bug 1758270

Description of problem:
When using FreeIPA hosted DNS over anycast (or other load balancer solutions), multiple 'nsupdate' actions during initial client registration (ipa-client-install) can cause replication conflicts to be created.

Version-Release number of selected component (if applicable):
Observed using ipa-client-4.6.4-10 with ipa-server-4.6.4-10, however should be reproducible using any client version that produces multiple 'nsupdate' calls.

How reproducible:
In our environment, we can reproduce 1 out of 10 times, given the proper circumstances.

Steps to Reproduce:
1. Host IPA DNS on 2 or more servers
2. Front the DNS service with a single virtual IP using a load-balancing solution (anycast in our case)
3. Set the client's 'nameserver' parameter in '/etc/resolv.conf' to the virtual IP
4. Perform ipa-client-install on client (unsure if we need to purge host's record prior - I would think not)
5. Observe, via '/var/log/ipaclient-install.log', an 'nsupdate' call to add the A/AAAA record followed by a second 'nsupdate' call to add SSHFP records

Actual results:
The 'nsupdate' calls are randomly selecting which FreeIPA DNS instance to talk to since 'nsupdate' uses 'nameserver' in '/etc/resolv.conf'.  When different servers are selected, the probability for a replication conflict is present.

Excerpts from '/var/log/ipaclient-install.log':

-----------------------

First 'nsupdate' call to create A/AAAA record:

2019-09-20T22:58:28Z DEBUG Writing nsupdate commands to /etc/ipa/.dns_update.txt:
2019-09-20T22:58:28Z DEBUG debug

update delete mbasslertest09.us-west-2b.aws.dreamworks.net. IN A
show
send

update delete mbasslertest09.us-west-2b.aws.dreamworks.net. IN AAAA
show
send

update add mbasslertest09.us-west-2b.aws.dreamworks.net. 1200 IN A 100.110.222.168
show
send

2019-09-20T22:58:28Z DEBUG Starting external process
2019-09-20T22:58:28Z DEBUG args=/usr/bin/nsupdate -g /etc/ipa/.dns_update.txt
2019-09-20T22:58:29Z DEBUG Process finished, return code=0
2019-09-20T22:58:29Z DEBUG stdout=Outgoing update query:
;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id:      0
;; flags:; ZONE: 0, PREREQ: 0, UPDATE: 0, ADDITIONAL: 0
;; UPDATE SECTION:
mbasslertest09.us-west-2b.aws.dreamworks.net. 0	ANY A 

Outgoing update query:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id:  58135
;; flags:; QUESTION: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; QUESTION SECTION:
;2577628563.sig-elevated.gld.dreamworks.net. ANY	TKEY


-----------------------

Second 'nsupdate' call to create SSHFP records:

2019-09-20T22:58:29Z DEBUG Writing nsupdate commands to /etc/ipa/.dns_update.txt:
2019-09-20T22:58:29Z DEBUG debug
update delete mbasslertest09.us-west-2b.aws.dreamworks.net. IN SSHFP
show
send
update add mbasslertest09.us-west-2b.aws.dreamworks.net. 1200 IN SSHFP 1 1 B650686C49D18DFC5EE5B29BEC330474CB8BDEE6
update add mbasslertest09.us-west-2b.aws.dreamworks.net. 1200 IN SSHFP 1 2 A155749101785B1253E8312CB5B4D9A3802AED4D10618211803FA21D434D16B2
update add mbasslertest09.us-west-2b.aws.dreamworks.net. 1200 IN SSHFP 3 1 01AE0E7AD869C0DC6C9A791A1C147857E435B1BF
update add mbasslertest09.us-west-2b.aws.dreamworks.net. 1200 IN SSHFP 3 2 FB447681BFE98A0BC589AF9B6293F13948A02D9BB96375F1F0A022A8D41CF4B1
update add mbasslertest09.us-west-2b.aws.dreamworks.net. 1200 IN SSHFP 4 1 CF0D1EA2F87DCE7EC841ACC590705B936CB19A2B
update add mbasslertest09.us-west-2b.aws.dreamworks.net. 1200 IN SSHFP 4 2 D378A9B78C3320E4A96C8E77CE13FAA44F0E95ED90C5BFE25A2182536CD86B56
show
send

2019-09-20T22:58:29Z DEBUG Starting external process
2019-09-20T22:58:29Z DEBUG args=/usr/bin/nsupdate -g /etc/ipa/.dns_update.txt
2019-09-20T22:58:30Z DEBUG Process finished, return code=0
2019-09-20T22:58:30Z DEBUG stdout=Outgoing update query:
;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id:      0
;; flags:; ZONE: 0, PREREQ: 0, UPDATE: 0, ADDITIONAL: 0
;; UPDATE SECTION:
mbasslertest09.us-west-2b.aws.dreamworks.net. 0	ANY SSHFP 

Outgoing update query:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id:  35923
;; flags:; QUESTION: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; QUESTION SECTION:
;1710008384.sig-duet.gld.dreamworks.net.	ANY TKEY

-----------------------

In this case, the following record was created with conflict record immediately following:

dn: idnsName=mbasslertest09,idnsname=us-west-2b.aws.dreamworks.net.,cn=dns,dc=dreamworks,dc=net
aRecord: 100.110.222.168
dNSTTL: 1200
objectClass: idnsRecord
objectClass: top
idnsName: mbasslertest09
creatorsName: krbprincipalname=dns/elevated.gld.dreamworks.net,cn=services,cn=accounts,dc=dreamworks,dc=net
modifiersName: krbprincipalname=dns/elevated.gld.dreamworks.net,cn=services,cn=accounts,dc=dreamworks,dc=net
createTimestamp: 20190920225829Z
modifyTimestamp: 20190920225829Z
nsUniqueId: 17da4c0e-dbfa11e9-9fa686e0-32b1d2f2
parentid: 63575
entryid: 117642
entryusn: 14813990
entrydn: idnsname=mbasslertest09,idnsname=us-west-2b.aws.dreamworks.net.,cn=dns,dc=dreamworks,dc=net

dn: idnsName=mbasslertest09+nsuniqueid=17da4c14-dbfa11e9-9488cd64-22289086,idnsname=us-west-2b.aws.dreamworks.net.,cn=dns,dc=dreamworks,dc=net
entryusn: 30224545
modifyTimestamp: 20190920225830Z
modifiersName: krbprincipalname=dns/duet.gld.dreamworks.net,cn=services,cn=accounts,dc=dreamworks,dc=net
dNSTTL: 1200
sSHFPRecord: 1 1 B650686C49D18DFC5EE5B29BEC330474CB8BDEE6
sSHFPRecord: 1 2 A155749101785B1253E8312CB5B4D9A3802AED4D10618211803FA21D 434D16B2
sSHFPRecord: 3 1 01AE0E7AD869C0DC6C9A791A1C147857E435B1BF
sSHFPRecord: 3 2 FB447681BFE98A0BC589AF9B6293F13948A02D9BB96375F1F0A022A8 D41CF4B1
sSHFPRecord: 4 1 CF0D1EA2F87DCE7EC841ACC590705B936CB19A2B
sSHFPRecord: 4 2 D378A9B78C3320E4A96C8E77CE13FAA44F0E95ED90C5BFE25A218253 6CD86B56
objectClass: idnsRecord
objectClass: top
objectClass: ldapsubentry
idnsName: mbasslertest09
creatorsName: krbprincipalname=dns/duet.gld.dreamworks.net,cn=services,cn=accounts,dc=dreamworks,dc=net
createTimestamp: 20190920225830Z
nsUniqueId: 17da4c14-dbfa11e9-9488cd64-22289086
parentid: 82705
entryid: 136773
nsds5ReplConflict: namingConflict (ADD) idnsname=mbasslertest09,idnsname=us-west-2b.aws.dreamworks.net.,cn=dns,dc=dreamworks,dc=net
ConflictCSN: 5d85c8ca12f0004d0000
entrydn: idnsname=mbasslertest09+nsuniqueid=17da4c14-dbfa11e9-9488cd64-22289086,idnsname=us-west-2b.aws.dreamworks.net.,cn=dns,dc=dreamworks,dc=net

Expected results:
No replication conflict possibility during initial registration via ipa-client-install.


Additional info: 
From my perspective, there's 2 possible ways to solve this problem:

- ensure that the 'server' parameter is used when writing nsupdate files where 'server' is the same as the initial server used to begin registration.
- write all nsupdate instructions into a single file and invoke nsupdate once

The first option carries the risk that DNS is not running, for some reason, on the IPA server in question.  The second option continues to honor the way nsupdate works by default and would seem to be more robust.