Description of problem: When using FreeIPA hosted DNS over anycast (or other load balancer solutions), multiple 'nsupdate' actions during initial client registration (ipa-client-install) can cause replication conflicts to be created. Version-Release number of selected component (if applicable): Observed using ipa-client-4.6.4-10 with ipa-server-4.6.4-10, however should be reproducible using any client version that produces multiple 'nsupdate' calls. How reproducible: In our environment, we can reproduce 1 out of 10 times, given the proper circumstances. Steps to Reproduce: 1. Host IPA DNS on 2 or more servers 2. Front the DNS service with a single virtual IP using a load-balancing solution (anycast in our case) 3. Set the client's 'nameserver' parameter in '/etc/resolv.conf' to the virtual IP 4. Perform ipa-client-install on client (unsure if we need to purge host's record prior - I would think not) 5. Observe, via '/var/log/ipaclient-install.log', an 'nsupdate' call to add the A/AAAA record followed by a second 'nsupdate' call to add SSHFP records Actual results: The 'nsupdate' calls are randomly selecting which FreeIPA DNS instance to talk to since 'nsupdate' uses 'nameserver' in '/etc/resolv.conf'. When different servers are selected, the probability for a replication conflict is present. Excerpts from '/var/log/ipaclient-install.log': ----------------------- First 'nsupdate' call to create A/AAAA record: 2019-09-20T22:58:28Z DEBUG Writing nsupdate commands to /etc/ipa/.dns_update.txt: 2019-09-20T22:58:28Z DEBUG debug update delete mbasslertest09.us-west-2b.aws.dreamworks.net. IN A show send update delete mbasslertest09.us-west-2b.aws.dreamworks.net. IN AAAA show send update add mbasslertest09.us-west-2b.aws.dreamworks.net. 1200 IN A 100.110.222.168 show send 2019-09-20T22:58:28Z DEBUG Starting external process 2019-09-20T22:58:28Z DEBUG args=/usr/bin/nsupdate -g /etc/ipa/.dns_update.txt 2019-09-20T22:58:29Z DEBUG Process finished, return code=0 2019-09-20T22:58:29Z DEBUG stdout=Outgoing update query: ;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id: 0 ;; flags:; ZONE: 0, PREREQ: 0, UPDATE: 0, ADDITIONAL: 0 ;; UPDATE SECTION: mbasslertest09.us-west-2b.aws.dreamworks.net. 0 ANY A Outgoing update query: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58135 ;; flags:; QUESTION: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 ;; QUESTION SECTION: ;2577628563.sig-elevated.gld.dreamworks.net. ANY TKEY ----------------------- Second 'nsupdate' call to create SSHFP records: 2019-09-20T22:58:29Z DEBUG Writing nsupdate commands to /etc/ipa/.dns_update.txt: 2019-09-20T22:58:29Z DEBUG debug update delete mbasslertest09.us-west-2b.aws.dreamworks.net. IN SSHFP show send update add mbasslertest09.us-west-2b.aws.dreamworks.net. 1200 IN SSHFP 1 1 B650686C49D18DFC5EE5B29BEC330474CB8BDEE6 update add mbasslertest09.us-west-2b.aws.dreamworks.net. 1200 IN SSHFP 1 2 A155749101785B1253E8312CB5B4D9A3802AED4D10618211803FA21D434D16B2 update add mbasslertest09.us-west-2b.aws.dreamworks.net. 1200 IN SSHFP 3 1 01AE0E7AD869C0DC6C9A791A1C147857E435B1BF update add mbasslertest09.us-west-2b.aws.dreamworks.net. 1200 IN SSHFP 3 2 FB447681BFE98A0BC589AF9B6293F13948A02D9BB96375F1F0A022A8D41CF4B1 update add mbasslertest09.us-west-2b.aws.dreamworks.net. 1200 IN SSHFP 4 1 CF0D1EA2F87DCE7EC841ACC590705B936CB19A2B update add mbasslertest09.us-west-2b.aws.dreamworks.net. 1200 IN SSHFP 4 2 D378A9B78C3320E4A96C8E77CE13FAA44F0E95ED90C5BFE25A2182536CD86B56 show send 2019-09-20T22:58:29Z DEBUG Starting external process 2019-09-20T22:58:29Z DEBUG args=/usr/bin/nsupdate -g /etc/ipa/.dns_update.txt 2019-09-20T22:58:30Z DEBUG Process finished, return code=0 2019-09-20T22:58:30Z DEBUG stdout=Outgoing update query: ;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id: 0 ;; flags:; ZONE: 0, PREREQ: 0, UPDATE: 0, ADDITIONAL: 0 ;; UPDATE SECTION: mbasslertest09.us-west-2b.aws.dreamworks.net. 0 ANY SSHFP Outgoing update query: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35923 ;; flags:; QUESTION: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 ;; QUESTION SECTION: ;1710008384.sig-duet.gld.dreamworks.net. ANY TKEY ----------------------- In this case, the following record was created with conflict record immediately following: dn: idnsName=mbasslertest09,idnsname=us-west-2b.aws.dreamworks.net.,cn=dns,dc=dreamworks,dc=net aRecord: 100.110.222.168 dNSTTL: 1200 objectClass: idnsRecord objectClass: top idnsName: mbasslertest09 creatorsName: krbprincipalname=dns/elevated.gld.dreamworks.net,cn=services,cn=accounts,dc=dreamworks,dc=net modifiersName: krbprincipalname=dns/elevated.gld.dreamworks.net,cn=services,cn=accounts,dc=dreamworks,dc=net createTimestamp: 20190920225829Z modifyTimestamp: 20190920225829Z nsUniqueId: 17da4c0e-dbfa11e9-9fa686e0-32b1d2f2 parentid: 63575 entryid: 117642 entryusn: 14813990 entrydn: idnsname=mbasslertest09,idnsname=us-west-2b.aws.dreamworks.net.,cn=dns,dc=dreamworks,dc=net dn: idnsName=mbasslertest09+nsuniqueid=17da4c14-dbfa11e9-9488cd64-22289086,idnsname=us-west-2b.aws.dreamworks.net.,cn=dns,dc=dreamworks,dc=net entryusn: 30224545 modifyTimestamp: 20190920225830Z modifiersName: krbprincipalname=dns/duet.gld.dreamworks.net,cn=services,cn=accounts,dc=dreamworks,dc=net dNSTTL: 1200 sSHFPRecord: 1 1 B650686C49D18DFC5EE5B29BEC330474CB8BDEE6 sSHFPRecord: 1 2 A155749101785B1253E8312CB5B4D9A3802AED4D10618211803FA21D 434D16B2 sSHFPRecord: 3 1 01AE0E7AD869C0DC6C9A791A1C147857E435B1BF sSHFPRecord: 3 2 FB447681BFE98A0BC589AF9B6293F13948A02D9BB96375F1F0A022A8 D41CF4B1 sSHFPRecord: 4 1 CF0D1EA2F87DCE7EC841ACC590705B936CB19A2B sSHFPRecord: 4 2 D378A9B78C3320E4A96C8E77CE13FAA44F0E95ED90C5BFE25A218253 6CD86B56 objectClass: idnsRecord objectClass: top objectClass: ldapsubentry idnsName: mbasslertest09 creatorsName: krbprincipalname=dns/duet.gld.dreamworks.net,cn=services,cn=accounts,dc=dreamworks,dc=net createTimestamp: 20190920225830Z nsUniqueId: 17da4c14-dbfa11e9-9488cd64-22289086 parentid: 82705 entryid: 136773 nsds5ReplConflict: namingConflict (ADD) idnsname=mbasslertest09,idnsname=us-west-2b.aws.dreamworks.net.,cn=dns,dc=dreamworks,dc=net ConflictCSN: 5d85c8ca12f0004d0000 entrydn: idnsname=mbasslertest09+nsuniqueid=17da4c14-dbfa11e9-9488cd64-22289086,idnsname=us-west-2b.aws.dreamworks.net.,cn=dns,dc=dreamworks,dc=net Expected results: No replication conflict possibility during initial registration via ipa-client-install. Additional info: From my perspective, there's 2 possible ways to solve this problem: - ensure that the 'server' parameter is used when writing nsupdate files where 'server' is the same as the initial server used to begin registration. - write all nsupdate instructions into a single file and invoke nsupdate once The first option carries the risk that DNS is not running, for some reason, on the IPA server in question. The second option continues to honor the way nsupdate works by default and would seem to be more robust.
This bug is moved to RHEL 8 as the fix is not as simple as initially estimated. It would require refactoring of ipa-client installer that is also used by our ansible modules.