Bug 2211020

Summary:	[Upgrade [OSP16.2 -> OSP17.1] ipa environment failed on upgrade at FATAL add reverse zone
Product:	Red Hat OpenStack	Reporter:	Juan Badia Payno <jbadiapa>
Component:	ansible-tripleo-ipa	Assignee:	Ade Lee <alee>
Status:	CLOSED DUPLICATE	QA Contact:	Jeremy Agee <jagee>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	17.1 (Wallaby)	CC:	alee, alfrgarc, ggrasza, jjung, johfulto, jpretori, lsvaty, mgarciac, pgrist, pweeks, twoerner
Target Milestone:	ga	Keywords:	Reopened, Triaged
Target Release:	17.1
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	ansible-tripleo-ipa-0.3.1-1.20230601230760.el9osttrunk	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-06-20 16:53:50 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	2214972
Bug Blocks:	2016660

Description Juan Badia Payno 2023-05-30 09:49:48 UTC

Some logs:
[stack@undercloud-0 ~]$ grep FATAL overcloud_upgrade_run-compute-0,controller-0,controller-1,controller-2,undercloud.log 
2023-05-29 13:57:04 | 2023-05-29 13:57:04.573258 | 52540061-8350-eacb-f3f0-00000000a2cf |      FATAL | add reverse zone | undercloud | error={"changed": false, "msg": "dnszone_add: 0.0.10.in-addr.arpa.: DNS zone 0.0.10.in-addr.arpa. already exists in DNS and is handled by server(s): ['trusted-eng1.ibx-001.prod.rdu2.dc.redhat.com.', 'trusted-eng1.ibx-001.prod.iad2.dc.redhat.com.']"}
2023-05-29 13:57:10 | 2023-05-29 13:57:10.926263 | 52540061-8350-eacb-f3f0-00000000a2e5 |      FATAL | add reverse zone | undercloud | error={"changed": false, "msg": "dnszone_add: 0.0.10.in-addr.arpa.: DNS zone 0.0.10.in-addr.arpa. already exists in DNS and is handled by server(s): ['trusted-eng1.ibx-001.prod.rdu2.dc.redhat.com.', 'trusted-eng1.ibx-001.prod.iad2.dc.redhat.com.']"}
2023-05-29 13:57:17 | 2023-05-29 13:57:17.485091 | 52540061-8350-eacb-f3f0-00000000a2fb |      FATAL | add reverse zone | undercloud | error={"changed": false, "msg": "dnszone_add: 0.0.10.in-addr.arpa.: DNS zone 0.0.10.in-addr.arpa. already exists in DNS and is handled by server(s): ['trusted-eng1.ibx-001.prod.iad2.dc.redhat.com.', 'trusted-eng1.ibx-001.prod.rdu2.dc.redhat.com.']"}
2023-05-29 13:57:40 | 2023-05-29 13:57:40.519070 | 52540061-8350-eacb-f3f0-00000000a348 |      FATAL | add reverse zone | undercloud | error={"changed": false, "msg": "dnszone_add: 0.0.10.in-addr.arpa.: DNS zone 0.0.10.in-addr.arpa. already exists in DNS and is handled by server(s): ['trusted-eng1.ibx-001.prod.rdu2.dc.redhat.com.', 'trusted-eng1.ibx-001.prod.iad2.dc.redhat.com.']"}
2023-05-29 13:58:31 | 2023-05-29 13:58:31.129621 | 52540061-8350-eacb-f3f0-00000000a1a5 |      FATAL | add dns forward and reverse records | undercloud | error={"changed": false, "msg": "'name'"}

Comment 2 Grzegorz Grasza 2023-05-31 15:13:28 UTC

This doesn't look like a bug, it is expected that adding the reverse zone might fail and there is a rescue block for adding entries without a PTR record.

Comment 3 Grzegorz Grasza 2023-05-31 15:15:58 UTC

Sorry, the output confused me. The error seems to be on the last line: 

2023-05-29 13:58:31 | 2023-05-29 13:58:31.129621 | 52540061-8350-eacb-f3f0-00000000a1a5 |      FATAL | add dns forward and reverse records | undercloud | error={"changed": false, "msg": "'name'"}

Comment 4 Ade Lee 2023-05-31 15:27:03 UTC

I suspect -- and need to confirm this with more info, that we're running into a situation where the installed version of ansible-freeipa does not support the addition of multiple DNS records at once.

So, to confirm this:
1. Can we have a full log (rather than just a grep) to see if there is any further information (error messages etc.) ?  We may need to enable --debug.
2. What version of ansible-freeipa was on the system to begin with when this failed?

Comment 5 Juan Badia Payno 2023-06-01 08:22:40 UTC

These are the last lines of the output

2023-05-29 13:57:40 | 2023-05-29 13:57:40.519070 | 52540061-8350-eacb-f3f0-00000000a348 |      FATAL | add reverse zone | undercloud | error={"changed": false, "msg": "dnszone_add: 0.0.10.in-addr.arpa.: DNS zone 0.0.10.in-addr.arpa. already exists in DNS and is handled by server(s): ['trusted-eng1.ibx-001.prod.rdu2.dc.redhat.com.', 'trusted-eng1.ibx-001.prod.iad2.dc.redhat.com.']"}
2023-05-29 13:57:40 | 2023-05-29 13:57:40.521056 | 52540061-8350-eacb-f3f0-00000000a348 |     TIMING | tripleo_ipa_dns : add reverse zone | undercloud | 0:20:41.356126 | 3.46s
2023-05-29 13:57:40 | 2023-05-29 13:57:40.538800 | 52540061-8350-eacb-f3f0-00000000a349 |       TASK | add ip to no_reverse_ips
2023-05-29 13:57:40 | 2023-05-29 13:57:40.593399 | 52540061-8350-eacb-f3f0-00000000a349 |         OK | add ip to no_reverse_ips | undercloud
2023-05-29 13:57:40 | 2023-05-29 13:57:40.595142 | 52540061-8350-eacb-f3f0-00000000a349 |     TIMING | tripleo_ipa_dns : add ip to no_reverse_ips | undercloud | 0:20:41.430210 | 0.05s
2023-05-29 13:57:40 | 2023-05-29 13:57:40.613440 | 52540061-8350-eacb-f3f0-00000000a353 |       TASK | add reverse zone
2023-05-29 13:57:44 | 2023-05-29 13:57:44.009713 | 52540061-8350-eacb-f3f0-00000000a353 |         OK | add reverse zone | undercloud
2023-05-29 13:57:44 | 2023-05-29 13:57:44.011432 | 52540061-8350-eacb-f3f0-00000000a353 |     TIMING | tripleo_ipa_dns : add reverse zone | undercloud | 0:20:44.846480 | 3.40s
2023-05-29 13:57:44 | 2023-05-29 13:57:44.027463 | 52540061-8350-eacb-f3f0-00000000a35e |       TASK | add reverse zone
2023-05-29 13:57:47 | 2023-05-29 13:57:47.339223 | 52540061-8350-eacb-f3f0-00000000a35e |         OK | add reverse zone | undercloud
2023-05-29 13:57:47 | 2023-05-29 13:57:47.341216 | 52540061-8350-eacb-f3f0-00000000a35e |     TIMING | tripleo_ipa_dns : add reverse zone | undercloud | 0:20:48.176291 | 3.31s
2023-05-29 13:57:47 | 2023-05-29 13:57:47.356846 | 52540061-8350-eacb-f3f0-00000000a369 |       TASK | add reverse zone
2023-05-29 13:57:50 | 2023-05-29 13:57:50.584182 | 52540061-8350-eacb-f3f0-00000000a369 |         OK | add reverse zone | undercloud
2023-05-29 13:57:50 | 2023-05-29 13:57:50.585863 | 52540061-8350-eacb-f3f0-00000000a369 |     TIMING | tripleo_ipa_dns : add reverse zone | undercloud | 0:20:51.420935 | 3.23s
2023-05-29 13:57:50 | 2023-05-29 13:57:50.603634 | 52540061-8350-eacb-f3f0-00000000a374 |       TASK | add reverse zone
2023-05-29 13:57:54 | 2023-05-29 13:57:54.111759 | 52540061-8350-eacb-f3f0-00000000a374 |         OK | add reverse zone | undercloud
2023-05-29 13:57:54 | 2023-05-29 13:57:54.114011 | 52540061-8350-eacb-f3f0-00000000a374 |     TIMING | tripleo_ipa_dns : add reverse zone | undercloud | 0:20:54.949038 | 3.51s
2023-05-29 13:57:54 | 2023-05-29 13:57:54.130549 | 52540061-8350-eacb-f3f0-00000000a37f |       TASK | add reverse zone
2023-05-29 13:57:57 | 2023-05-29 13:57:57.373806 | 52540061-8350-eacb-f3f0-00000000a37f |         OK | add reverse zone | undercloud
2023-05-29 13:57:57 | 2023-05-29 13:57:57.375504 | 52540061-8350-eacb-f3f0-00000000a37f |     TIMING | tripleo_ipa_dns : add reverse zone | undercloud | 0:20:58.210575 | 3.24s
2023-05-29 13:57:57 | 2023-05-29 13:57:57.393774 | 52540061-8350-eacb-f3f0-00000000a38a |       TASK | add reverse zone
2023-05-29 13:58:00 | 2023-05-29 13:58:00.593639 | 52540061-8350-eacb-f3f0-00000000a38a |         OK | add reverse zone | undercloud
2023-05-29 13:58:00 | 2023-05-29 13:58:00.595195 | 52540061-8350-eacb-f3f0-00000000a38a |     TIMING | tripleo_ipa_dns : add reverse zone | undercloud | 0:21:01.430269 | 3.20s
2023-05-29 13:58:00 | 2023-05-29 13:58:00.613937 | 52540061-8350-eacb-f3f0-00000000a395 |       TASK | add reverse zone
2023-05-29 13:58:03 | 2023-05-29 13:58:03.644455 | 52540061-8350-eacb-f3f0-00000000a395 |         OK | add reverse zone | undercloud
2023-05-29 13:58:03 | 2023-05-29 13:58:03.646076 | 52540061-8350-eacb-f3f0-00000000a395 |     TIMING | tripleo_ipa_dns : add reverse zone | undercloud | 0:21:04.481152 | 3.03s
2023-05-29 13:58:03 | 2023-05-29 13:58:03.661668 | 52540061-8350-eacb-f3f0-00000000a3a0 |       TASK | add reverse zone
2023-05-29 13:58:06 | 2023-05-29 13:58:06.732330 | 52540061-8350-eacb-f3f0-00000000a3a0 |         OK | add reverse zone | undercloud
2023-05-29 13:58:06 | 2023-05-29 13:58:06.734308 | 52540061-8350-eacb-f3f0-00000000a3a0 |     TIMING | tripleo_ipa_dns : add reverse zone | undercloud | 0:21:07.569381 | 3.07s
2023-05-29 13:58:06 | 2023-05-29 13:58:06.750602 | 52540061-8350-eacb-f3f0-00000000a3ab |       TASK | add reverse zone
2023-05-29 13:58:10 | 2023-05-29 13:58:10.069291 | 52540061-8350-eacb-f3f0-00000000a3ab |         OK | add reverse zone | undercloud
2023-05-29 13:58:10 | 2023-05-29 13:58:10.070918 | 52540061-8350-eacb-f3f0-00000000a3ab |     TIMING | tripleo_ipa_dns : add reverse zone | undercloud | 0:21:10.905992 | 3.32s
2023-05-29 13:58:10 | 2023-05-29 13:58:10.087003 | 52540061-8350-eacb-f3f0-00000000a3b6 |       TASK | add reverse zone
2023-05-29 13:58:13 | 2023-05-29 13:58:13.739074 | 52540061-8350-eacb-f3f0-00000000a3b6 |         OK | add reverse zone | undercloud
2023-05-29 13:58:13 | 2023-05-29 13:58:13.740917 | 52540061-8350-eacb-f3f0-00000000a3b6 |     TIMING | tripleo_ipa_dns : add reverse zone | undercloud | 0:21:14.575972 | 3.65s
2023-05-29 13:58:13 | 2023-05-29 13:58:13.758401 | 52540061-8350-eacb-f3f0-00000000a3c1 |       TASK | add reverse zone
2023-05-29 13:58:16 | 2023-05-29 13:58:16.969872 | 52540061-8350-eacb-f3f0-00000000a3c1 |         OK | add reverse zone | undercloud
2023-05-29 13:58:16 | 2023-05-29 13:58:16.971267 | 52540061-8350-eacb-f3f0-00000000a3c1 |     TIMING | tripleo_ipa_dns : add reverse zone | undercloud | 0:21:17.806344 | 3.21s
2023-05-29 13:58:16 | 2023-05-29 13:58:16.986805 | 52540061-8350-eacb-f3f0-00000000a3cc |       TASK | add reverse zone
2023-05-29 13:58:19 | 2023-05-29 13:58:19.990303 | 52540061-8350-eacb-f3f0-00000000a3cc |         OK | add reverse zone | undercloud
2023-05-29 13:58:19 | 2023-05-29 13:58:19.992273 | 52540061-8350-eacb-f3f0-00000000a3cc |     TIMING | tripleo_ipa_dns : add reverse zone | undercloud | 0:21:20.827345 | 3.00s
2023-05-29 13:58:20 | 2023-05-29 13:58:20.009131 | 52540061-8350-eacb-f3f0-00000000a3d7 |       TASK | add reverse zone
2023-05-29 13:58:23 | 2023-05-29 13:58:23.423920 | 52540061-8350-eacb-f3f0-00000000a3d7 |         OK | add reverse zone | undercloud
2023-05-29 13:58:23 | 2023-05-29 13:58:23.426213 | 52540061-8350-eacb-f3f0-00000000a3d7 |     TIMING | tripleo_ipa_dns : add reverse zone | undercloud | 0:21:24.261217 | 3.42s
2023-05-29 13:58:23 | 2023-05-29 13:58:23.442669 | 52540061-8350-eacb-f3f0-00000000a3e2 |       TASK | add reverse zone
2023-05-29 13:58:26 | 2023-05-29 13:58:26.990355 | 52540061-8350-eacb-f3f0-00000000a3e2 |         OK | add reverse zone | undercloud
2023-05-29 13:58:26 | 2023-05-29 13:58:26.993328 | 52540061-8350-eacb-f3f0-00000000a3e2 |     TIMING | tripleo_ipa_dns : add reverse zone | undercloud | 0:21:27.828389 | 3.55s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.011423 | 52540061-8350-eacb-f3f0-00000000a1a4 |       TASK | set create_reverse to false for elements of no_reverse_ips
2023-05-29 13:58:27 | 2023-05-29 13:58:27.077819 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'undercloud-0', 'zone_name': 'ctlplane.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '192.168.24.1'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.080192 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:27.915225 | 0.07s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.103532 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'overcloud', 'zone_name': 'ctlplane.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '192.168.24.150'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.104862 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:27.939935 | 0.09s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.128363 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'overcloud', 'zone_name': 'storage.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.3.150'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.129815 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:27.964888 | 0.12s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.142390 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'overcloud', 'zone_name': 'storagemgmt.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.4.150'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.143761 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:27.978836 | 0.13s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.167525 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'overcloud', 'zone_name': 'internalapi.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.1.150'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.169030 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.004104 | 0.16s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.182232 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'overcloud', 'zone_name': 'redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '10.0.0.150'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.184604 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.019668 | 0.17s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.198234 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-0', 'zone_name': 'redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.1.54'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.200198 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.035252 | 0.19s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.224329 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-0', 'zone_name': 'storage.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.3.44'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.225932 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.061006 | 0.21s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.238750 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-0', 'zone_name': 'storagemgmt.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.4.68'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.240426 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.075486 | 0.23s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.264477 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-0', 'zone_name': 'internalapi.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.1.54'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.265939 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.101014 | 0.25s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.279352 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-0', 'zone_name': 'tenant.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.2.121'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.280889 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.115962 | 0.27s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.305358 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-0', 'zone_name': 'external.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '10.0.0.107'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.307028 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.142103 | 0.29s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.320192 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-0', 'zone_name': 'ctlplane.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '192.168.24.38'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.321883 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.156958 | 0.31s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.346283 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-1', 'zone_name': 'redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.1.106'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.348008 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.183061 | 0.33s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.361398 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-1', 'zone_name': 'storage.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.3.69'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.363014 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.198087 | 0.35s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.388467 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-1', 'zone_name': 'storagemgmt.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.4.56'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.390268 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.225340 | 0.38s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.404491 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-1', 'zone_name': 'internalapi.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.1.106'}

2023-05-29 13:58:27 | 2023-05-29 13:58:27.407279 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.242342 | 0.39s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.421293 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-1', 'zone_name': 'tenant.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.2.141'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.422953 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.258028 | 0.41s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.448997 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-1', 'zone_name': 'external.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '10.0.0.132'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.450938 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.286012 | 0.44s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.475725 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-1', 'zone_name': 'ctlplane.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '192.168.24.8'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.477468 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.312531 | 0.46s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.491503 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-2', 'zone_name': 'redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.1.82'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.493327 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.328399 | 0.48s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.518378 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-2', 'zone_name': 'storage.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.3.46'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.520203 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.355276 | 0.51s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.534458 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-2', 'zone_name': 'storagemgmt.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.4.55'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.536502 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.371573 | 0.52s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.561702 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-2', 'zone_name': 'internalapi.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.1.82'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.563503 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.398566 | 0.55s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.577707 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-2', 'zone_name': 'tenant.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.2.149'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.579652 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.414714 | 0.57s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.604859 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-2', 'zone_name': 'external.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '10.0.0.146'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.606861 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.441932 | 0.59s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.621233 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'controller-2', 'zone_name': 'ctlplane.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '192.168.24.48'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.623173 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.458220 | 0.61s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.648664 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'compute-0', 'zone_name': 'redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.1.88'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.650662 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.485734 | 0.64s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.664764 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'compute-0', 'zone_name': 'storage.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.3.143'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.666637 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.501710 | 0.65s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.692360 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'compute-0', 'zone_name': 'internalapi.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.1.88'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.694438 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.529511 | 0.68s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.709081 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'compute-0', 'zone_name': 'tenant.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '172.17.2.58'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.711085 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.546158 | 0.70s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.736903 | 52540061-8350-eacb-f3f0-00000000a1a4 |         OK | set create_reverse to false for elements of no_reverse_ips | undercloud | item={'record_name': 'compute-0', 'zone_name': 'ctlplane.redhat.local', 'record_type': 'A', 'create_reverse': 'true', 'a_rec': '192.168.24.33'}
2023-05-29 13:58:27 | 2023-05-29 13:58:27.739008 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.574074 | 0.73s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.775661 | 52540061-8350-eacb-f3f0-00000000a1a4 |     TIMING | tripleo_ipa_dns : set create_reverse to false for elements of no_reverse_ips | undercloud | 0:21:28.610706 | 0.76s
2023-05-29 13:58:27 | 2023-05-29 13:58:27.824675 | 52540061-8350-eacb-f3f0-00000000a1a5 |       TASK | add dns forward and reverse records
2023-05-29 13:58:31 | 2023-05-29 13:58:31.129621 | 52540061-8350-eacb-f3f0-00000000a1a5 |      FATAL | add dns forward and reverse records | undercloud | error={"changed": false, "msg": "'name'"}
2023-05-29 13:58:31 | 2023-05-29 13:58:31.134010 | 52540061-8350-eacb-f3f0-00000000a1a5 |     TIMING | tripleo_ipa_dns : add dns forward and reverse records | undercloud | 0:21:31.969042 | 3.31s

Comment 6 Ade Lee 2023-06-01 08:47:12 UTC

Unfortunately, the output doesn't seem to show much in terms of the real failure -- which takes place in the last line.
We probably need --debug for more info.

What is the version of ansible-freeipa when the error occurs?

Comment 7 Juan Badia Payno 2023-06-01 13:41:17 UTC

The package is 
2023-05-30T21:40:15+0000 DEBUG Installed: ansible-freeipa-0.3.2-2.el8.noarch

Comment 9 Ade Lee 2023-06-05 13:40:53 UTC

I don't see anything obvious here in the changelog for these builds

Thomas,

Any thoughts here?  The code that is failing is:

https://opendev.org/x/tripleo-ipa/src/branch/stable/wallaby/tripleo_ipa/roles/tripleo_ipa_dns/tasks/main.yml#L56-L59

It fails when the version of ansible-freeipa is ansible-freeipa-0.3.2-2.el8.noarch , but passes for https://download.eng.bos.redhat.com/brewroot/vol/rhel-8/packages/ansible-freeipa/1.9.2/2.el8_8/noarch/ansible-freeipa-1.9.2-2.el8_8.noarch.rpm

Is there something that would have caused multiple DNS entry updates to fail in the lower build?
Or maybe the format changed?

For reference - https://opendev.org/x/tripleo-ipa/src/branch/stable/wallaby/tripleo_ipa/ansible_plugins/filter/dns_data.py#L73-L85 shows how the entries were constructed.

Comment 11 Lukas Svaty 2023-06-08 11:05:20 UTC

Raising severity due to blocker?

Comment 23 Lon Hohberger 2023-06-20 16:53:50 UTC


*** This bug has been marked as a duplicate of bug 2214972 ***

Comment 26 Red Hat Bugzilla 2023-12-02 04:26:42 UTC

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days