Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1698540

Summary:	Registering Nodes for the Overcloud hangs when undercloud.conf has custom undercloud_nameservers
Product:	Red Hat OpenStack	Reporter:	Yuri Obshansky <yobshans>
Component:	openstack-ironic-inspector	Assignee:	RHOS Maint <rhos-maint>
Status:	CLOSED DUPLICATE	QA Contact:	mlammon
Severity:	urgent	Docs Contact:
Priority:	unspecified
Version:	15.0 (Stein)	CC:	bfournie, jslagle, mburns, michele, slinaber
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-04-11 04:53:41 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Yuri Obshansky 2019-04-10 14:47:32 UTC

Description of problem:
Registering Nodes for the Overcloud hangs when Undercloud installed with custom parameter undercloud_nameservers
Spine-leaf network topology on virt environment:

# cat undercloud.conf 
[DEFAULT]
local_interface = eth0
local_ip = 192.168.24.1/24
undercloud_ntp_servers=clock.redhat.com
undercloud_nameservers=10.11.5.19
hieradata_override = /home/stack/hiera_override.yaml
container_images_file=/home/stack/containers-prepare-parameter.yaml
container_insecure_registries=brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888
undercloud_service_certificate = /etc/pki/instack-certs/undercloud.pem
enable_routed_networks = true
subnets = leaf0,leaf1,leaf2
local_subnet = leaf0

[leaf0]
cidr = 192.168.24.0/24
dhcp_start = 192.168.24.10
dhcp_end = 192.168.24.90
inspection_iprange = 192.168.24.100,192.168.24.190
gateway = 192.168.24.254
masquerade = False

[leaf1]
cidr = 192.168.34.0/24
dhcp_start = 192.168.34.10
dhcp_end = 192.168.34.90
inspection_iprange = 192.168.34.100,192.168.34.190
gateway = 192.168.34.254
masquerade = False

[leaf2]
cidr = 192.168.44.0/24
dhcp_start = 192.168.44.10
dhcp_end = 192.168.44.90
inspection_iprange = 192.168.44.100,192.168.44.190
gateway = 192.168.44.254
masquerade = False

Generated on undercloud
$ cat /etc/resolv.conf
# Generated by NetworkManager
search redhat.local
# NOTE: the libc resolver may not support more than 3 nameservers.
# The nameservers listed below may not be recognized.
nameserver 10.11.5.19

Subnets has updated also
$ openstack subneshow leaf0 |grep dns
(undercloud) [stack@site-undercloud-0 ~]$ openstack subnet show leaf0 |grep dns
| dns_nameservers   | 10.11.5.19                                                                                                                                                                        (undercloud) [stack@site-undercloud-0 ~]$ openstack subnet show leaf1 |grep dns
| dns_nameservers   | 10.11.5.19                                                                                                                                                                        (undercloud) [stack@site-undercloud-0 ~]$ openstack subnet show leaf2 |grep dns
| dns_nameservers   | 10.11.5.19                            

Errors:
$ sudo cat dnsmasq.log 
Apr 10 12:15:25 dnsmasq[9]: started, version 2.79 DNS disabled
Apr 10 12:15:25 dnsmasq[9]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN2 DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth DNSSEC loop-detect inotify
Apr 10 12:15:25 dnsmasq-dhcp[9]: DHCP, IP range 192.168.44.100 -- 192.168.44.190, lease time 10m
Apr 10 12:15:25 dnsmasq-dhcp[9]: DHCP, IP range 192.168.34.100 -- 192.168.34.190, lease time 10m
Apr 10 12:15:25 dnsmasq-dhcp[9]: DHCP, IP range 192.168.24.100 -- 192.168.24.190, lease time 10m
Apr 10 12:15:27 dnsmasq[9]: inotify, new or changed file /var/lib/ironic-inspector/dhcp-hostsdir/unknown_hosts_filter
Apr 10 12:15:27 dnsmasq-dhcp[9]: read /var/lib/ironic-inspector/dhcp-hostsdir/unknown_hosts_filter
Apr 10 12:40:42 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:43 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:43 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:43 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:43 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:43 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:43 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:46 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:46 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:46 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:46 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:46 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:46 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:47 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:47 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:47 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:47 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:47 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:47 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:49 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:49 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:49 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:49 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:49 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:40:49 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:41:21 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:41:21 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:41:22 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:41:22 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:41:22 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:41:22 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:41:22 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 12:41:22 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 13:00:08 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 13:00:12 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 13:00:12 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 13:00:12 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 13:00:12 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 13:00:12 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 13:00:12 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 13:00:12 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.35.64.12
Apr 10 13:12:35 dnsmasq-dhcp[9]: no address range available for DHCP request via 172.16.10.1
Apr 10 13:12:40 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.0.10.1
Apr 10 13:39:13 dnsmasq-dhcp[9]: no address range available for DHCP request via 172.16.10.1
Apr 10 13:39:42 dnsmasq-dhcp[9]: no address range available for DHCP request via 10.0.10.1

Verified DNS on undercloud
$ nmcli dev show | grep DNS
IP4.DNS[1]:                             172.16.10.1
IP4.DNS[1]:                             10.0.10.1
IP6.DNS[1]:                             fe80::5054:ff:fe17:d6e6
IP6.DNS[2]:                             2620:52:0:13b8::fe

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. 
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 James Slagle 2019-04-10 15:53:22 UTC

Yuri, can you put exactly what command you're using here that failed? Was it node introspection or registration? Registration doesn't do anything with dnsmasq/dhcp, so I think you might mean introspection.

You will probably also want to attach your full undercloud.conf and the networking configuration that got applied on the undercloud (e.g. "ip a" for starters).

Comment 2 Yuri Obshansky 2019-04-10 16:10:15 UTC

Looks like duplication of 
https://bugzilla.redhat.com/show_bug.cgi?id=1698555
Issue is not related to custom undercloud parameter undercloud_nameservers