Bug 1296678

Summary:	rhel-osp-director: 8.0 bulk introspecting 8 nodes with default range (192.0.2.100,192.0.2.120), the last node doesn't get an IP and causes the introspection to stuck.
Product:	Red Hat OpenStack	Reporter:	Alexander Chuzhoy <sasha>
Component:	rhosp-director	Assignee:	Hugh Brock <hbrock>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Shai Revivo <srevivo>
Severity:	high	Docs Contact:
Priority:	high
Version:	8.0 (Liberty)	CC:	dtantsur, jcoufal, mburns, mcornea, rhel-osp-director-maint
Target Milestone:	---
Target Release:	10.0 (Newton)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-10-14 15:43:06 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Alexander Chuzhoy 2016-01-07 20:05:09 UTC

rhel-osp-director: 8.0 bulk introspecting 8 nodes with default range (192.0.2.100,192.0.2.120), the last node doesn't get an IP and causes the introspection to stuck.

Environment:
openstack-ironic-conductor-4.2.2-3.el7ost.noarch
python-ironic-inspector-client-1.2.0-5.el7ost.noarch
dnsmasq-2.66-14.el7_1.x86_64
openstack-ironic-inspector-2.2.2-1.el7ost.noarch
openstack-ironic-api-4.2.2-3.el7ost.noarch
python-ironicclient-0.8.1-1.el7ost.noarch
openstack-ironic-common-4.2.2-3.el7ost.noarch
dnsmasq-utils-2.66-14.el7_1.x86_64

Steps to reproduce:
Attempt to bulk introspect 8 nodes or more with the default introspection range.

Result:
The introspection doesn't finish.
Checking the console of the node - it doesn't get an IP.
Looking at the logs:
Jan 07 20:31:56 instack.localdomain dnsmasq-dhcp[30022]: DHCPNAK(br-ctlplane) 192.0.2.105 00:ed:dc:54:1d:42 address in use
Jan 07 20:31:57 instack.localdomain dnsmasq-dhcp[30022]: DHCPNAK(br-ctlplane) 192.0.2.105 00:ed:dc:54:1d:42 address in use
Jan 07 20:31:57 instack.localdomain dnsmasq-dhcp[30022]: DHCPNAK(br-ctlplane) 192.0.2.105 00:ed:dc:54:1d:42 address in use

Note: Enlarged the range and restarted the openstack-ironic-inspector.service and openstack-ironic-inspector.service services. Restarted the introspection - completed successfully. Reduced the range to default and restarted the same services - the introspection got stuck again with the same issue.

Expected result:
Bulk introspection completed successfully for all nodes as long as their count doesn't go above the range.

Comment 3 Mike Burns 2016-04-07 21:03:37 UTC

This bug did not make the OSP 8.0 release.  It is being deferred to OSP 10.

Comment 5 Dmitry Tantsur 2016-10-14 15:43:06 UTC

Hi! I believe this problem is fixed now that we're using dhcp-sequential-ip for introspection dnsmasq (see https://github.com/openstack/puppet-ironic/commit/fc2496a4ef7f123bf4cdb3cb1ea7c32ce8c23262). Please feel free to reopen the bug if you can reproduce it with the latest bits.