Bug 1306417

Summary: rhel-osp-director: introspection doesn't complete for a subset of nodes.
Product: Red Hat OpenStack Reporter: Alexander Chuzhoy <sasha>
Component: rhosp-directorAssignee: Angus Thomas <athomas>
Status: CLOSED DUPLICATE QA Contact: yeylon <yeylon>
Severity: high Docs Contact:
Priority: high    
Version: 7.0 (Kilo)CC: dbecker, mburns, mkovacik, morazi, ohochman, rhel-osp-director-maint, srevivo
Target Milestone: ga   
Target Release: 8.0 (Liberty)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-02-29 12:40:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Alexander Chuzhoy 2016-02-10 19:17:03 UTC 
rhel-osp-director: introspection doesn't complete for a subset of nodes.


Environment:
openstack-ironic-api-2015.1.2-2.el7ost.noarch
instack-undercloud-2.1.2-39.el7ost.noarch
python-ironic-discoverd-1.1.0-8.el7ost.noarch
openstack-ironic-conductor-2015.1.2-2.el7ost.noarch
openstack-ironic-common-2015.1.2-2.el7ost.noarch
python-ironicclient-0.5.1-12.el7ost.noarch
openstack-ironic-discoverd-1.1.0-8.el7ost.noarch

Steps to reproduce:
Run bulk introspection on at least 8 nodes.

Result:
The introspection doesn't complete for 1 node out of 8.


"ERROR: rdomanager_oscplugin.utils.wait_for_node_discovery Discovery
didn't finish for node....

in the log file I see:
Feb 10 09:32:51 instack dnsmasq-dhcp[30169]: DHCPACK(br-ctlplane) 192.0.2.102 00:c8:37:bc:d0:89
Feb 10 09:32:51 instack dnsmasq-dhcp[30169]: DHCPREQUEST(br-ctlplane) 192.0.2.116 00:ac:98:c8:d1:d1
Feb 10 09:32:51 instack dnsmasq-dhcp[30169]: DHCPNAK(br-ctlplane) 192.0.2.116 00:ac:98:c8:d1:d1 address in use
Feb 10 09:32:51 instack dnsmasq-dhcp[30169]: DHCPREQUEST(br-ctlplane) 192.0.2.116 00:ac:98:c8:d1:d1
Feb 10 09:32:51 instack dnsmasq-dhcp[30169]: DHCPNAK(br-ctlplane) 192.0.2.116 00:ac:98:c8:d1:d1 address in use
Feb 10 09:32:51 instack dnsmasq-dhcp[30169]: DHCPREQUEST(br-ctlplane) 192.0.2.102 00:c8:37:bc:d0:89
Feb 10 09:32:51 instack dnsmasq-dhcp[30169]: DHCPACK(br-ctlplane) 192.0.2.102 00:c8:37:bc:d0:89
Feb 10 09:32:51 instack dnsmasq-dhcp[30169]: DHCPREQUEST(br-ctlplane) 192.0.2.116 00:ac:98:c8:d1:d1
Feb 10 09:32:51 instack dnsmasq-dhcp[30169]: DHCPNAK(br-ctlplane) 192.0.2.116 00:ac:98:c8:d1:d1 address in use


Expected result:
The introspection should complete for all.
Comment 1 Angus Thomas 2016-02-11 14:16:15 UTC 
I'm pushing this to the next release, because we don't have a consistent reproducer, and it is very likely a timeout caused by underspecced test hardware.
Comment 2 Omri Hochman 2016-02-11 18:57:54 UTC 
reproduced on 7.3GA  : 
---------------------
openstack-tripleo-heat-templates-0.8.6-94.el7ost.noarch
openstack-ironic-api-2015.1.2-2.el7ost.noarch
python-ironic-discoverd-1.1.0-8.el7ost.noarch
openstack-ironic-conductor-2015.1.2-2.el7ost.noarch
openstack-ironic-common-2015.1.2-2.el7ost.noarch
python-ironicclient-0.5.1-12.el7ost.noarch
openstack-ironic-discoverd-1.1.0-8.el7ost.noarch
Comment 3 mkovacik 2016-02-29 12:40:15 UTC 

*** This bug has been marked as a duplicate of bug 1301659 ***