Bug 1320159

Summary: Bulk introspection fails with baremetal
Product: Red Hat OpenStack Reporter: Joe Talerico <jtaleric>
Component: openstack-ironic-inspectorAssignee: Dmitry Tantsur <dtantsur>
Status: CLOSED WORKSFORME QA Contact: Raviv Bar-Tal <rbartal>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 9.0 (Mitaka)CC: chris.brown, jason.dobies, jcoufal, jtaleric, mburns, rbartal, slinaber, tvignaud
Target Milestone: ---Keywords: Triaged
Target Release: 10.0 (Newton)   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-09-11 08:23:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
inspector log none

Description Joe Talerico 2016-03-22 12:59:16 UTC
Created attachment 1139057 [details]
inspector log

Description of problem:
Bulk introspection fails with baremetal

Version-Release number of selected component (if applicable):
openstack-ironic-inspector-3.0.1-0.20160303184238.e8ecb99.el7.centos.noarch
python-ironic-inspector-client-1.4.1-0.20160229171554.f9a9e27.el7.centos.noarch

How reproducible:
N/A

Steps to Reproduce:
1. Inspect 4 baremetal nodes with bulk introspection

Actual results:

Introspection for UUID 57c2d59e-6e39-4f6b-833e-02a77338c4c2 finished successfully.
Introspection for UUID c959d164-5e0b-43ad-9268-e3724c711174 finished successfully.
Introspection for UUID 227a6503-e38b-4d3a-b517-10c49413e990 finished successfully.
Introspection for UUID fe043cac-3036-4365-9038-f46b84f57013 finished with error: Preprocessing hook validate_interfaces: No suitable interfaces found in {u'ib0': {'ip': None, 'mac': u'80:00:02:40:fe:80:00:00:00:00:00:00:f4:52:14:03:00:31:ae:22'}, u'eth4': {'ip': None, 'mac': u'f4:52:14:31:ae:21'}, u'eth3': {'ip': None, 'mac': u'40:a8:f0:20:65:5f'}, u'eth2': {'ip': None, 'mac': u'40:a8:f0:20:65:5e'}, u'eth1': {'ip': None, 'mac': u'40:a8:f0:20:65:5d'}, u'eth0': {'ip': None, 'mac': u'40:a8:f0:20:65:5c'}}
Introspection for UUID e3c9a81c-81cd-41fa-803c-06dc92f9f170 finished successfully.
Setting manageable nodes to available...
Node 57c2d59e-6e39-4f6b-833e-02a77338c4c2 has been set to available.
Node 227a6503-e38b-4d3a-b517-10c49413e990 has been set to available.
Node e3c9a81c-81cd-41fa-803c-06dc92f9f170 has been set to available.
Node c959d164-5e0b-43ad-9268-e3724c711174 has been set to available.
Introspection completed with errors:
fe043cac-3036-4365-9038-f46b84f57013: Preprocessing hook validate_interfaces: No suitable interfaces found in {u'ib0': {'ip': None, 'mac': u'80:00:02:40:fe:80:00:00:00:00:00:00:f4:52:14:03:00:31:ae:22'}, u'eth4': {'ip': None, 'mac': u'f4:52:14:31:ae:21'}, u'eth3': {'ip': None, 'mac': u'40:a8:f0:20:65:5f'}, u'eth2': {'ip': None, 'mac': u'40:a8:f0:20:65:5e'}, u'eth1': {'ip': None, 'mac': u'40:a8:f0:20:65:5d'}, u'eth0': {'ip': None, 'mac': u'40:a8:f0:20:65:5c'}}

Expected results:
All nodes to introspect.

Additional info:
Manually running openstack baremetal introspection on the failed node, fixes the issue.

Comment 2 Dmitry Tantsur 2016-03-28 17:56:02 UTC
Looks like for some reason DHCP did not work for the nodes. Please grab the ramdisk logs, they can be found in /var/log/ironic-inspector/ramdisk as tarballs with weird names :)

Also please grab the output of 'sudo journalctl -u openstack-ironic-inspector-dnsmasq'

Comment 3 Jaromir Coufal 2016-06-08 18:16:22 UTC
Joe, could you provide the needed info? If not re-producable we should close the bug. Thanks, Jarda

Comment 4 Joe Talerico 2016-06-08 20:19:41 UTC
Jarda can someone from QE attempt to recreate? If not then go ahead and close the bug.

Comment 5 Raviv Bar-Tal 2016-09-11 08:23:24 UTC
I have introspection BM machines many times with Mitack release and did not find specific with introspection or the DHCP service.

Joe if you have more information or reproduce steps for you problem, Please open new bug with the information and relevant Ironic logs