Bug 1374937

Summary: rhel-osp-director: The introspection doesn't exit - gets stuck on "Setting manageable nodes to available..."
Product: Red Hat OpenStack Reporter: Alexander Chuzhoy <sasha>
Component: openstack-tripleo-commonAssignee: Dougal Matthews <dmatthew>
Status: CLOSED CURRENTRELEASE QA Contact: Alexander Chuzhoy <sasha>
Severity: unspecified Docs Contact:
Priority: high    
Version: 10.0 (Newton)CC: dbecker, mburns, mcornea, mgould, morazi, rhel-osp-director-maint, sasha, slinaber
Target Milestone: ga   
Target Release: 10.0 (Newton)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-10-05 09:57:28 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
ironic log none

Description Alexander Chuzhoy 2016-09-10 17:16:31 UTC 
rhel-osp-director:   The introspection doesn't exit - gets stuck on "Setting manageable nodes to available..."

environment:
puppet-ironic-9.1.0-0.20160823143141.dd03027.el7ost.noarch
python-ironicclient-1.6.0-0.20160802195836.f02f43d.el7ost.noarch
openstack-ironic-inspector-4.1.1-0.20160823024001.faac9e1.el7ost.noarch
python-ironic-tests-6.1.1-0.20160823055226.46dc395.el7ost.noarch
instack-undercloud-5.0.0-0.20160818065636.41ef775.el7ost.noarch
openstack-ironic-conductor-6.1.1-0.20160823055226.46dc395.el7ost.noarch
python-ironic-lib-2.0.0-0.20160802195403.6fec307.el7ost.noarch
openstack-ironic-api-6.1.1-0.20160823055226.46dc395.el7ost.noarch
python-ironic-inspector-client-1.8.0-0.20160802201835.fdbefc5.el7ost.noarch
openstack-ironic-common-6.1.1-0.20160823055226.46dc395.el7ost.noarch


Steps to reproduce:
1. Deploy undercloud.
2. Register overcloud images in glance, register nodes in ironic and run introspection.

Result:
The introspection doesn't exit:
[stack@undercloud ~]$     openstack baremetal configure boot && openstack baremetal introspection bulk start
This command is deprecated. Please use "openstack overcloud node introspect" to introspect manageable nodes instead.

Setting nodes for introspection to manageable...
Starting introspection of manageable nodes
Waiting for introspection to finish...



Introspection for UUID 035c606e-738b-4355-acaa-8d25896b788c finished successfully.
Introspection for UUID 23a0d53d-f34b-422e-ba1b-3424b8ed6128 finished successfully.
Introspection for UUID 713e2e61-95c5-4489-a52a-27cae72c4af2 finished successfully.
Introspection for UUID c28d6745-a15e-42cd-98de-105df6741fdb finished successfully.
Introspection for UUID f21a4490-8d5e-4f88-9a7b-017a9871bda4 finished successfully.
Introspection for UUID e35c21bd-0858-4258-ad59-9014a218b2dc finished successfully.
Introspection for UUID 1736092b-1652-45d5-bb71-339e1d8d64ac finished successfully.
Introspection for UUID 37a4767b-2201-4696-9b7d-62fbec0b201d finished successfully.
No nodes in manageable state found for introspection.
Introspection completed.
Setting manageable nodes to available...




Checking from another terminal I see that all the nodes were set to available:
[stack@undercloud ~]$ ironic node-list
+--------------------------------------+------+---------------+-------------+--------------------+-------------+
| UUID                                 | Name | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+------+---------------+-------------+--------------------+-------------+
| c28d6745-a15e-42cd-98de-105df6741fdb | None | None          | power off   | available          | False       |
| 035c606e-738b-4355-acaa-8d25896b788c | None | None          | power off   | available          | False       |
| f21a4490-8d5e-4f88-9a7b-017a9871bda4 | None | None          | power off   | available          | False       |
| 37a4767b-2201-4696-9b7d-62fbec0b201d | None | None          | power off   | available          | False       |
| 713e2e61-95c5-4489-a52a-27cae72c4af2 | None | None          | power off   | available          | False       |
| 1736092b-1652-45d5-bb71-339e1d8d64ac | None | None          | power off   | available          | False       |
| e35c21bd-0858-4258-ad59-9014a218b2dc | None | None          | power off   | available          | False       |
| 23a0d53d-f34b-422e-ba1b-3424b8ed6128 | None | None          | power off   | available          | False       |
+--------------------------------------+------+---------------+-------------+--------------------+-------------+




Hitting Ctrl+C to break the run of introspection and checking the status I see that the introspection is completed:
[stack@undercloud ~]$ openstack baremetal introspection bulk status
+--------------------------------------+----------+-------+
| Node UUID                            | Finished | Error |
+--------------------------------------+----------+-------+
| c28d6745-a15e-42cd-98de-105df6741fdb | True     | None  |
| 035c606e-738b-4355-acaa-8d25896b788c | True     | None  |
| f21a4490-8d5e-4f88-9a7b-017a9871bda4 | True     | None  |
| 37a4767b-2201-4696-9b7d-62fbec0b201d | True     | None  |
| 713e2e61-95c5-4489-a52a-27cae72c4af2 | True     | None  |
| 1736092b-1652-45d5-bb71-339e1d8d64ac | True     | None  |
| e35c21bd-0858-4258-ad59-9014a218b2dc | True     | None  |
| 23a0d53d-f34b-422e-ba1b-3424b8ed6128 | True     | None  |
+--------------------------------------+----------+-------+
Comment 2 Alexander Chuzhoy 2016-09-10 17:19:52 UTC 
Created attachment 1199793 [details]
ironic log
Comment 3 Dmitry Tantsur 2016-09-12 12:48:19 UTC 
As you see by "Introspection completed", this has nothing to do with ironic-inspector.
Comment 4 Miles Gould 2016-10-03 16:04:32 UTC 
Hi Alexander, can you still reproduce this error with the latest puddle?
Comment 5 Alexander Chuzhoy 2016-10-04 21:46:42 UTC 
Hi,
I didn't reproduce it.
Thanks.