Bug 1404902

Summary:	openstack baremetal introspection bulk status gives error when some nodes are maintenance true
Product:	Red Hat OpenStack	Reporter:	Kevin Jones <kejones>
Component:	python-tripleoclient	Assignee:	Dmitry Tantsur <dtantsur>
Status:	CLOSED ERRATA	QA Contact:	mlammon
Severity:	high	Docs Contact:
Priority:	high
Version:	10.0 (Newton)	CC:	aschultz, bfournie, dbecker, hbrock, jjoyce, jslagle, mburns, mlammon, morazi, rhel-osp-director-maint
Target Milestone:	z4	Keywords:	Triaged, ZStream
Target Release:	10.0 (Newton)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	python-tripleoclient-5.4.2-2.el7ost	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-09-06 17:09:30 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Kevin Jones 2016-12-15 03:28:48 UTC

Description of problem:
After issuing introspection bulk start with some nodes maintenance true, introspection bulk status throws a no node in cache message

[stack@undercloud ~]$ ironic node-list
+--------------------------------------+---------------------+---------------+-------------+--------------------+-------------+
| UUID                                 | Name                | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+---------------------+---------------+-------------+--------------------+-------------+
| 86c63ea0-d88e-4f93-93f0-46789c8fe1c7 | overcloud-control0  | None          | power on    | manageable         | False       |
| 1a5aed27-2457-4678-8b35-06f4a32880a8 | overcloud-control1  | None          | power on    | manageable         | False       |
| 8acec886-3b9c-4161-8228-4d0ee344ec54 | overcloud-control2  | None          | power on    | manageable         | False       |
| 1f052927-3ac5-4d7c-8138-b40c4b74a1cb | overcloud-compute0  | None          | power on    | manageable         | False       |
| 225ab0a4-00e7-49d9-ba39-f36b8c4fb298 | overcloud-compute1  | None          | power on    | manageable         | False       |
| d383fc5e-2b97-4f1d-8aa9-2cebddab205c | overcloud-compute2  | None          | power on    | manageable         | False       |
| d182967e-0383-41c2-a0ec-9f0e4a95348f | overcloud-compute3  | None          | power on    | manageable         | False       |
| 7b47616e-2145-470a-bfdb-75e8812025fa | overcloud-compute4  | None          | power on    | manageable         | False       |
| b98d461e-5159-4ce8-b26b-8c8aefba9c63 | overcloud-ceph-hci0 | None          | power off   | available          | True        |
| cb336f2d-97b9-431d-a8d0-0a6565fc186d | overcloud-ceph-hci1 | None          | power off   | available          | True        |
| a81ef45d-c336-41a8-98ed-8b64051c381d | overcloud-ceph-hci2 | None          | power on    | manageable         | False       |
+--------------------------------------+---------------------+---------------+-------------+--------------------+-------------+
[stack@undercloud ~]$ openstack baremetal introspection bulk status
Could not find node b98d461e-5159-4ce8-b26b-8c8aefba9c63 in cache



Version-Release number of selected component (if applicable):
RHOSP10

How reproducible:
Every time

Steps to Reproduce:
1. Import nodes into ironic
2. Configure boot
3. openstack baremetal introspection bulk start
4. openstack baremetal introspection bulk status

Actual results:
Could not find node b98d461e-5159-4ce8-b26b-8c8aefba9c63 in cache message is thrown


Expected results:
Should show status for the nodes that are being introspected and ignore those in maintenance mode

Additional info:

Comment 3 mlammon 2017-08-23 18:22:00 UTC

This bug can now be verified.  There were seven nodes in instackenv.json.  There were two nodes placed into maintenance mode = True
Introspection completed successfully for five remaining nodes without errors (see below)

Basic steps to verify:
1. Install RHOSP 10 latest undercloud (2017-08-22.2) and configure
2. openstack baremetal import --json instackenv.json
3. openstack baremetal node maintenance set <uuid> # put two nodes in maintenance mode True
4. openstack baremetal configure boot
5. openstack baremetal introspection bulk start
6. openstack baremetal introspection bulk status
7. openstack baremetal node maintenance unset <uuid> # put two nodes in maintenance mode False
8. openstack baremetal configure boot
9. openstack baremetal introspection bulk start
10. openstack baremetal introspection bulk status


Environment:
python-tripleoclient-5.4.2-2.el7ost.noarch

openstack-ironic-api-6.2.4-1.el7ost.noarch
openstack-ironic-common-6.2.4-1.el7ost.noarch
openstack-ironic-inspector-4.2.2-3.el7ost.noarch
openstack-ironic-conductor-6.2.4-1.el7ost.noarch

[stack@undercloud74 ~]$ source stackrc && openstack baremetal import --json instackenv.json
Started Mistral Workflow. Execution ID: 65eaaa47-8c65-453f-81ae-ec3ef19d39a4
Successfully registered node UUID 334c6d9b-3851-40ac-88f4-9d013aacc1cc
Successfully registered node UUID c491ad8f-97bd-469b-8e65-afc348473d90
Successfully registered node UUID fbe20c2e-99e0-4fd2-993f-f75f0ccc2d0b
Successfully registered node UUID a7658693-1698-4ac5-a5c8-f9af5bd72cfd
Successfully registered node UUID d9ab970e-dcd1-4ee2-b80a-88243798c53e
Successfully registered node UUID d6309297-c92b-4ab4-8ffb-70068f6d855b
Successfully registered node UUID 89d64528-c605-439d-a157-9318527f7e8d
Started Mistral Workflow. Execution ID: f8e60f96-66c5-4037-8d07-8d5688cce8aa
Successfully set all nodes to available.
[stack@undercloud74 ~]$ ironic node-list
+--------------------------------------+------+---------------+-------------+--------------------+-------------+
| UUID                                 | Name | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+------+---------------+-------------+--------------------+-------------+
| 334c6d9b-3851-40ac-88f4-9d013aacc1cc | None | None          | power off   | available          | False       |
| c491ad8f-97bd-469b-8e65-afc348473d90 | None | None          | power off   | available          | False       |
| fbe20c2e-99e0-4fd2-993f-f75f0ccc2d0b | None | None          | power off   | available          | False       |
| a7658693-1698-4ac5-a5c8-f9af5bd72cfd | None | None          | power off   | available          | False       |
| d9ab970e-dcd1-4ee2-b80a-88243798c53e | None | None          | power off   | available          | False       |
| d6309297-c92b-4ab4-8ffb-70068f6d855b | None | None          | power off   | available          | False       |
| 89d64528-c605-439d-a157-9318527f7e8d | None | None          | power off   | available          | False       |
+--------------------------------------+------+---------------+-------------+--------------------+-------------+

[stack@undercloud74 ~]$ openstack baremetal node maintenance set 334c6d9b-3851-40ac-88f4-9d013aacc1cc
[stack@undercloud74 ~]$ openstack baremetal node maintenance set 89d64528-c605-439d-a157-9318527f7e8d
[stack@undercloud74 ~]$ ironic node-list
+--------------------------------------+------+---------------+-------------+--------------------+-------------+
| UUID                                 | Name | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+------+---------------+-------------+--------------------+-------------+
| 334c6d9b-3851-40ac-88f4-9d013aacc1cc | None | None          | power off   | available          | True        |
| c491ad8f-97bd-469b-8e65-afc348473d90 | None | None          | power off   | available          | False       |
| fbe20c2e-99e0-4fd2-993f-f75f0ccc2d0b | None | None          | power off   | available          | False       |
| a7658693-1698-4ac5-a5c8-f9af5bd72cfd | None | None          | power off   | available          | False       |
| d9ab970e-dcd1-4ee2-b80a-88243798c53e | None | None          | power off   | available          | False       |
| d6309297-c92b-4ab4-8ffb-70068f6d855b | None | None          | power off   | available          | False       |
| 89d64528-c605-439d-a157-9318527f7e8d | None | None          | power off   | available          | True        |
+--------------------------------------+------+---------------+-------------+--------------------+-------------+

[stack@undercloud74 ~]$ openstack baremetal configure boot && openstack baremetal introspection bulk start
Setting nodes for introspection to manageable...
Starting introspection of manageable nodes
Started Mistral Workflow. Execution ID: cb036573-d101-4423-810c-527e03cb81fa
Waiting for introspection to finish...
Introspection for UUID a7658693-1698-4ac5-a5c8-f9af5bd72cfd finished successfully.
Introspection for UUID d6309297-c92b-4ab4-8ffb-70068f6d855b finished successfully.
Introspection for UUID fbe20c2e-99e0-4fd2-993f-f75f0ccc2d0b finished successfully.
Introspection for UUID d9ab970e-dcd1-4ee2-b80a-88243798c53e finished successfully.
Introspection for UUID c491ad8f-97bd-469b-8e65-afc348473d90 finished successfully.
Introspection completed.
Setting manageable nodes to available...
Started Mistral Workflow. Execution ID: 63b0e4b6-b892-4cbc-a8e7-7264896cef79


[stack@undercloud74 ~]$ openstack baremetal introspection bulk status
+--------------------------------------+----------+-------+
| Node UUID                            | Finished | Error |
+--------------------------------------+----------+-------+
| c491ad8f-97bd-469b-8e65-afc348473d90 | True     | None  |
| fbe20c2e-99e0-4fd2-993f-f75f0ccc2d0b | True     | None  |
| a7658693-1698-4ac5-a5c8-f9af5bd72cfd | True     | None  |
| d9ab970e-dcd1-4ee2-b80a-88243798c53e | True     | None  |
| d6309297-c92b-4ab4-8ffb-70068f6d855b | True     | None  |
+--------------------------------------+----------+-------+


## REMOVE TWO NODES FROM MAINTENANCE MODE && config/bulk start again
[stack@undercloud74 ~]$ openstack baremetal node maintenance unset 334c6d9b-3851-40ac-88f4-9d013aacc1cc
[stack@undercloud74 ~]$ openstack baremetal node maintenance unset 89d64528-c605-439d-a157-9318527f7e8d
[stack@undercloud74 ~]$ ironic node-list
+--------------------------------------+------+---------------+-------------+--------------------+-------------+
| UUID                                 | Name | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+------+---------------+-------------+--------------------+-------------+
| 334c6d9b-3851-40ac-88f4-9d013aacc1cc | None | None          | power off   | available          | False       |
| c491ad8f-97bd-469b-8e65-afc348473d90 | None | None          | power off   | available          | False       |
| fbe20c2e-99e0-4fd2-993f-f75f0ccc2d0b | None | None          | power off   | available          | False       |
| a7658693-1698-4ac5-a5c8-f9af5bd72cfd | None | None          | power off   | available          | False       |
| d9ab970e-dcd1-4ee2-b80a-88243798c53e | None | None          | power off   | available          | False       |
| d6309297-c92b-4ab4-8ffb-70068f6d855b | None | None          | power off   | available          | False       |
| 89d64528-c605-439d-a157-9318527f7e8d | None | None          | power off   | available          | False       |
+--------------------------------------+------+---------------+-------------+--------------------+-------------+


[stack@undercloud74 ~]$ openstack baremetal configure boot && openstack baremetal introspection bulk start
Setting nodes for introspection to manageable...
Starting introspection of manageable nodes
Started Mistral Workflow. Execution ID: a2dbdab0-7a4c-4fd6-bd33-7a5457dddb8e
Waiting for introspection to finish...
Introspection for UUID a7658693-1698-4ac5-a5c8-f9af5bd72cfd finished successfully.
Introspection for UUID 89d64528-c605-439d-a157-9318527f7e8d finished successfully.
Introspection for UUID d6309297-c92b-4ab4-8ffb-70068f6d855b finished successfully.
Introspection for UUID c491ad8f-97bd-469b-8e65-afc348473d90 finished successfully.
Introspection for UUID 334c6d9b-3851-40ac-88f4-9d013aacc1cc finished successfully.
Introspection for UUID fbe20c2e-99e0-4fd2-993f-f75f0ccc2d0b finished successfully.
Introspection for UUID d9ab970e-dcd1-4ee2-b80a-88243798c53e finished successfully.
Introspection completed.
Setting manageable nodes to available...
Started Mistral Workflow. Execution ID: d4837adb-0dd8-4910-96ff-c1e2feb1557c

[stack@undercloud74 ~]$ openstack baremetal introspection bulk status
+--------------------------------------+----------+-------+
| Node UUID                            | Finished | Error |
+--------------------------------------+----------+-------+
| 334c6d9b-3851-40ac-88f4-9d013aacc1cc | True     | None  |
| c491ad8f-97bd-469b-8e65-afc348473d90 | True     | None  |
| fbe20c2e-99e0-4fd2-993f-f75f0ccc2d0b | True     | None  |
| a7658693-1698-4ac5-a5c8-f9af5bd72cfd | True     | None  |
| d9ab970e-dcd1-4ee2-b80a-88243798c53e | True     | None  |
| d6309297-c92b-4ab4-8ffb-70068f6d855b | True     | None  |
| 89d64528-c605-439d-a157-9318527f7e8d | True     | None  |
+--------------------------------------+----------+-------+
[stack@undercloud74 ~]$

Comment 5 errata-xmlrpc 2017-09-06 17:09:30 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2654