Bug 1248172 - inspection: clean failed with pxe_ilo
Summary: inspection: clean failed with pxe_ilo
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-proliantutils
Version: Director
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: y1
: 7.0 (Kilo)
Assignee: RHOS Maint
QA Contact: yeylon@redhat.com
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-07-29 19:12 UTC by Nicolas Hicher
Modified: 2023-02-22 23:02 UTC (History)
14 users (show)

Fixed In Version: python-proliantutils-2.1.0-4.el7ost
Doc Type: Bug Fix
Doc Text:
The iLO drivers lacked support for introspection cleaning tasks, which caused the ironic introspection to fail. A fix has been pushed to the proliantutils library to correct the iLO drivers. Alternatively, as a workaround, disable the cleaning task in /etc/ironic/ironic.conf: [ilo] ... clean_priority_reset_ilo=0 clean_priority_reset_bios_to_default=0 clean_priority_reset_secure_boot_keys_to_default=0 clean_priority_clear_secure_boot_keys=0 clean_priority_reset_ilo_credential=0
Clone Of:
Environment:
Last Closed: 2015-10-08 12:15:54 UTC
Target Upstream Version:
Embargoed:
nhicher: needinfo-


Attachments (Terms of Use)
the json file used for the inspection (3.06 KB, text/plain)
2015-07-29 19:12 UTC, Nicolas Hicher
no flags Details


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 208371 0 None None None Never
Red Hat Issue Tracker OSP-6358 0 None None None 2022-08-10 16:48:29 UTC
Red Hat Product Errata RHSA-2015:1862 0 normal SHIPPED_LIVE Moderate: Red Hat Enterprise Linux OpenStack Platform 7 director update 2015-10-08 16:05:50 UTC

Description Nicolas Hicher 2015-07-29 19:12:47 UTC
Created attachment 1057384 [details]
the json file used for the inspection

Description of problem:

WhenI run inspection with pxe_ilo on hd server with a recent firwmare (2.20), The inspection fail with this message for each node:

| maintenance_reason     | Node 97247c47-6ad1-472f-8535-aabd4b140407 failed step {'priority': 20,   |
|                        | 'interface': 'management', 'step': 'reset_secure_boot_keys_to_default'}: |
|                        | Clean step reset_secure_boot_keys failed on node                         |
|                        | 97247c47-6ad1-472f-8535-aabd4b140407 with error:                         |
|                        | ['iLO.0.10.UnsupportedOperationInSystemBIOS']                            |
| provision_state        | clean failed     



Version-Release number of selected component (if applicable):


How reproducible:

everytime

Steps to Reproduce:
openstack baremetal import --json director-hp-pod/hp-pod-env.json 
openstack baremetal configure boot
openstack baremetal introspection bulk start

Actual results:

[stack@instack-bm ~]$ ironic node-list
+--------------------------------------+------+---------------+-------------+-----------------+-------------+
| UUID                                 | Name | Instance UUID | Power State | Provision State | Maintenance |
+--------------------------------------+------+---------------+-------------+-----------------+-------------+
| 97247c47-6ad1-472f-8535-aabd4b140407 | None | None          | power off   | clean failed    | True        |
| 59838b58-8fa2-4557-b696-497858cbf6a1 | None | None          | power off   | clean failed    | True        |
| 0424de6c-6896-40a3-a64b-f5672c4d0b05 | None | None          | power off   | clean failed    | True        |
| 8cff6caf-e748-4023-a9ca-93061f517d8b | None | None          | power off   | clean failed    | True        |
| 9cf8da1a-27c7-49b3-8b1e-be6924947245 | None | None          | power off   | clean failed    | True        |
| 7085cdf8-bf64-4c4f-bb83-c77dce29352e | None | None          | power off   | clean failed    | True        |
| ae225ff8-c871-43b4-a97a-9e7936db8a64 | None | None          | power off   | clean failed    | True        |
| 0a85d41a-aa35-437c-b2fd-ca46f848761b | None | None          | power off   | clean failed    | True        |
| 513d78d6-723f-47ff-bb10-edd4a6f777e5 | None | None          | power off   | clean failed    | True        |
+--------------------------------------+------+---------------+-------------+-----------------+-------------+



Expected results:
+--------------------------------------+------+---------------+-------------+-----------------+-------------+
| UUID                                 | Name | Instance UUID | Power State | Provision State | Maintenance |
+--------------------------------------+------+---------------+-------------+-----------------+-------------+
| 4ae193eb-87e8-4f2a-891c-2c0e3e4311ad | None | None          | power off   | available       | False       |
| e8ef0f10-42b4-4b9c-aa25-1bc87e2ad0a2 | None | None          | power off   | available       | False       |
| 46207569-8020-47f0-9860-569e938d82bd | None | None          | power off   | available       | False       |
| a2e713e8-abe0-41f6-bfc3-7ef7db2ff4c8 | None | None          | power off   | available       | False       |
| 9b617428-ba5d-4b0a-9e40-a823d92ab5ce | None | None          | power off   | available       | False       |
| 3aa5739b-7a21-4eb8-bd8f-0b12ea81b24e | None | None          | power off   | available       | False       |
| 54859f43-8c50-4aa8-a5ba-8adb5d94d210 | None | None          | power off   | available       | False       |
| e065ab50-e6a3-4324-8ef6-c849fa2c51c7 | None | None          | power off   | available       | False       |
| 9633967b-dadc-4014-a169-f4cc5f6374ba | None | None          | power off   | available       | False       |
+--------------------------------------+------+---------------+-------------+-----------------+-------------+




Additional info:

Comment 3 Lucas Alvares Gomes 2015-07-31 13:30:32 UTC
Oh right... Seems that the hardware we have doesn't support that cleaning task. The iLO driver perhaps should handle that better and not fail like that.

As a workaround for now, you can disable those cleaning tasks. In the /etc/ironic/ironic.conf file, under the [ilo] section, please set the priority for those cleaning tasks to 0 (zero) so they won't be executed, e.g:

[ilo]

...

clean_priority_reset_ilo=0
clean_priority_reset_bios_to_default=0
clean_priority_reset_secure_boot_keys_to_default=0
clean_priority_clear_secure_boot_keys=0
clean_priority_reset_ilo_credential=0

Comment 4 Lucas Alvares Gomes 2015-08-04 08:37:17 UTC
So the problem is now fixed in the proliantutils library [1] according to the upstream iLO developers, it will be available on the next release (or we could backport it as well).

[1] https://review.openstack.org/#/c/208371/

Comment 6 John Trowbridge 2015-08-25 14:32:54 UTC
@Nicolas do you have an environment that you can reproduce this issue in? I have the upstream fix backported, but no way to test it.

Comment 7 JF Bibeau 2015-08-25 17:54:22 UTC
Hey there - I just ran into this issue today. Trying to run introspection on HP DL360p and it fails. ironic node-list shows "clean-failed":

+--------------------------------------+------+---------------+-------------+-----------------+-------------+
| UUID                                 | Name | Instance UUID | Power State | Provision State | Maintenance |
+--------------------------------------+------+---------------+-------------+-----------------+-------------+
| 489a247c-36cb-414e-99a6-7362888765e2 | None | None          | power on    | clean failed    | True        |
| 51ed5cf3-a121-4099-8aa7-a1333ce6905a | None | None          | power on    | clean failed    | True        |
+--------------------------------------+------+---------------+-------------+-----------------+-------------+

Anything I can do to help test a fix?

Comment 10 errata-xmlrpc 2015-10-08 12:15:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2015:1862


Note You need to log in before you can comment on or make changes to this bug.