Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1248172 - inspection: clean failed with pxe_ilo [NEEDINFO]
inspection: clean failed with pxe_ilo
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-proliantutils (Show other bugs)
Director
Unspecified Unspecified
high Severity unspecified
: y1
: 7.0 (Kilo)
Assigned To: RHOS Maint
yeylon@redhat.com
: OtherQA, Triaged, ZStream
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-07-29 15:12 EDT by Nicolas Hicher
Modified: 2016-04-18 02:55 EDT (History)
15 users (show)

See Also:
Fixed In Version: python-proliantutils-2.1.0-4.el7ost
Doc Type: Bug Fix
Doc Text:
The iLO drivers lacked support for introspection cleaning tasks, which caused the ironic introspection to fail. A fix has been pushed to the proliantutils library to correct the iLO drivers. Alternatively, as a workaround, disable the cleaning task in /etc/ironic/ironic.conf: [ilo] ... clean_priority_reset_ilo=0 clean_priority_reset_bios_to_default=0 clean_priority_reset_secure_boot_keys_to_default=0 clean_priority_clear_secure_boot_keys=0 clean_priority_reset_ilo_credential=0
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-10-08 08:15:54 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
jtrowbri: needinfo? (nhicher)


Attachments (Terms of Use)
the json file used for the inspection (3.06 KB, text/plain)
2015-07-29 15:12 EDT, Nicolas Hicher
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
OpenStack gerrit 208371 None None None Never
Red Hat Product Errata RHSA-2015:1862 normal SHIPPED_LIVE Moderate: Red Hat Enterprise Linux OpenStack Platform 7 director update 2015-10-08 12:05:50 EDT

  None (edit)
Description Nicolas Hicher 2015-07-29 15:12:47 EDT
Created attachment 1057384 [details]
the json file used for the inspection

Description of problem:

WhenI run inspection with pxe_ilo on hd server with a recent firwmare (2.20), The inspection fail with this message for each node:

| maintenance_reason     | Node 97247c47-6ad1-472f-8535-aabd4b140407 failed step {'priority': 20,   |
|                        | 'interface': 'management', 'step': 'reset_secure_boot_keys_to_default'}: |
|                        | Clean step reset_secure_boot_keys failed on node                         |
|                        | 97247c47-6ad1-472f-8535-aabd4b140407 with error:                         |
|                        | ['iLO.0.10.UnsupportedOperationInSystemBIOS']                            |
| provision_state        | clean failed     



Version-Release number of selected component (if applicable):


How reproducible:

everytime

Steps to Reproduce:
openstack baremetal import --json director-hp-pod/hp-pod-env.json 
openstack baremetal configure boot
openstack baremetal introspection bulk start

Actual results:

[stack@instack-bm ~]$ ironic node-list
+--------------------------------------+------+---------------+-------------+-----------------+-------------+
| UUID                                 | Name | Instance UUID | Power State | Provision State | Maintenance |
+--------------------------------------+------+---------------+-------------+-----------------+-------------+
| 97247c47-6ad1-472f-8535-aabd4b140407 | None | None          | power off   | clean failed    | True        |
| 59838b58-8fa2-4557-b696-497858cbf6a1 | None | None          | power off   | clean failed    | True        |
| 0424de6c-6896-40a3-a64b-f5672c4d0b05 | None | None          | power off   | clean failed    | True        |
| 8cff6caf-e748-4023-a9ca-93061f517d8b | None | None          | power off   | clean failed    | True        |
| 9cf8da1a-27c7-49b3-8b1e-be6924947245 | None | None          | power off   | clean failed    | True        |
| 7085cdf8-bf64-4c4f-bb83-c77dce29352e | None | None          | power off   | clean failed    | True        |
| ae225ff8-c871-43b4-a97a-9e7936db8a64 | None | None          | power off   | clean failed    | True        |
| 0a85d41a-aa35-437c-b2fd-ca46f848761b | None | None          | power off   | clean failed    | True        |
| 513d78d6-723f-47ff-bb10-edd4a6f777e5 | None | None          | power off   | clean failed    | True        |
+--------------------------------------+------+---------------+-------------+-----------------+-------------+



Expected results:
+--------------------------------------+------+---------------+-------------+-----------------+-------------+
| UUID                                 | Name | Instance UUID | Power State | Provision State | Maintenance |
+--------------------------------------+------+---------------+-------------+-----------------+-------------+
| 4ae193eb-87e8-4f2a-891c-2c0e3e4311ad | None | None          | power off   | available       | False       |
| e8ef0f10-42b4-4b9c-aa25-1bc87e2ad0a2 | None | None          | power off   | available       | False       |
| 46207569-8020-47f0-9860-569e938d82bd | None | None          | power off   | available       | False       |
| a2e713e8-abe0-41f6-bfc3-7ef7db2ff4c8 | None | None          | power off   | available       | False       |
| 9b617428-ba5d-4b0a-9e40-a823d92ab5ce | None | None          | power off   | available       | False       |
| 3aa5739b-7a21-4eb8-bd8f-0b12ea81b24e | None | None          | power off   | available       | False       |
| 54859f43-8c50-4aa8-a5ba-8adb5d94d210 | None | None          | power off   | available       | False       |
| e065ab50-e6a3-4324-8ef6-c849fa2c51c7 | None | None          | power off   | available       | False       |
| 9633967b-dadc-4014-a169-f4cc5f6374ba | None | None          | power off   | available       | False       |
+--------------------------------------+------+---------------+-------------+-----------------+-------------+




Additional info:
Comment 3 Lucas Alvares Gomes 2015-07-31 09:30:32 EDT
Oh right... Seems that the hardware we have doesn't support that cleaning task. The iLO driver perhaps should handle that better and not fail like that.

As a workaround for now, you can disable those cleaning tasks. In the /etc/ironic/ironic.conf file, under the [ilo] section, please set the priority for those cleaning tasks to 0 (zero) so they won't be executed, e.g:

[ilo]

...

clean_priority_reset_ilo=0
clean_priority_reset_bios_to_default=0
clean_priority_reset_secure_boot_keys_to_default=0
clean_priority_clear_secure_boot_keys=0
clean_priority_reset_ilo_credential=0
Comment 4 Lucas Alvares Gomes 2015-08-04 04:37:17 EDT
So the problem is now fixed in the proliantutils library [1] according to the upstream iLO developers, it will be available on the next release (or we could backport it as well).

[1] https://review.openstack.org/#/c/208371/
Comment 6 John Trowbridge 2015-08-25 10:32:54 EDT
@Nicolas do you have an environment that you can reproduce this issue in? I have the upstream fix backported, but no way to test it.
Comment 7 JF Bibeau 2015-08-25 13:54:22 EDT
Hey there - I just ran into this issue today. Trying to run introspection on HP DL360p and it fails. ironic node-list shows "clean-failed":

+--------------------------------------+------+---------------+-------------+-----------------+-------------+
| UUID                                 | Name | Instance UUID | Power State | Provision State | Maintenance |
+--------------------------------------+------+---------------+-------------+-----------------+-------------+
| 489a247c-36cb-414e-99a6-7362888765e2 | None | None          | power on    | clean failed    | True        |
| 51ed5cf3-a121-4099-8aa7-a1333ce6905a | None | None          | power on    | clean failed    | True        |
+--------------------------------------+------+---------------+-------------+-----------------+-------------+

Anything I can do to help test a fix?
Comment 10 errata-xmlrpc 2015-10-08 08:15:54 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2015:1862

Note You need to log in before you can comment on or make changes to this bug.