Bug 2160159 - OSP 17 Undercloud is not able to introspect host with Dell iDRAC 6.10 using Redfish
Summary: OSP 17 Undercloud is not able to introspect host with Dell iDRAC 6.10 using R...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-sushy
Version: 17.0 (Wallaby)
Hardware: All
OS: All
high
high
Target Milestone: ga
: 17.1
Assignee: Julia Kreger
QA Contact: James E. LaBarre
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-01-11 15:34 UTC by coldford@redhat.com
Modified: 2024-04-15 16:16 UTC (History)
6 users (show)

Fixed In Version: python-sushy-3.7.6-1.20230118170856.9b3cf44.el9ost
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-08-16 01:13:10 UTC
Target Upstream Version:
Embargoed:
jkreger: needinfo-


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack Storyboard 2010537 0 None None None 2023-01-17 19:27:47 UTC
OpenStack gerrit 865507 0 None MERGED Fix setting boot related attributes 2023-01-18 15:17:12 UTC
RDO 46658 0 None None None 2023-01-18 15:17:12 UTC
Red Hat Issue Tracker OSP-21331 0 None None None 2023-01-11 15:35:50 UTC
Red Hat Product Errata RHEA-2023:4577 0 None None None 2023-08-16 01:13:31 UTC

Description coldford@redhat.com 2023-01-11 15:34:40 UTC
Description of problem:

Redfish introspection with OSP 16.2 UC works for old and new iDRAC versions while OSP 17.0 UC fails with iDRAC 6.10.00.

   OSP 17.0 UC - iDRAC 5.10.10 -  Success
   OSP 17.0 UC - iDRAC 6.10.00 - Fails

   OSP 16.2 UC - iDRAC 5.10.10 -  Success
   OSP 16.2 UC - iDRAC 6.10.00 - Success

Version-Release number of selected component (if applicable):
- openstack-ironic-common-17.0.5-0.20220705231707.dd2bfab.el9ost.noarch

How reproducible:
- Every time

Steps to Reproduce:
1. Try introspection of baremetal node using IDRAC 6.10.00

Actual results:

"2023-01-10 08:20:14,858 ERROR os_tripleo_baremetal_node_introspection: Introspection of node 90c8ea13-9482-42da-82af-762668748b82 failed on attempt 1: Node 90c8ea13-9482-42da-82af-762668748b82 reached failure state \"inspect failed\"; the last error is ironic-inspector inspection failed: Failed to set boot device to PXE: Failed to set boot device for node 90c8ea13-9482-42da-82af-762668748b82: Server Error for url: http://<IP_REMOVED>:6385/v1/nodes/90c8ea13-9482-42da-82af-762668748b82/management/boot_device, Redfish exception occurred. Error: Redfish set boot device failed for node 90c8ea13-9482-42da-82af-762668748b82. Error: HTTP PATCH https://<IP_REMOVED>/redfish/v1/Systems/System.Embedded.1/Settings returned code 400. Base.1.12.GeneralError: The property BootSourceOverrideEnabled is not in the list of valid properties for the resource. Extended information: [{'Message': 'The property BootSourceOverrideEnabled is not in the list of valid properties for the resource.', 'MessageArgs': ['BootSourceOverrideEnabled'], 'MessageArgs': 1, 'MessageId': 'Base.1.12.PropertyUnknown', 'RelatedProperties': ['#/Boot/BootSourceOverrideEnabled'], 'RelatedProperties': 1, 'Resolution': 'Remove the unknown property from the request body and resubmit the request if the operation failed.', 'Severity': 'Warning'}, {'Message': 'Unable to complete the operation because the property BootSourceOverrideEnabled is not in the list of valid properties for the resource.', 'MessageArgs': ['BootSourceOverrideEnabled'], 'MessageArgs': 1, 'MessageId': 'IDRAC.2.8.SYS429', 'RelatedProperties': ['#/Boot/BootSourceOverrideEnabled'], 'RelatedProperties': 1, 'Resolution': 'Remove the unknown property from the request body and retry the operation.', 'Severity': 'Warning'}, {'Message': 'The property BootSourceOverrideTarget is not in the list of valid properties for the resource.', 'MessageArgs': ['BootSourceOverrideTarget'], 'MessageArgs': 1, 'MessageId': 'Base.1.12.PropertyUnknown', 'RelatedProperties': ['#/Boot/BootSourceOverrideTarget'], 'RelatedProperties': 1, 'Resolution': 'Remove the unknown property from the request body and resubmit the request if the operation failed.', 'Severity': 'Warning'}, {'Message': 'Unable to complete the operation because the property BootSourceOverrideTarget is not in the list of valid properties for the resource.', 'MessageArgs': ['BootSourceOverrideTarget'], 'MessageArgs': 1, 'MessageId': 'IDRAC.2.8.SYS429', 'RelatedProperties': ['#/Boot/BootSourceOverrideTarget'], 'RelatedProperties': 1, 'Resolution': 'Remove the unknown property from the request body and retry the operation.', 'Severity': 'Warning'}]",



Expected results:

Sucessful introspection.

Additional info:

The following change may be responsible:

2022-07-05 OSP Prod Chain <dev-null> 1:17.0.5-0.20220705231707.dd2bfab
    - Update patches
    - Fix redfish-virtual-media for newer iDRACs

Note, the workaround setting force_persistent_boot_device=Never has no effect.
- from: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/17.0/html/bare_metal_provisioning/troubleshooting-the-bare-metal-provisioning-service#troubleshooting-idrac-issues 

$ openstack baremetal node set --driver-info force_persistent_boot_device=Never node
$ openstack baremetal node show -c driver_info --fit baremetal-node
+-------------+----------------------------------------------------------------------------------------------------------------------+
| Field       | Value                                                                                                                |
+-------------+----------------------------------------------------------------------------------------------------------------------+
| driver_info | {'deploy_kernel': 'file:///var/lib/ironic/httpboot/agent.kernel', 'rescue_kernel':                                   |
|             | 'file:///var/lib/ironic/httpboot/agent.kernel', 'deploy_ramdisk': 'file:///var/lib/ironic/httpbo>ot/agent.ramdisk',   |
|             | 'rescue_ramdisk': 'file:///var/lib/ironic/httpboot/agent.ramdisk', 'redfish_username': '<REMOVED>', 'redfish_password': |
|             | '******', 'redfish_address': '<IP_REMOVED>', 'redfish_system_id': '/redfish/v1/Systems/System.Embedded.1',          |
|             | 'redfish_verify_ca': False, 'force_persistent_boot_device': 'Never'}                                                 |
+-------------+----------------------------------------------------------------------------------------------------------------------+

Comment 18 Julia Kreger 2023-01-18 15:24:17 UTC
We've heard back from Dell Engineering and they pointed us to a very recent upstream patch which they believe resolves this issue.

I'm going to see if we can get this into 17.1.

Comment 28 James E. LaBarre 2023-07-18 16:00:39 UTC
Development/validation handled by partner (Dell)

Code already merged into OSP upstream

Comment 34 errata-xmlrpc 2023-08-16 01:13:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Release of components for Red Hat OpenStack Platform 17.1 (Wallaby)), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2023:4577

Comment 35 Jim Bagwell 2024-04-15 15:49:17 UTC
I wanted to notify this thread that I am seeing this issue in Wallaby with python3-sushy-4.3.3-2.el9.noarch installed on the UC. 

iDRAC Firmware Version	6.10.80.00


ERROR os_tripleo_baremetal_node_introspection: Introspection of node 170dbb83-9c2d-49c2-850b-acd53faa5384 failed on attempt 1: Node 170dbb83-9c2d-49c2-850b-acd53faa5384 reached failure state \"inspect failed\"; the last error is ironic-inspector inspection failed: Failed to set boot device to PXE: Failed to set boot device for node 170dbb83-9c2d-49c2-850b-acd53faa5384: Server Error for url: http://172.31.0.1:6385/v1/nodes/170dbb83-9c2d-49c2-850b-acd53faa5384/management/boot_device, Redfish exception occurred. Error: Redfish set boot device failed for node 170dbb83-9c2d-49c2-850b-acd53faa5384. Error: HTTP PATCH https://100.78.8.174/redfish/v1/Systems/System.Embedded.1/Settings returned code 400. Base.1.12.GeneralError: The property BootSourceOverrideEnabled is not in the list of valid properties for the resource. Extended information: [{'Message': 'The property BootSourceOverrideEnabled is not in the list of valid properties for the resource.', 'MessageArgs': ['BootSourceOverrideEnabled'], 'MessageArgs': 1, 'MessageId': 'Base.1.12.PropertyUnknown', 'RelatedProperties': ['#/Boot/BootSourceOverrideEnabled'], 'RelatedProperties': 1, 'Resolution': 'Remove the unknown property from the request body and resubmit the request if the operation failed.', 'Severity': 'Warning'}, {'Message': 'Unable to complete the operation because the property BootSourceOverrideEnabled is not in the list of valid properties for the resource.', 'MessageArgs': ['BootSourceOverrideEnabled'],

Comment 36 Julia Kreger 2024-04-15 15:57:15 UTC
Installed on the undercloud, or installed into the ironic-conductor container which is operating on the undercloud?

Comment 37 Jim Bagwell 2024-04-15 16:16:50 UTC
Hi Julia, thanks for the tip ; 

on UC : python3-sushy-4.3.3-2.el9.noarch 
inside ironic_conductor container operating on the UC: python3-sushy-3.7.6-0.20221111103907.9b3cf44.el9.noarch

Which is obviously not the version its corrected in. Thanks, will try updating the rpm inside the container to the version where its at least fixed.


Note You need to log in before you can comment on or make changes to this bug.