Bug 1731166

Summary: PXE boot failures using iDrac driver
Product: Red Hat OpenStack Reporter: Bob Fournier <bfournie>
Component: openstack-ironicAssignee: RHOS Maint <rhos-maint>
Status: CLOSED CURRENTRELEASE QA Contact: mlammon
Severity: high Docs Contact:
Priority: high    
Version: 13.0 (Queens)CC: amcleod, arkady_kanevsky, bfournie, christopher_dearborn, dhill, dtantsur, ealcaniz, jhardee, kholtz, kurt_hey, mburns, morazi, rhos-maint
Target Milestone: z9Keywords: TestOnly, Triaged, ZStream
Target Release: 13.0 (Queens)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-ironic-10.1.8-6.el7ost Doc Type: Bug Fix
Doc Text:
Previously, PXE boot with the idrac driver failed because it did not check the `pxe_enabled` flag. With this update, idrac uses the `pxe_enabled` flag properly and you can PXE boot using the idrac driver successfully.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2019-11-08 11:47:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1581398, 1588541    

Description Bob Fournier 2019-07-18 14:01:08 UTC 
PXE boot is failing when using the iDrac driver in OSP-13z7.  Chris has documented the sequence that introduced the failure:

1. We submitted this fix upstream to ironic/master, and it was merged in December of last year: https://review.opendev.org/#/c/617951/
   a. The fix made it into stein and is presently in master
2. We backported it to queens and rocky:
   https://review.opendev.org/#/c/624388/
   https://review.opendev.org/#/c/624272/
3. Red Hat cut z6 which included these patches
4. We removed the monkey patch from JetStream that patched in the fix since the fix was present in z6
5. The ironic community insisted that we remove the fix from queens and rocky because it relied on a newer version of python-dracclient that was beyond what was in driver-requirements.txt.  The fix worked fine for RHOSP, but that was because RHOSP was shipping the latest python-dracclient.  The community’s point was that the fix would not work for other OpenStack distributions if they elected to not ship the latest python-dracclient, but instead ship the version specified in driver-requirements.txt
6. We backed out the change from queens and rocky even though it was working fine with RHOSP:
     https://review.opendev.org/#/c/650134/
     https://review.opendev.org/#/c/650116/
7. Digambar started on a new version of the backport that did not depend on a newer version of python-dracclient.  At the time, we thought the backport would be quick, but it has taken quite a while to reach its current completed form: https://review.opendev.org/#/c/648360/
This patch has really been through the mill, so hopefully we’ll get some +2’s on it soon
Note that this patch is for stable/rocky.  The plan was (and is) that once this is approved, backport it to queens, which should be identical to the rocky backport
8. RH cut z7 after the fix was backed out of stable/queens and before the new backport landed
9. But the monkey patch was no longer present in JetStream
10. BOOM!
 
To fix the issue that we were seeing, I put the monkey patch back in JetStream for the time being, and all is well.
Comment 1 Bob Fournier 2019-07-18 14:08:22 UTC 
The fix in https://review.opendev.org/#/c/648360/ will need to be backported to queens.
Comment 3 Bob Fournier 2019-08-07 01:21:27 UTC 
The patch from Dell is upstream - https://review.opendev.org/#/c/648360/.  We are waiting for it to merge and will backport it to osp-13.
Comment 5 Chris Dearborn 2019-08-22 13:56:04 UTC 
The backport to stable/queens / OSP13 has merged:
https://review.opendev.org/#/c/677660/
Comment 14 Alex McLeod 2019-10-31 11:29:10 UTC 
If this bug requires doc text for errata release, please set the 'Doc Type' and provide draft text according to the template in the 'Doc Text' field. The documentation team will review, edit, and approve the text.

If this bug does not require doc text, please set the 'requires_doc_text' flag to -.
Comment 16 Bob Fournier 2019-11-04 17:54:03 UTC 
Because of lack of hardware was not able to test the fix but have verified that the package is available and verified that it contains the patch.

$ sudo rpm -qa | grep ironic
python-ironic-inspector-client-3.1.2-1.el7ost.noarch
openstack-ironic-common-10.1.9-1.el7ost.noarch
puppet-ironic-12.4.0-5.el7ost.noarch
openstack-ironic-api-10.1.9-1.el7ost.noarch
openstack-ironic-inspector-7.2.4-1.el7ost.noarch
python-ironic-lib-2.12.3-1.el7ost.noarch
openstack-ironic-conductor-10.1.9-1.el7ost.noarch
python2-ironic-neutron-agent-1.0.1-1.el7ost.noarch
python2-ironicclient-2.2.2-1.el7ost.noarch
openstack-ironic-staging-drivers-0.9.1-2.el7ost.noarch
Comment 18 Chris Dearborn 2019-11-05 22:16:27 UTC 
Sorry for the lateness of this reply, but I have verified that the code is present in the 10/31 puddle.  I'm working on verifying it in the lab, but apparently I had a switch die, so I won't be able to test it until tomorrow at the earliest.
Comment 19 Chris Dearborn 2019-11-06 18:11:08 UTC 
FWIW, I have verified that the fix is working as intended on Dell hardware.
Comment 20 Lon Hohberger 2019-11-08 11:47:03 UTC 
According to our records, this should be resolved by openstack-ironic-10.1.9-1.el7ost.  This build is available now.
Comment 21 Red Hat Bugzilla 2023-09-14 05:32:01 UTC 
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days