Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1852617

Summary: HPE workers UEFI boot is changed to Legacy BIOS during deployment
Product: OpenShift Container Platform Reporter: vkhanna
Component: Bare Metal Hardware ProvisioningAssignee: Steven Hardy <shardy>
Bare Metal Hardware Provisioning sub component: baremetal-operator QA Contact: Amit Ugol <augol>
Status: CLOSED DUPLICATE Docs Contact:
Severity: medium    
Priority: unspecified CC: iheim, shardy, sreichar, vkhanna
Version: 4.4Keywords: Triaged
Target Milestone: ---   
Target Release: 4.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-07-27 16:59:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
master node console snapshot
none
worker node console snapshot none

Description vkhanna 2020-06-30 21:29:51 UTC 
Description of problem:
HPE workers UEFI boot is changed to Legacy BIOS during deployment.

HPE ProLiant DL360 Gen10: Identical servers; Same firmware and iLO versions; All servers are initially set to UEFI boot enabled.
- Master nodes boot mode remains set to UEFI during cluster deployment.
- Worker nodes, however, boot mode gets changed to Legacy BIOS during deployment.

Is this expected behavior?


How reproducible:
Reproducible every time

Steps to Reproduce:
1. Set servers boot mode to UEFI
2. Deploy cluster

Actual results:
Worker nodes boot mode changes to Legacy BIOS

Expected results:
Worker nodes boot mode remains set to UEFI

Additional info:
Attaching snapshots for both master and worker node consoles.
Comment 1 vkhanna 2020-06-30 21:31:12 UTC 
Created attachment 1699389 [details]
master node console snapshot
Comment 2 vkhanna 2020-06-30 21:32:01 UTC 
Created attachment 1699391 [details]
worker node console snapshot
Comment 3 Steven Hardy 2020-07-01 16:22:37 UTC 
Hi thanks for the report - please can you provide the following information:

1. The exact 4.4 version deployed (e.g from oc version or the pullspec used for the deployment of the controlplane)

2. A sanitised copy of the install-config.yaml - in particular the hosts section so we can see what BMC interface you are using
Comment 4 vkhanna 2020-07-01 21:44:08 UTC 
Hi Steven

This cluster is being deployed using ansible playbooks - https://github.com/openshift-kni/baremetal-deploy/tree/master/ansible-ipi-install , so the install config is autogenerated. 

Please find the requested details below. I do see hardwareProfile being different for masters and workers, could that be an issue here?

$ oc get clusterversion
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.4.5     True        False         6h23m   Cluster version is 4.4.5

$ oc version
Client Version: 4.4.5
Server Version: 4.4.5
Kubernetes Version: v1.17.1


install-config.yaml hosts section, it is using the host's iLO interface.

platform:
  baremetal:
    hosts:
      - name: master-0
        role: master
        bmc:
          address: ****1:623
          username: ****
          password: ****
        bootMACAddress: ****:69:90
        hardwareProfile: default
      - name: master-1
        role: master
        bmc:
          address: ****2:623
          username: ****
          password: ****
        bootMACAddress: ****:7f:50
        hardwareProfile: default
      - name: master-2
        role: master
        bmc:
          address: ****3:623
          username: ****
          password: ****
        bootMACAddress: ****:a8:f0
        hardwareProfile: default
      - name: worker-1
        role: worker
        bmc:
          address: ****5:623
          username: ****
          password: ****
        bootMACAddress: ****:7d:40
        hardwareProfile: unknown
      - name: worker-2
        role: worker
        bmc:
          address: ****6:623
          username: ****
          password: ****
        bootMACAddress: ****:97:70
        hardwareProfile: unknown
Comment 5 Steve Reichard 2020-07-24 15:23:41 UTC 
I can reproduce this by doing inspections or provisions of workers after deploy the 4.4.5 master only cluster.  Here is teh data from a worker inspection:

[kni@worker-3 ansible-ipi-install]$ oc get node
NAME                                         STATUS   ROLES           AGE    VERSION
master-0.clus0.t5g.lab.eng.rdu2.redhat.com   Ready    master,worker   12h    v1.17.1
master-1.clus0.t5g.lab.eng.rdu2.redhat.com   Ready    master,worker   12h    v1.17.1
master-2.clus0.t5g.lab.eng.rdu2.redhat.com   Ready    master,worker   12h    v1.17.1
worker-0.clus0.t5g.lab.eng.rdu2.redhat.com   Ready    worker          119m   v1.17.1
[kni@worker-3 ansible-ipi-install]$ oc get bmh
NAME       STATUS   PROVISIONING STATUS      CONSUMER               BMC                    HARDWARE PROFILE   ONLINE   ERROR
master-0   OK       externally provisioned   clus0-master-0         ipmi://10.1.25.1:623                      true     
master-1   OK       externally provisioned   clus0-master-1         ipmi://10.1.25.2:623                      true     
master-2   OK       externally provisioned   clus0-master-2         ipmi://10.1.25.3:623                      true     
worker-0   OK       provisioned              clus0-worker-0-k2kxk   ipmi://10.1.25.4:623   unknown            true     
[kni@worker-3 ansible-ipi-install]$ 

[sreichar@spr-vnc ~]$ ilorest bootorder  --url 10.1.25.5 -u root -p 100Mgmt-
iLOrest : RESTful Interface Tool version 3.0.1
Copyright (c) 2014, 2019 Hewlett Packard Enterprise Development LP
------------------------------------------------------------------------------------------------------------------------------------
Discovering data...Done

Current Persistent Boot Order:
1. HD.EmbRAID.1.3 (Red Hat Enterprise Linux)
2. Generic.USB.1.1 (Generic USB Boot)
3. HD.EmbRAID.1.2 (Embedded RAID 1 : HPE Smart Array P816i-a SR Gen10 - 447.1 GiB, RAID1 Logical Drive 1(Target:0, Lun:0))
4. HD.EmbRAID.7.1 (Embedded RAID 1 : HPE Smart Array P816i-a SR Gen10 - 1.7 TiB, RAID0 Logical Drive 2(Target:0, Lun:1))
5. HD.EmbRAID.8.1 (Embedded RAID 1 : HPE Smart Array P816i-a SR Gen10 - 1.7 TiB, RAID0 Logical Drive 3(Target:0, Lun:2))
6. NIC.FlexLOM.1.1.IPv4 (Embedded FlexibleLOM 1 Port 1 : HPE Ethernet 10/25Gb 2-Port 631FLR-SFP28 Adapter - NIC (PXE IPv4))
7. NIC.FlexLOM.1.1.IPv6 (Embedded FlexibleLOM 1 Port 1 : HPE Ethernet 10/25Gb 2-Port 631FLR-SFP28 Adapter - NIC (PXE IPv6))

Continuous and one time boot options:
1. None
2. Cd
3. Hdd
4. Usb
5. SDCard
6. Utilities
7. Diags
8. BiosSetup
9. Pxe
10. UefiShell
11. UefiHttp
12. UefiTarget

Continuous and one time boot uefi options:
1. HD.EmbRAID.1.3 (Red Hat Enterprise Linux)
2. Generic.USB.1.1 (Generic USB Boot)
3. HD.EmbRAID.1.2 (Embedded RAID 1 : HPE Smart Array P816i-a SR Gen10 - 447.1 GiB, RAID1 Logical Drive 1(Target:0, Lun:0))
4. HD.EmbRAID.7.1 (Embedded RAID 1 : HPE Smart Array P816i-a SR Gen10 - 1.7 TiB, RAID0 Logical Drive 2(Target:0, Lun:1))
5. HD.EmbRAID.8.1 (Embedded RAID 1 : HPE Smart Array P816i-a SR Gen10 - 1.7 TiB, RAID0 Logical Drive 3(Target:0, Lun:2))

[sreichar@spr-vnc ~]$ 

[kni@worker-3 ansible-ipi-install]$ cat worker-1.yaml 
---
apiVersion: v1
kind: Secret
metadata:
  name: worker-1-bmc-secret
type: Opaque
data:
  username: cm9vdA==
  password: XXXXX
---
apiVersion: metal3.io/v1alpha1
kind: BareMetalHost
metadata:
  name: worker-1
  namespace: openshift-machine-api
spec:
  bmc:
    address: ipmi://10.1.25.5:623
    credentialsName: worker-1-bmc-secret
  bootMACAddress: f4:03:43:d1:65:d0
  hardwareProfile: unknown
  online: true
[kni@worker-3 ansible-ipi-install]$ 
[kni@worker-3 ansible-ipi-install]$ oc apply -f  worker-1.yaml 
secret/worker-1-bmc-secret created
Warning: oc apply should be used on resource created by either oc create --save-config or oc apply
baremetalhost.metal3.io/worker-1 configured
[kni@worker-3 ansible-ipi-install]$ oc get bmh
NAME       STATUS   PROVISIONING STATUS      CONSUMER               BMC                    HARDWARE PROFILE   ONLINE   ERROR
master-0   OK       externally provisioned   clus0-master-0         ipmi://10.1.25.1:623                      true     
master-1   OK       externally provisioned   clus0-master-1         ipmi://10.1.25.2:623                      true     
master-2   OK       externally provisioned   clus0-master-2         ipmi://10.1.25.3:623                      true     
worker-0   OK       provisioned              clus0-worker-0-k2kxk   ipmi://10.1.25.4:623   unknown            true     
worker-1   OK       inspecting                                      ipmi://10.1.25.5:623                      true     
[kni@worker-3 ansible-ipi-install]$ 

[sreichar@spr-vnc ~]$ ilorest bootorder  --url 10.1.25.5 -u root -p XXXX
iLOrest : RESTful Interface Tool version 3.0.1
Copyright (c) 2014, 2019 Hewlett Packard Enterprise Development LP
------------------------------------------------------------------------------------------------------------------------------------
Discovering data...Done

Current Persistent Boot Order:
1. HD.EmbRAID.1.3 (Red Hat Enterprise Linux)
2. Generic.USB.1.1 (Generic USB Boot)
3. HD.EmbRAID.1.2 (Embedded RAID 1 : HPE Smart Array P816i-a SR Gen10 - 447.1 GiB, RAID1 Logical Drive 1(Target:0, Lun:0))
4. HD.EmbRAID.7.1 (Embedded RAID 1 : HPE Smart Array P816i-a SR Gen10 - 1.7 TiB, RAID0 Logical Drive 2(Target:0, Lun:1))
5. HD.EmbRAID.8.1 (Embedded RAID 1 : HPE Smart Array P816i-a SR Gen10 - 1.7 TiB, RAID0 Logical Drive 3(Target:0, Lun:2))
6. NIC.FlexLOM.1.1.IPv4 (Embedded FlexibleLOM 1 Port 1 : HPE Ethernet 10/25Gb 2-Port 631FLR-SFP28 Adapter - NIC (PXE IPv4))
7. NIC.FlexLOM.1.1.IPv6 (Embedded FlexibleLOM 1 Port 1 : HPE Ethernet 10/25Gb 2-Port 631FLR-SFP28 Adapter - NIC (PXE IPv6))

Continuous and one time boot options:
1. None
2. Cd
3. Hdd
4. Usb
5. Utilities
6. Diags
7. BiosSetup
8. Pxe
9. UefiShell

[sreichar@spr-vnc ~]$ 


Notice the continuous and one time uefi option are no longer listed  - the node is booting BIOS and the wrong nic be default
Comment 7 Steven Hardy 2020-07-27 16:59:06 UTC 
I think this is a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1801461 so closing, feel free to reopen if you think that is incorrect, thanks!

*** This bug has been marked as a duplicate of bug 1801461 ***