Bug 1929246 - Missing info for Operational Status, Provisioning status, BMC, Hostname, ID for BMH for OCP deployed with assisted installer
Summary: Missing info for Operational Status, Provisioning status, BMC, Hostname, ID f...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: assisted-installer
Version: 4.7
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 4.7.0
Assignee: Angus Salkeld
QA Contact: Amit Ugol
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-02-16 14:42 UTC by Sasha Smolyak
Modified: 2021-04-05 13:55 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-04-05 13:55:11 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2021:1005 0 None None None 2021-04-05 13:55:33 UTC

Description Sasha Smolyak 2021-02-16 14:42:18 UTC
Description of problem:
Missing info for Operational Status, Provisioning status, BMC, Hostname, ID for BMH for OCP deployed with assisted installer

Version-Release number of selected component (if applicable):
4.7.0-fc.0

How reproducible:
100%

Steps to Reproduce:
1. Deploy OCP with assisted installer
2. oc get bmh --all-namespaces
3. oc get bmh openshift-master-0 -o yaml

Actual results:

1. NAMESPACE               NAME                 STATUS   PROVISIONING STATUS   CONSUMER                                    BMC   HARDWARE PROFILE   ONLINE   ERROR
openshift-machine-api   openshift-master-0                                  ocp-edge-cluster-cdv-mnhdp-master-0                                  true
openshift-machine-api   openshift-master-1                                  ocp-edge-cluster-cdv-mnhdp-master-1                                  true
openshift-machine-api   openshift-master-2                                  ocp-edge-cluster-cdv-mnhdp-master-2                                  true
openshift-machine-api   openshift-worker-0                                  ocp-edge-cluster-cdv-mnhdp-worker-0-fhxnn                            true
openshift-machine-api   openshift-worker-1                                  ocp-edge-cluster-cdv-mnhdp-worker-0-hd6p8                            true

2. 
In the yaml output there is no Hostname and no ID in the bmh info:
Provisioning:
    ID:
    Image:
      Checksum:
      URL:
    State:

Expected results:
1. 
NAMESPACE               NAME                   STATUS   PROVISIONING STATUS      CONSUMER                                  BMC                                                                                    HARDWARE PROFILE   ONLINE   ERROR
openshift-machine-api   openshift-master-0-0   OK       externally provisioned   ocp-edge-cluster-0-z6g98-master-0         redfish://192.168.123.1:8000/redfish/v1/Systems/2c05f44b-9c3f-4f0f-801b-6b800818ae08                      true
openshift-machine-api   openshift-master-0-1   OK       externally provisioned   ocp-edge-cluster-0-z6g98-master-1         redfish://192.168.123.1:8000/redfish/v1/Systems/d9ad3d40-57f5-4c8c-9464-8cf723c490c9                      true
openshift-machine-api   openshift-master-0-2   OK       externally provisioned   ocp-edge-cluster-0-z6g98-master-2         redfish://192.168.123.1:8000/redfish/v1/Systems/3e89168f-ba52-483d-814f-dc6b77face6c                      true
openshift-machine-api   openshift-worker-0-0   OK       provisioned              ocp-edge-cluster-0-z6g98-worker-0-ggb5b   redfish://192.168.123.1:8000/redfish/v1/Systems/590a9fc8-ca07-4cde-8efc-ad5aefb49b76   unknown            true
openshift-machine-api   openshift-worker-0-1   OK       provisioned              ocp-edge-cluster-0-z6g98-worker-0-slddl   redfish://192.168.123.1:8000/redfish/v1/Systems/02a95fbe-0999-4a08-940c-2eff5612446d   unknown            true

2. Columns for status, provisioning status, BMC not empty. In the yaml output Hostname and ID are not empty:
Provisioning:
    ID:  a834bccd-e512-4f2a-a9a6-4295512c2e61
    Image:
      Checksum:
      URL:
    State:       externally provisioned



Additional info:

Comment 1 Ronnie Lazar 2021-02-16 15:00:09 UTC
asalkeld are these fields relevant?

Comment 2 Omri Hochman 2021-02-16 17:42:55 UTC
I guess if we want to use ironic for day2 operation we need to provide this info.  I'm not sure if there's a practical way to deliver this info to ironic
 *post-deployment*  (maybe, running manually 'ironic introspection' and manually create a .json ?) 


AFAIU, this will impact: 
backup/restore 
scale cluster using ironic 
soft/hard reboot
etc.. 


Then, say we find a way to close this gap by running manual steps post-AI deployment, how do we plan to document it and provide this information  
to cloud.redhat.com users?

Comment 3 Angus Salkeld 2021-02-17 06:54:44 UTC
@ssmolyak can you please retest with a 4.7 nightly build. I did get this merged not so long ago, so it should work fine.
Note that the version you used was too old though.

Comment 4 Angus Salkeld 2021-02-17 06:58:07 UTC
@ssmolyak one more thing. the assisted installer will not setup the BMC details, if you really want those, you will need to do that yourself.
I have not tested assisted installer + bmc details. You would need to carefully see what the baremetal operator does.

Comment 5 Rom Freiman 2021-02-17 07:01:30 UTC
Comparing to 4.6.8 where metal3 completely removed:

KUBECONFIG=/home/rom/Downloads/940459.txt oc get clusterversion
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.6.8     True        False         11h

UBECONFIG=/home/rom/Downloads/940459.txt oc get bmh --all-namespaces
NAMESPACE               NAME                 STATUS   PROVISIONING STATUS   CONSUMER                BMC   HARDWARE PROFILE   ONLINE   ERROR
openshift-machine-api   openshift-master-0                                  clus3a-z6b24-master-0                            true     
openshift-machine-api   openshift-master-1                                  clus3a-z6b24-master-1                            true     
openshift-machine-api   openshift-master-2                                  clus3a-z6b24-master-2                            true



KUBECONFIG=/home/rom/Downloads/940459.txt oc get bmh openshift-master-0 -o yaml -n openshift-machine-api
apiVersion: metal3.io/v1alpha1
kind: BareMetalHost
metadata:
  annotations:
    baremetalhost.metal3.io/paused: ""
  creationTimestamp: "2021-02-16T18:23:57Z"
  finalizers:
  - machine.machine.openshift.io
  generation: 2
  managedFields:
  - apiVersion: metal3.io/v1alpha1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:baremetalhost.metal3.io/paused: {}
      f:spec:
        .: {}
        f:bmc:
          .: {}
          f:address: {}
          f:credentialsName: {}
        f:bootMACAddress: {}
        f:bootMode: {}
        f:consumerRef:
          .: {}
          f:apiVersion: {}
          f:kind: {}
          f:name: {}
          f:namespace: {}
        f:externallyProvisioned: {}
        f:hardwareProfile: {}
        f:online: {}
      f:status:
        .: {}
        f:errorMessage: {}
        f:goodCredentials: {}
        f:hardwareProfile: {}
        f:operationHistory:
          .: {}
          f:deprovision:
            .: {}
            f:end: {}
            f:start: {}
          f:inspect:
            .: {}
            f:end: {}
            f:start: {}
          f:provision:
            .: {}
            f:end: {}
            f:start: {}
          f:register:
            .: {}
            f:end: {}
            f:start: {}
        f:operationalStatus: {}
        f:provisioning:
          .: {}
          f:ID: {}
          f:image:
            .: {}
            f:checksum: {}
            f:url: {}
          f:state: {}
        f:triedCredentials: {}
    manager: cluster-bootstrap
    operation: Update
    time: "2021-02-16T18:23:57Z"
  - apiVersion: metal3.io/v1alpha1
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        f:hardware:
          .: {}
          f:cpu:
            .: {}
            f:arch: {}
            f:clockMegahertz: {}
            f:count: {}
            f:flags: {}
            f:model: {}
          f:firmware:
            .: {}
            f:bios:
              .: {}
              f:date: {}
              f:vendor: {}
              f:version: {}
          f:hostname: {}
          f:nics: {}
          f:ramMebibytes: {}
          f:storage: {}
          f:systemVendor:
            .: {}
            f:manufacturer: {}
            f:productName: {}
            f:serialNumber: {}
        f:lastUpdated: {}
        f:poweredOn: {}
    manager: assisted-installer-controller
    operation: Update
    time: "2021-02-16T18:52:35Z"
  - apiVersion: metal3.io/v1alpha1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:finalizers:
          .: {}
          v:"machine.machine.openshift.io": {}
      f:spec:
        f:image:
          .: {}
          f:checksum: {}
          f:url: {}
        f:userData:
          .: {}
          f:name: {}
          f:namespace: {}
    manager: machine-controller-manager
    operation: Update
    time: "2021-02-16T18:52:44Z"
  name: openshift-master-0
  namespace: openshift-machine-api
  resourceVersion: "9257"
  selfLink: /apis/metal3.io/v1alpha1/namespaces/openshift-machine-api/baremetalhosts/openshift-master-0
  uid: e9b01346-73ce-416a-9454-942ca4f2ba24
spec:
  bmc:
    address: ""
    credentialsName: ""
  bootMACAddress: e4:43:4b:bd:90:2a
  bootMode: legacy
  consumerRef:
    apiVersion: machine.openshift.io/v1beta1
    kind: Machine
    name: clus3a-z6b24-master-0
    namespace: openshift-machine-api
  externallyProvisioned: true
  hardwareProfile: unknown
  image:
    checksum: http://172.22.0.3:6180/images/rhcos-46.82.202011260640-0-openstack.x86_64.qcow2/rhcos-46.82.202011260640-0-compressed.x86_64.qcow2.md5sum
    url: http://172.22.0.3:6180/images/rhcos-46.82.202011260640-0-openstack.x86_64.qcow2/rhcos-46.82.202011260640-0-compressed.x86_64.qcow2
  online: true
  userData:
    name: master-user-data
    namespace: openshift-machine-api
status:
  errorMessage: ""
  goodCredentials: {}
  hardware:
    cpu:
      arch: x86_64
      clockMegahertz: 3900
      count: 80
      flags:
      - fpu
      - vme
      - de
      - pse
      - tsc
      - msr
      - pae
      - mce
      - cx8
      - apic
      - sep
      - mtrr
      - pge
      - mca
      - cmov
      - pat
      - pse36
      - clflush
      - dts
      - acpi
      - mmx
      - fxsr
      - sse
      - sse2
      - ss
      - ht
      - tm
      - pbe
      - syscall
      - nx
      - pdpe1gb
      - rdtscp
      - lm
      - constant_tsc
      - art
      - arch_perfmon
      - pebs
      - bts
      - rep_good
      - nopl
      - xtopology
      - nonstop_tsc
      - cpuid
      - aperfmperf
      - pni
      - pclmulqdq
      - dtes64
      - monitor
      - ds_cpl
      - vmx
      - smx
      - est
      - tm2
      - ssse3
      - sdbg
      - fma
      - cx16
      - xtpr
      - pdcm
      - pcid
      - dca
      - sse4_1
      - sse4_2
      - x2apic
      - movbe
      - popcnt
      - tsc_deadline_timer
      - aes
      - xsave
      - avx
      - f16c
      - rdrand
      - lahf_lm
      - abm
      - 3dnowprefetch
      - cpuid_fault
      - epb
      - cat_l3
      - cdp_l3
      - invpcid_single
      - intel_ppin
      - ssbd
      - mba
      - ibrs
      - ibpb
      - stibp
      - ibrs_enhanced
      - tpr_shadow
      - vnmi
      - flexpriority
      - ept
      - vpid
      - fsgsbase
      - tsc_adjust
      - bmi1
      - hle
      - avx2
      - smep
      - bmi2
      - erms
      - invpcid
      - rtm
      - cqm
      - mpx
      - rdt_a
      - avx512f
      - avx512dq
      - rdseed
      - adx
      - smap
      - clflushopt
      - clwb
      - intel_pt
      - avx512cd
      - avx512bw
      - avx512vl
      - xsaveopt
      - xsavec
      - xgetbv1
      - xsaves
      - cqm_llc
      - cqm_occup_llc
      - cqm_mbm_total
      - cqm_mbm_local
      - dtherm
      - ida
      - arat
      - pln
      - pts
      - pku
      - ospke
      - avx512_vnni
      - md_clear
      - flush_l1d
      - arch_capabilities
      model: Intel(R) Xeon(R) Gold 6230 CPU @ 2.10GHz
    firmware:
      bios:
        date: ""
        vendor: ""
        version: ""
    hostname: ""
    nics:
    - ip: 10.19.32.198
      mac: e4:43:4b:bd:91:8c
      model: "0x158a"
      name: ens1f0
      pxe: false
      speedGbps: 24
      vlanId: 0
    - ip: ""
      mac: e4:43:4b:bd:91:8d
      model: "0x158a"
      name: ens1f1
      pxe: false
      speedGbps: 24
      vlanId: 0
    - ip: ""
      mac: e4:43:4b:bd:92:c0
      model: "0x158a"
      name: ens4f0
      pxe: false
      speedGbps: 24
      vlanId: 0
    - ip: ""
      mac: e4:43:4b:bd:92:c1
      model: "0x158a"
      name: ens4f1
      pxe: false
      speedGbps: 24
      vlanId: 0
    ramMebibytes: 196608
    storage:
    - model: Dell Express Flash PM1725b 1.6TB SFF
      name: nvme0n1
      rotational: false
      serialNumber: S5CUNAUN501142
      sizeBytes: 1600321314816
      wwn: eui.354355554e5011420025384100000004
    - model: Dell Express Flash PM1725b 1.6TB SFF
      name: nvme1n1
      rotational: false
      serialNumber: S5CUNAUN501141
      sizeBytes: 1600321314816
      wwn: eui.354355554e5011410025384100000004
    - hctl: "0:2:0:0"
      model: PERC_H730P_MX
      name: sda
      rotational: true
      serialNumber: 62cea7f047dfcb0027a5bec410e572e7
      sizeBytes: 479559942144
      vendor: DELL
      wwn: 0x62cea7f047dfcb0027a5bec410e572e7
    - hctl: "0:2:1:0"
      model: PERC_H730P_MX
      name: sdb
      rotational: true
      serialNumber: 62cea7f047dfcb0027a5bec410e9c789
      sizeBytes: 479559942144
      vendor: DELL
      wwn: 0x62cea7f047dfcb0027a5bec410e9c789
    - hctl: "11:0:0:1"
      model: Virtual_Floppy
      name: sdc
      rotational: true
      serialNumber: "1028_123456"
      sizeBytes: 0
      vendor: Linux
    - hctl: "11:0:0:0"
      model: Virtual_CD/DVD
      name: sr0
      rotational: false
      serialNumber: "1028_123456"
      sizeBytes: 918552576
      vendor: Linux
    systemVendor:
      manufacturer: Dell Inc.
      productName: PowerEdge MX740c
      serialNumber: 9VZS243
  hardwareProfile: ""
  lastUpdated: "2021-02-16T18:52:35Z"
  operationHistory:
    deprovision:
      end: null
      start: null
    inspect:
      end: null
      start: null
    provision:
      end: null
      start: null
    register:
      end: null
      start: null
  operationalStatus: ""
  poweredOn: true
  provisioning:
    ID: ""
    image:
      checksum: ""
      url: ""
    state: ""
  triedCredentials: {}

Comment 6 Rom Freiman 2021-02-17 07:03:52 UTC
@shardy  stbenjam what is the expected result?

Comment 7 Angus Salkeld 2021-02-18 03:42:20 UTC
With a new 4.7 it looks like this

root@hardprov-fx2-24 (master):~/assisted-test-infra# oc get clusterversion
NAME      VERSION                        AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.7.0-0.ci-2021-02-17-020828   True        False         21s     Cluster version is 4.7.0-0.ci-2021-02-17-020828
root@hardprov-fx2-24 (master):~/assisted-test-infra# oc get bmh --all-namespaces
NAMESPACE               NAME                 STATUS       PROVISIONING STATUS   CONSUMER                                     BMC   HARDWARE PROFILE   ONLINE   ERROR
openshift-machine-api   openshift-master-0   discovered   unmanaged             coffee-assisted-insta-vhd48-master-0                                  true     
openshift-machine-api   openshift-master-1   discovered   unmanaged             coffee-assisted-insta-vhd48-master-1                                  true     
openshift-machine-api   openshift-master-2   discovered   unmanaged             coffee-assisted-insta-vhd48-master-2                                  true     
openshift-machine-api   openshift-worker-0   discovered   unmanaged             coffee-assisted-insta-vhd48-worker-0-4zbv4                            true     
openshift-machine-api   openshift-worker-1   discovered   unmanaged             coffee-assisted-insta-vhd48-worker-0-lmvvd                            true     
root@hardprov-fx2-24 (master):~/assisted-test-infra# oc get bmh openshift-master-0 -o yaml -n openshift-machine-api
apiVersion: metal3.io/v1alpha1
kind: BareMetalHost
metadata:
  creationTimestamp: "2021-02-18T03:11:14Z"
  finalizers:
  - baremetalhost.metal3.io
  generation: 2
  managedFields:
  - apiVersion: metal3.io/v1alpha1
    fieldsType: FieldsV1
    fieldsV1:
      f:spec:
        .: {}
        f:bmc:
          .: {}
          f:address: {}
          f:credentialsName: {}
        f:bootMACAddress: {}
        f:bootMode: {}
        f:consumerRef:
          .: {}
          f:apiVersion: {}
          f:kind: {}
          f:name: {}
          f:namespace: {}
        f:externallyProvisioned: {}
        f:hardwareProfile: {}
        f:online: {}
      f:status:
        .: {}
        f:errorCount: {}
        f:errorMessage: {}
        f:goodCredentials: {}
        f:hardwareProfile: {}
        f:operationHistory:
          .: {}
          f:deprovision:
            .: {}
            f:end: {}
            f:start: {}
          f:inspect:
            .: {}
            f:end: {}
            f:start: {}
          f:provision:
            .: {}
            f:end: {}
            f:start: {}
          f:register:
            .: {}
            f:end: {}
            f:start: {}
        f:provisioning:
          .: {}
          f:ID: {}
          f:image:
            .: {}
            f:checksum: {}
            f:url: {}
        f:triedCredentials: {}
    manager: cluster-bootstrap
    operation: Update
    time: "2021-02-18T03:11:14Z"
  - apiVersion: metal3.io/v1alpha1
    fieldsType: FieldsV1
    fieldsV1:
      f:spec:
        f:image:
          .: {}
          f:checksum: {}
          f:url: {}
        f:userData:
          .: {}
          f:name: {}
          f:namespace: {}
    manager: machine-controller-manager
    operation: Update
    time: "2021-02-18T03:18:01Z"
  - apiVersion: metal3.io/v1alpha1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:finalizers:
          .: {}
          v:"baremetalhost.metal3.io": {}
      f:status:
        f:hardware:
          .: {}
          f:cpu:
            .: {}
            f:arch: {}
            f:clockMegahertz: {}
            f:count: {}
            f:flags: {}
            f:model: {}
          f:firmware:
            .: {}
            f:bios:
              .: {}
              f:date: {}
              f:vendor: {}
              f:version: {}
          f:hostname: {}
          f:nics: {}
          f:ramMebibytes: {}
          f:storage: {}
          f:systemVendor:
            .: {}
            f:manufacturer: {}
            f:productName: {}
            f:serialNumber: {}
        f:lastUpdated: {}
        f:operationalStatus: {}
        f:poweredOn: {}
        f:provisioning:
          f:state: {}
    manager: baremetal-operator
    operation: Update
    time: "2021-02-18T03:22:49Z"
  name: openshift-master-0
  namespace: openshift-machine-api
  resourceVersion: "14024"
  selfLink: /apis/metal3.io/v1alpha1/namespaces/openshift-machine-api/baremetalhosts/openshift-master-0
  uid: a3926770-73e7-4518-9f88-f25c46815e3f
spec:
  bmc:
    address: ""
    credentialsName: ""
  bootMACAddress: 02:00:00:5b:eb:85
  bootMode: legacy
  consumerRef:
    apiVersion: machine.openshift.io/v1beta1
    kind: Machine
    name: coffee-assisted-insta-vhd48-master-0
    namespace: openshift-machine-api
  externallyProvisioned: true
  hardwareProfile: unknown
  image:
    checksum: http://192.168.126.102:6181/images/rhcos-47.83.202102090044-0-openstack.x86_64.qcow2/rhcos-47.83.202102090044-0-compressed.x86_64.qcow2.md5sum
    url: http://192.168.126.102:6181/images/rhcos-47.83.202102090044-0-openstack.x86_64.qcow2/rhcos-47.83.202102090044-0-compressed.x86_64.qcow2
  online: true
  userData:
    name: master-user-data
    namespace: openshift-machine-api
status:
  errorCount: 0
  errorMessage: ""
  goodCredentials: {}
  hardware:
    cpu:
      arch: x86_64
      clockMegahertz: 2099.996
      count: 4
      flags:
      - fpu
      - vme
      - de
      - pse
      - tsc
      - msr
      - pae
      - mce
      - cx8
      - apic
      - sep
      - mtrr
      - pge
      - mca
      - cmov
      - pat
      - pse36
      - clflush
      - mmx
      - fxsr
      - sse
      - sse2
      - ss
      - syscall
      - nx
      - pdpe1gb
      - rdtscp
      - lm
      - constant_tsc
      - arch_perfmon
      - rep_good
      - nopl
      - xtopology
      - cpuid
      - tsc_known_freq
      - pni
      - pclmulqdq
      - vmx
      - ssse3
      - fma
      - cx16
      - pcid
      - sse4_1
      - sse4_2
      - x2apic
      - movbe
      - popcnt
      - tsc_deadline_timer
      - aes
      - xsave
      - avx
      - f16c
      - rdrand
      - hypervisor
      - lahf_lm
      - abm
      - 3dnowprefetch
      - cpuid_fault
      - invpcid_single
      - pti
      - ssbd
      - ibrs
      - ibpb
      - stibp
      - tpr_shadow
      - vnmi
      - flexpriority
      - ept
      - vpid
      - ept_ad
      - fsgsbase
      - tsc_adjust
      - bmi1
      - hle
      - avx2
      - smep
      - bmi2
      - erms
      - invpcid
      - rtm
      - rdseed
      - adx
      - smap
      - xsaveopt
      - arat
      - umip
      - arch_capabilities
      model: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
    firmware:
      bios:
        date: ""
        vendor: ""
        version: ""
    hostname: ""
    nics:
    - ip: 192.168.126.11
      mac: 02:00:00:c7:4c:e9
      model: "0x0001"
      name: ens3
      pxe: false
      speedGbps: 0
      vlanId: 0
    - ip: 192.168.141.11
      mac: 02:00:00:cf:37:f2
      model: "0x0001"
      name: ens4
      pxe: false
      speedGbps: 0
      vlanId: 0
    ramMebibytes: 16984
    storage:
    - hctl: "0:0:0:0"
      model: QEMU_DVD-ROM
      name: sr0
      rotational: false
      serialNumber: QM00001
      sizeBytes: 950009856
      vendor: QEMU
    - name: vda
      rotational: true
      serialNumber: ""
      sizeBytes: 128849018880
      vendor: "0x1af4"
    systemVendor:
      manufacturer: Red Hat
      productName: KVM
      serialNumber: ""
  hardwareProfile: ""
  lastUpdated: "2021-02-18T03:22:50Z"
  operationHistory:
    deprovision:
      end: null
      start: null
    inspect:
      end: null
      start: null
    provision:
      end: null
      start: null
    register:
      end: null
      start: null
  operationalStatus: discovered
  poweredOn: true
  provisioning:
    ID: ""
    image:
      checksum: ""
      url: ""
    state: unmanaged
  triedCredentials: {}

Comment 8 Omri Hochman 2021-03-01 12:23:34 UTC
adding info from Marius Cornea that was able to scale-up cluster deployed by AI (virt-env0):
 
 
alright, so adding a worker node with redfish-virtualmedia worked
==================================================================

[root@seal55 ~]# oc -n openshift-machine-api get nodes
NAME         STATUS   ROLES    AGE    VERSION
master-0-0   Ready    master   156m   v1.20.0+ba45583
master-0-1   Ready    master   139m   v1.20.0+ba45583
master-0-2   Ready    master   156m   v1.20.0+ba45583
worker-0-0   Ready    worker   142m   v1.20.0+ba45583
worker-0-1   Ready    worker   143m   v1.20.0+ba45583
worker-0-2   Ready    worker   2m2s   v1.20.0+ba45583
==================================================================
[root@seal55 ~]# oc -n openshift-machine-api get bmh
NAME                 STATUS       PROVISIONING STATUS   CONSUMER                                BMC                                                                                                 HARDWARE PROFILE   ONLINE   ERROR
openshift-master-0   discovered   unmanaged             seal55-cluster-0-lhpwd-master-0                                                                                                                                true     
openshift-master-1   discovered   unmanaged             seal55-cluster-0-lhpwd-master-1                                                                                                                                true     
openshift-master-2   discovered   unmanaged             seal55-cluster-0-lhpwd-master-2                                                                                                                                true     
openshift-worker-0   discovered   unmanaged             seal55-cluster-0-lhpwd-worker-0-kgtss                                                                                                                          true     
openshift-worker-1   discovered   unmanaged             seal55-cluster-0-lhpwd-worker-0-kqs4s                                                                                                                          true     
openshift-worker-2   OK           provisioned           seal55-cluster-0-lhpwd-worker-0-65vpw   redfish-virtualmedia://192.168.123.1:8000/redfish/v1/Systems/50f8abd3-11d8-4462-bcf8-af3e91f568a7   unknown            true  

=================================================================
this is the required objects for creating it as a baremetal-operator host:


---
apiVersion: metal3.io/v1alpha1
kind: BareMetalHost
metadata:
  creationTimestamp: null
  name: openshift-worker-2
  namespace: openshift-machine-api
spec:
  bmc:
    address: "redfish-virtualmedia://192.168.123.1:8000/redfish/v1/Systems/50f8abd3-11d8-4462-bcf8-af3e91f568a7"
    credentialsName: "openshift-worker-2-bmc-secret"
    disableCertificateVerification: true
  bootMACAddress: 52:54:00:5d:3c:d4
  online: false
  userData:
    name: master-user-data
    namespace: openshift-machine-api
---
apiVersion: v1
data:
  password: cGFzc3dvcmQ=
  username: YWRtaW4=
kind: Secret
metadata:
  name: "openshift-worker-2-bmc-secret"
  namespace: openshift-machine-api
type: Opaque



=================================================================
on the libvirt side I had to create the VM XML with UEFI firmware it didn't work with BIOS,
I guess that's due to redfish-virtualmedia as it requires UEFI

leaving the XML here as well:

<domain type='kvm' id='19'>
  <name>worker-0-2</name>
  <uuid>50f8abd3-11d8-4462-bcf8-af3e91f568a7</uuid>
  <metadata xmlns:sushy="http://openstack.org/xmlns/libvirt/sushy">
    <sushy:bios>
      <sushy:attributes>
        <sushy:attribute name="BootMode" value="Uefi"/>
        <sushy:attribute name="EmbeddedSata" value="Raid"/>
        <sushy:attribute name="NicBoot1" value="NetworkBoot"/>
        <sushy:attribute name="ProcTurboMode" value="Enabled"/>
      </sushy:attributes>
    </sushy:bios>
  </metadata>
  <memory unit='KiB'>16777216</memory>
  <currentMemory unit='KiB'>16777216</currentMemory>
  <vcpu placement='static'>8</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-q35-rhel8.2.0'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/OVMF/OVMF_CODE.secboot.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/worker-0-2.fd</nvram>
    <bootmenu enable='no'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <cpu mode='host-passthrough' check='none'/>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='unsafe'/>
      <source file='/var/lib/libvirt/images/linchpin//worker-0-2.qcow2' index='1'/>
      <backingStore/>
      <target dev='sda' bus='scsi'/>
      <boot order='1'/>
      <alias name='scsi0-0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='scsi' index='0' model='virtio-scsi'>
      <alias name='scsi0'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='usb' index='0' model='piix3-uhci'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x01' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'>
      <alias name='pcie.0'/>
    </controller>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x8'/>
      <alias name='pci.1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x9'/>
      <alias name='pci.2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-to-pci-bridge'>
      <model name='pcie-pci-bridge'/>
      <alias name='pci.3'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0xa'/>
      <alias name='pci.4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0xb'/>
      <alias name='pci.5'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>
    </controller>
    <controller type='sata' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:5d:3c:d4'/>
      <source network='baremetal-0' portid='f20cf466-bb2c-4d3a-a96b-572a61d6b7ab' bridge='baremetal-0'/>
      <target dev='vnet10'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/10'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/10'>
      <source path='/dev/pts/10'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <input type='mouse' bus='ps2'>
      <alias name='input0'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input1'/>
    </input>
    <graphics type='vnc' port='5905' autoport='yes' listen='127.0.0.1' keymap='en-us'>
      <listen type='address' address='127.0.0.1'/>
    </graphics>
    <video>
      <model type='cirrus' vram='16384' heads='1' primary='yes'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+107:+107</label>
    <imagelabel>+107:+107</imagelabel>
  </seclabel>
</domain>


=========================================================================

to actually trigger the worker scale out I used: 

oc -n openshift-machine-api scale --replicas=3 machineset seal55-cluster-0-lhpwd-worker-0


(otherwise new bmh just stays in "ready" state without getting provisioned)


==========================================================================

Comment 10 Sasha Smolyak 2021-03-16 13:49:20 UTC
These changes have been tested. Since now it's possible to test bmh-related features, closing this bug

Comment 13 errata-xmlrpc 2021-04-05 13:55:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.5 security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:1005


Note You need to log in before you can comment on or make changes to this bug.