Bug 2099686

Summary: FIPS issue on OCP SNO with RT Kernel via performance profile
Product: OpenShift Container Platform Reporter: OpenShift BugZilla Robot <openshift-bugzilla-robot>
Component: Machine Config OperatorAssignee: Yu Qi Zhang <jerzhang>
Machine Config Operator sub component: Machine Config Operator QA Contact: Rio Liu <rioliu>
Status: CLOSED ERRATA Docs Contact:
Severity: urgent    
Priority: high CC: dagray, jerzhang, mco-triage, mkrejci, nm-s, sregidor
Version: 4.10   
Target Milestone: ---   
Target Release: 4.10.z   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-07-25 07:07:10 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2096496    
Bug Blocks:    

Comment 2 Sergio 2022-07-19 16:45:17 UTC
Verified using single node IPI on AWS version:

$ oc get clusterversion
NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.10.0-0.nightly-2022-07-19-055833   True        False         70m     Cluster version is 4.10.0-0.nightly-2022-07-19-055833

1) Fips is enabled
$ oc debug node/ip-10-0-144-230.us-east-2.compute.internal  -- chroot /host cat /proc/sys/crypto/fips_enabled
Starting pod/ip-10-0-144-230us-east-2computeinternal-debug ...
To use host binaries, run `chroot /host`
1

Removing debug pod ...


2) oc create -f- <<'EOF'
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: master
  name: 50-fips-bz-poc
spec:
  config:
    ignition:
      version: 3.2.0
  kernelArguments:
  - trigger-sno-fips-issue=1
EOF

3) Wait for MCP to be updated.

$ oc get mc 50-fips-bz-poc
NAME             GENERATEDBYCONTROLLER   IGNITIONVERSION   AGE
50-fips-bz-poc                           3.2.0             5m42s

$ oc get mcp
NAME     CONFIG                                             UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT   AGE
master   rendered-master-6e0bc7db10a35dcb8794186b1f7efa2a   True      False      False      1              1                   1                     0                      99m
worker   rendered-worker-8a1a843742ad468f40058019dd121333   True      False      False      0              0                   0                     0                      99m


4) oc create -f- <<'EOF'
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: master
  name: 50-realtime-kernel
spec:
  config:
    ignition:
      version: 3.2.0
  kernelType: "realtime"
EOF

5) Wait for MCP to be updated

$ oc get mc 50-realtime-kernel
NAME                 GENERATEDBYCONTROLLER   IGNITIONVERSION   AGE
50-realtime-kernel                           3.2.0             7s
$ oc get mcp
NAME     CONFIG                                             UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT   AGE
master   rendered-master-43bfa9d8f45ae928c8c8beb83f1bb805   True      False      False      1              1                   1                     0                      110m
worker   rendered-worker-8a1a843742ad468f40058019dd121333   True      False      False      0              0                   0                     0                      110m

$ oc get mcp/master -o yaml| yq -y '.status'
conditions:
  - lastTransitionTime: 2022-07-19T14:52:02Z
    message: ''
    reason: ''
    status: 'False'
    type: RenderDegraded
  - lastTransitionTime: 2022-07-19T14:52:28Z
    message: ''
    reason: ''
    status: 'False'
    type: NodeDegraded
  - lastTransitionTime: 2022-07-19T14:52:28Z
    message: ''
    reason: ''
    status: 'False'
    type: Degraded
  - lastTransitionTime: 2022-07-19T16:41:13Z
    message: All nodes are updated with rendered-master-43bfa9d8f45ae928c8c8beb83f1bb805
    reason: ''
    status: 'True'
    type: Updated
  - lastTransitionTime: 2022-07-19T16:41:13Z
    message: ''
    reason: ''
    status: 'False'
    type: Updating
configuration:
  name: rendered-master-43bfa9d8f45ae928c8c8beb83f1bb805
  source:
    - apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfig
      name: 00-master
    - apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfig
      name: 01-master-container-runtime
    - apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfig
      name: 01-master-kubelet
    - apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfig
>>      name: 50-fips-bz-poc
    - apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfig
>>      name: 50-realtime-kernel
    - apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfig
      name: 99-master-fips
    - apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfig
      name: 99-master-generated-crio-seccomp-use-default
    - apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfig
      name: 99-master-generated-registries
    - apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfig
      name: 99-master-ssh
degradedMachineCount: 0
machineCount: 1
observedGeneration: 4
readyMachineCount: 1
unavailableMachineCount: 0
updatedMachineCount: 1

6) Verify configuration

$ oc debug node/ip-10-0-144-230.us-east-2.compute.internal -- chroot /host cat /proc/cmdline
Starting pod/ip-10-0-144-230us-east-2computeinternal-debug ...
To use host binaries, run `chroot /host`
>> BOOT_IMAGE=(hd0,gpt3)/ostree/rhcos-984c4fbfe5839f2ebafbd17591562311d8ddf25928baeca1939db50526cf05be/vmlinuz-4.18.0-305.49.1.rt7.121.el8_4.x86_64 random.trust_cpu=on console=tty0 console=ttyS0,115200n8 ostree=/ostree/boot.0/rhcos/984c4fbfe5839f2ebafbd17591562311d8ddf25928baeca1939db50526cf05be/0 ignition.platform.id=aws fips=1 boot=LABEL=boot root=UUID=91283154-17d4-4a03-a037-6ab24c738bf1 rw rootflags=prjquota trigger-sno-fips-issue=1

Removing debug pod ...


We move the status to VERIFIED

Comment 5 errata-xmlrpc 2022-07-25 07:07:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.10.24 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5664