Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 2023042

Summary: CRI-O filters custom runtime allowed annotation when both custom workload and custom runtime sections specified under the config
Product: OpenShift Container Platform Reporter: Artyom <alukiano>
Component: NodeAssignee: Peter Hunt <pehunt>
Node sub component: CRI-O QA Contact: Weinan Liu <weinliu>
Status: CLOSED ERRATA Docs Contact:
Severity: urgent    
Priority: high CC: aos-bugs, dgonyier, fromani, ykashtan
Version: 4.10   
Target Milestone: ---   
Target Release: 4.10.0   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-03-10 16:27:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Artyom 2021-11-14 09:38:59 UTC
Description of problem:
When both custom runtime and custom workload are specified under the CRI-O configuration, the CRI-O will filter runtime allowed annotation.

Version-Release number of selected component (if applicable):
1.23.0-12.rhaos4.10.git6ee64e9.el8

How reproducible:
Always

Steps to Reproduce:

1. Under the CRI-O config specify
# cat /etc/crio/crio.conf.d/00-default
...

[crio.runtime.workloads.openshift-builder]
activation_annotation = "io.openshift.builder"
allowed_annotations = [
  "io.kubernetes.cri-o.userns-mode",
  "io.kubernetes.cri-o.Devices"
]

# cat /etc/crio/crio.conf.d/99-runtimes.conf
...

[crio.runtime.runtimes.high-performance]
runtime_path = "/bin/runc"
runtime_type = "oci"
runtime_root = "/run/runc"
allowed_annotations = ["cpu-load-balancing.crio.io", "cpu-quota.crio.io", "irq-load-balancing.crio.io"]

2. Create a new RuntimeClass that points to custom runtime
apiVersion: node.k8s.io/v1
handler: high-performance
kind: RuntimeClass
metadata:
  name: performance-manual

3. Create a pod that will use custom runtime and the "cpu-load-balancing.crio.io" annotation
apiVersion: v1
kind: Pod
metadata:
  name: busybox-1
  labels:
    app: busybox
  annotations:
    cpu-load-balancing.crio.io: "disable"
spec:
  runtimeClassName: performance-manual
  containers:
  - image: busybox
    command:
      - /bin/sh
      - -c
      - sleep 600
    name: busybox-2
    imagePullPolicy: IfNotPresent
    resources:
      limits:
        cpu: 1
        memory: 64Mi
  restartPolicy: Always

4. Check under the node via "crictl inspect <container_id>" container annotations


Actual results:
The "cpu-load-balancing.crio.io" annotation does not exist under the container

Expected results:
The "cpu-load-balancing.crio.io" annotation should exist under the container

Additional info:
If I remove the custom workload section from the config, all starts work as expected.

Comment 1 Martin Sivák 2021-11-15 07:41:59 UTC
Raising the severity to urgent. This breaks an important use case (low latency tuning for Telcos) and there is no workaround.

Comment 2 Peter Hunt 2021-11-16 15:14:53 UTC
this is caused by a combination of https://github.com/openshift/machine-config-operator/pull/2805 and https://github.com/cri-o/cri-o/pull/5358/commits/83518f0981759138ec6fcde414def7f2c751d641. I am in conversation with Artyom to decide on the best path forward

Comment 4 Peter Hunt 2021-11-16 16:43:07 UTC
fixed in attached PR

Comment 5 Peter Hunt 2021-11-29 14:47:23 UTC
PR merged

Comment 10 errata-xmlrpc 2022-03-10 16:27:22 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056