Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1976243

Summary: OLM operator index pod for Performance Addon Operator is missing Workload Partitioning Annotation
Product: OpenShift Container Platform Reporter: Ian Miller <imiller>
Component: Performance Addon OperatorAssignee: Ian Miller <imiller>
Status: CLOSED ERRATA QA Contact: yliu1
Severity: low Docs Contact:
Priority: low    
Version: 4.8CC: aos-bugs, dgonyier, eparis, jokerman, keyoung, minmli, mniranja, shajmakh, yliu1
Target Milestone: ---   
Target Release: 4.9.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-10-18 17:36:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Ian Miller 2021-06-25 15:39:04 UTC
Description of problem:
Ther Performance Addon Operator index pod is created in the openshift-marketplace namespace without needed Workload Partitioning annotation. 

apiVersion: v1
kind: Pod
metadata:
  annotations:
    k8s.v1.cni.cncf.io/network-status: |-
<snip>
    k8s.v1.cni.cncf.io/networks-status: |-
<snip>
    openshift.io/scc: anyuid
  creationTimestamp: "2021-06-24T20:59:17Z"
  generateName: performance-addon-operator-
  labels:
    olm.catalogSource: performance-addon-operator
  managedFields:
<snip>
  name: performance-addon-operator-99cwx
  namespace: openshift-marketplace
  ownerReferences:
  - apiVersion: operators.coreos.com/v1alpha1
    blockOwnerDeletion: false
    controller: false
    kind: CatalogSource
    name: performance-addon-operator
    uid: 0ac2cca5-e34a-49b3-bb6e-2f390dc3a7dd
  resourceVersion: "726656"
  uid: 56d493d8-1f05-493e-a20b-039493217f8f

When workload partitioning is enabled this pod is not moved to the management cores and takes/requires CPU requests from user workload capacity.

Version-Release number of selected component (if applicable): 4.8


How reproducible: always


Steps to Reproduce:
1. Install PAO in SNO cluster with workload partitioning enabled
2. oc describe node shows 10m CPU request for pao pod in openshift-marketplace


Actual results:

  openshift-marketplace                             performance-addon-operator-h9mk8                                              10m (0%)      0 (0%)      50Mi (0%)        0 (0%)         25h


Expected results:

  openshift-marketplace                             performance-addon-operator-h9mk8                                              0 (0%)      0 (0%)      50Mi (0%)        0 (0%)         25h


Additional info:

Comment 1 Ian Miller 2021-06-25 16:50:50 UTC
CatalogSource CR needs to be annotated for workload partitioning:
apiVersion: operators.coreos.com/v1alpha1
metadata:
  name: performance-addon-operator
  namespace: openshift-marketplace
  annotations:
    target.workload.openshift.io/management: '{"effect": "PreferredDuringScheduling"}'
kind: CatalogSource
spec:
  displayName: Openshift Performance Addon Operator
  icon:
    base64data: ""
    mediatype: ""
  image: quay.io/openshift-kni/performance-addon-operator-index:4.8-snapshot
  publisher: Red Hat
  sourceType: grpc


Testing underway with this change

Comment 5 MinLi 2021-09-25 14:43:48 UTC
test on 4.8.0-0.nightly-2021-09-21-220555, I try to install PAO in SNO cluster by 2 ways: 
1 using default CatalogSource : redhat-operators , FYI:https://docs.openshift.com/container-platform/4.8/scalability_and_performance/cnf-performance-addon-operator-for-low-latency-nodes.html#install-operator-cli_cnf-master
2 using custom CatalogSource like Comment 1
But both don't get expected results:

openshift-marketplace                             performance-addon-operator-xjxps                           10m (0%)      0 (0%)      50Mi (0%)        0 (0%)         36m

Comment 6 Ian Miller 2021-09-28 12:12:04 UTC
Hi @minmli,
Was this cluster deployed with Workload Partitioning enabled? If Workload Partitioning is not enabled on the cluster then the CPU requests field will not be mutated (converted over to "cores" and request set to 0). 
Thanks!
Ian

(In reply to MinLi from comment #5)
> test on 4.8.0-0.nightly-2021-09-21-220555, I try to install PAO in SNO
> cluster by 2 ways: 
> 1 using default CatalogSource : redhat-operators ,
> FYI:https://docs.openshift.com/container-platform/4.8/
> scalability_and_performance/cnf-performance-addon-operator-for-low-latency-
> nodes.html#install-operator-cli_cnf-master
> 2 using custom CatalogSource like Comment 1
> But both don't get expected results:
> 
> openshift-marketplace                            
> performance-addon-operator-xjxps                           10m (0%)      0
> (0%)      50Mi (0%)        0 (0%)         36m

Comment 7 MinLi 2021-09-29 08:19:02 UTC
I don't know how to enable Workload Partitioning, so I think qe from Performance Addon Operator team should re-test this fix.
@Gowrishankar Rajaiyan, please, thanks!

Comment 8 yliu1 2021-09-30 16:30:20 UTC
Verified on 4.8.13.
 
[yliu1@yliu1 ~]$ oc version
Client Version: 4.9.0-rc.0
Server Version: 4.8.13
Kubernetes Version: v1.21.1+a620f50

[yliu1@yliu1 ~]$ oc describe nodes  | grep perfo
  openshift-marketplace                             performance-addon-operator-42n86                                              0 (0%)        0 (0%)      50Mi (0%)        0 (0%)         37h
  openshift-performance-addon-operator              performance-operator-79fc9c6947-685qg                                         0 (0%)        0 (0%)      25Mi (0%)        0 (0%)         37h


[yliu1@yliu1 ~]$ oc get pods -n openshift-marketplace performance-addon-operator-42n86 -o yaml |grep workload
    resources.workload.openshift.io/registry-server: '{"cpushares": 10}'
    target.workload.openshift.io/management: '{"effect":"PreferredDuringScheduling"}'
        management.workload.openshift.io/cores: "10"
        management.workload.openshift.io/cores: "10"

Comment 10 errata-xmlrpc 2021-10-18 17:36:51 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.9.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:3759