Bug 1960339

Summary: manifests: unset "preemptionPolicy" makes CVO hotloop
Product: OpenShift Container Platform Reporter: Vadim Rutkovsky <vrutkovs>
Component: config-operatorAssignee: Stefan Schimanski <sttts>
Status: CLOSED ERRATA QA Contact: Xingxing Xia <xxia>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.8CC: aos-bugs
Target Milestone: ---   
Target Release: 4.8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-07-27 23:08:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Vadim Rutkovsky 2021-05-13 17:12:00 UTC
Manifest doesn't have preemptionPolicy set, so CVO tries to remove it on every sync cycle

Comment 2 Vadim Rutkovsky 2021-06-02 15:27:00 UTC
Moving to POST as another change in cluster-config-operator is required

Comment 5 Xingxing Xia 2021-06-29 12:05:13 UTC
Sorry did not timely check it. Was fully occupied by burning down other on_qa bugs and other new coming work.

Today checked in 4.8.0-0.nightly-2021-06-29-033219 fresh env:
oc get pc openshift-user-critical -o yaml
apiVersion: scheduling.k8s.io/v1
description: This priority class should be used for user facing OpenShift workload
  pods only.
kind: PriorityClass
metadata:
  annotations:
    include.release.openshift.io/self-managed-high-availability: "true"
    include.release.openshift.io/single-node-developer: "true"
  creationTimestamp: "2021-06-29T09:11:37Z"
  generation: 1
  name: openshift-user-critical
  resourceVersion: "922"
  uid: 87054743-27b7-44d7-94e2-70904ffe8daa
preemptionPolicy: PreemptLowerPriority
value: 1000000000

It sets preemptionPolicy and has no globalDefault as the PRs do.
But checked CVO pod logs:
oc logs cluster-version-operator-694d98bc85-mwz5t -n openshift-cluster-version | grep -i priorityclass
...
I0629 11:19:09.549293       1 sync_worker.go:752] Running sync for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:19:09.579146       1 sync_worker.go:764] Done syncing for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:22:25.764389       1 sync_worker.go:752] Running sync for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:22:25.803163       1 sync_worker.go:764] Done syncing for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:25:42.095430       1 sync_worker.go:752] Running sync for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:25:42.129337       1 sync_worker.go:764] Done syncing for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:28:58.312801       1 sync_worker.go:752] Running sync for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:28:58.349362       1 sync_worker.go:764] Done syncing for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:32:14.523692       1 sync_worker.go:752] Running sync for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:32:14.562926       1 sync_worker.go:764] Done syncing for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:35:30.746833       1 sync_worker.go:752] Running sync for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:35:30.774766       1 sync_worker.go:764] Done syncing for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:38:47.053778       1 sync_worker.go:752] Running sync for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:38:47.090787       1 sync_worker.go:764] Done syncing for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:42:03.322813       1 sync_worker.go:752] Running sync for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:42:03.361631       1 sync_worker.go:764] Done syncing for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:45:19.533299       1 sync_worker.go:752] Running sync for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:45:19.571770       1 sync_worker.go:764] Done syncing for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:48:35.753839       1 sync_worker.go:752] Running sync for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:48:35.792080       1 sync_worker.go:764] Done syncing for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:51:51.960179       1 sync_worker.go:752] Running sync for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:51:51.998018       1 sync_worker.go:764] Done syncing for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:55:08.181023       1 sync_worker.go:752] Running sync for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:55:08.217326       1 sync_worker.go:764] Done syncing for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:58:24.391219       1 sync_worker.go:752] Running sync for priorityclass "openshift-user-critical" (411 of 676)
I0629 11:58:24.427691       1 sync_worker.go:764] Done syncing for priorityclass "openshift-user-critical" (411 of 676)
I0629 12:01:40.609428       1 sync_worker.go:752] Running sync for priorityclass "openshift-user-critical" (411 of 676)
I0629 12:01:40.662608       1 sync_worker.go:764] Done syncing for priorityclass "openshift-user-critical" (411 of 676)

Is this every-3-min repetition expected?

Comment 6 Vadim Rutkovsky 2021-06-29 15:57:22 UTC
(In reply to Xingxing Xia from comment #5)
> Is this every-3-min repetition expected?

The periodic sync is expected. These lines mean this manifest is being handled by CVO, but it doesn't mean its being hotlooping.

Check CVO logs, if it has "... updated due to diff..." periodically, it means we have a hotloop again.

Comment 7 Xingxing Xia 2021-06-30 02:29:26 UTC
Thx, no that periodical stuff is found, thus moving to VERIFIED

Comment 9 errata-xmlrpc 2021-07-27 23:08:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2438