Bug 1987914

Summary: Missing part-of label on prometheus-operator deployment
Product: OpenShift Container Platform Reporter: Robert Bost <rbost>
Component: MonitoringAssignee: Jayapriya Pai <janantha>
Status: CLOSED DUPLICATE QA Contact: Junqi Zhao <juzhao>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.8CC: alegrand, amuller, anpicker, aos-bugs, arajkuma, erooth, kakkoyun, pkrupa, vrutkovs
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-07-30 08:51:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Robert Bost 2021-07-29 16:53:52 UTC 
Description of problem: During upgrade from 4.7.21 -> 4.8.2, CVO is spinning on prometheus operator webhook:

E0729 16:48:21.339429       1 task.go:112] error running apply for prometheusrule "openshift-marketplace/marketplace-alert-rules" (523 of 676): Internal error occurred: failed calling webhook "prometheusrules.openshift.io": Post "https://prometheus-operator.openshift-monitoring.svc:8080/admission-prometheusrules/validate?timeout=5s": no endpoints available for service "prometheus-operator"

Upon further inspection, the prometheus-operator service's labels do not match the running pod or deployment:

$ oc get pod -n openshift-monitoring -l app.kubernetes.io/component=controller,app.kubernetes.io/name=prometheus-operator,app.kubernetes.io/part-of=openshift-monitoring
No resources found in openshift-monitoring namespace.


Version-Release number of selected component (if applicable): 4.7.21 -> 4.8.2


How reproducible: Unsure


Steps to Reproduce:
1. Initiated upgrade to 4.8.2 via fast channel

Actual results: Stalling upgrade, potentially held up on monitoring cluster operator.


Additional info: This currently impacts the vSphere (VMC) OpenShift CI cluster.
Comment 3 Vadim Rutkovsky 2021-07-30 08:51:13 UTC 
CMO updated the service with a new selector, but deployment could not be updated. Closing as duplicate of #1956308

*** This bug has been marked as a duplicate of bug 1956308 ***