Bug 2227761

Summary: openshift-monitoring apiservice on FailedDiscoveryCheck
Product: Container Native Virtualization (CNV) Reporter: Roni Kishner <rkishner>
Component: MetricsAssignee: Shirly Radco <sradco>
Status: ASSIGNED --- QA Contact: Natalie Gavrielov <ngavrilo>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.14.0CC: kmajcher, stirabos
Target Milestone: ---   
Target Release: 4.14.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Roni Kishner 2023-07-31 10:17:27 UTC 
Description of problem:
After trying to delete a namespace with some resource inside (Secret/VM), the apiservice resource 'openshift-monitoring/prometheus-adapter' is on FailedDiscoveryCheck, this causes the namespace deletion to be stuck on 'Terminating'

After deleting the apiservice (oc delete apiservice v1beta1.metrics.k8s.io) the namespace do get deleted after 1-3 minutes, and the issue does not repeat.

Version-Release number of selected component (if applicable):
4.14

How reproducible:
Need more investigation but seen more then a few times on different clusters

Steps to Reproduce:
1. Create a VM in a namespace
2. Delete the namespace

Actual results:
The namespace is stuck on Terminating

Expected results:
The namespace and all resources should be deleted

Additional info:
It is not clear if the apiservice failure is a side effect of another issue or it is what is causing this.
Comment 1 Kedar Bidarkar 2023-08-02 12:35:03 UTC 
Changing the component to Metrics, so that your team  could triage this bug. 
Discussed with Virt Devs and feel it could be related to Metrics thing.