Bug 2003896

Summary: (release-4.10) ApiRequestCounts conditional gatherer
Product: OpenShift Container Platform Reporter: Tomas Remes <tremes>
Component: Insights OperatorAssignee: Tomas Remes <tremes>
Status: CLOSED ERRATA QA Contact: Dmitry Misharov <dmisharo>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.10CC: aos-bugs, dmisharo, inecas, mklika, tremes
Target Milestone: ---   
Target Release: 4.10.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 2003893 Environment:
Last Closed: 2022-03-10 16:10:15 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2003893    

Description Tomas Remes 2021-09-14 06:16:39 UTC 
If the APIRemovedInNextEUSReleaseInUse alert is firing then look at its labels, get the resource, version and group from them and find the corresponding APIRequestCounts "record" for this resource and store the record.
Comment 1 Tomas Remes 2021-09-14 06:18:30 UTC 
Implemented with https://github.com/openshift/insights-operator/pull/492
Comment 2 Dmitry Misharov 2021-10-19 15:23:22 UTC 
Verified on 4.10.0-0.nightly-2021-10-16-173656.

Verification steps:

1. Disable cluster-monitoring-operator:
merge=$(cat <<-__EOF
    {
      "spec": {
        "overrides": [
          [ .spec | .? | .overrides[] | .? | select(.name != "cluster-monitoring-operator")] +
          [{
            "group": "apps/v1",
            "kind": "Deployment",
            "name": "cluster-monitoring-operator",
            "namespace": "openshift-monitoring",
            "unmanaged": true
          }]
        ] | flatten
      }
    }
__EOF
  )

overrides=$(oc get clusterversion version -o json | jq "$merge")
oc patch clusterversion/version --type=merge  -p="$overrides"
oc scale --replicas=0 deployment/cluster-monitoring-operator -n openshift-monitoring
2. Disable prometheus-operator
oc scale --replicas=0 deployment/prometheus-operator -n openshift-monitoring
3. Adjust "APIRemovedInNextEUSReleaseInUse" rule in "prometheus-k8s-rulefiles-0" config map to trigger it
4. Restart "prometheus-k8s" stateful set
oc scale --replicas=0 statefulset/prometheus-k8s -n openshift-monitoring
oc scale --replicas=2 statefulset/prometheus-k8s -n openshift-monitoring
5. Wait until the alert will be firing. You can check it in Prometheus UI:
https://prometheus-k8s-openshift-monitoring.apps.<cluster hostname>
6. Restart the Insights Operator
7. Wait until the archive will be created
8. Download and check the content of the archive.

The archive should content conditional/alerts/APIRemovedInNextEUSReleaseInUse/api_request_counts.json file
Comment 5 errata-xmlrpc 2022-03-10 16:10:15 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056