Bug 2117679
| Summary: | kube-controller-manager needs to stop watching all events | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | OpenShift BugZilla Robot <openshift-bugzilla-robot> |
| Component: | kube-controller-manager | Assignee: | Ben Luddy <bluddy> |
| Status: | CLOSED ERRATA | QA Contact: | RamaKasturi <knarra> |
| Severity: | urgent | Docs Contact: | |
| Priority: | urgent | ||
| Version: | 4.12 | CC: | bluddy, knarra, mfojtik |
| Target Milestone: | --- | Keywords: | FastFix |
| Target Release: | 4.11.z | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2023-02-07 13:22:41 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 2117569, 2118318 | ||
| Bug Blocks: | |||
@benluddy could you please help add the other PR as well to the bug as that is needed to have complete fix. Version where this was tested: =============================== [knarra@knarra openshift-tests-private]$ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.11.0-0.ci.test-2022-10-06-065450-ci-ln-bpwy562-latest True False 26m Cluster version is 4.11.0-0.ci.test-2022-10-06-065450-ci-ln-bpwy562-latest Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: OpenShift Container Platform 4.11.26 security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2023:0565 |
Have tested this bug via per-merge testing and below are the steps i have followed to test the same. Steps followed to test the build with fix : ========================================= 1. create a cluster using cluster-bot with the command "launch openshift/kubernetes#1343,openshift/kubernetes#1379 no-spot" 2. Login to the cluster, run command 'oc debug node/<masternode>'; chroot /host 3. Run less /var/log/kube-apiserver/audit.log | jq -c 'select(.verb=="watch" and .objectRef.resource=="events" and .user.username=="system:kube-controller-manager")' 4. Verify that no output gets displayed which is expected. Results: =============== sh-4.4# less /var/log/kube-apiserver/audit.log | jq -c 'select(.verb=="watch" and .objectRef.resource=="events" and .user.username=="system:kube-controller-manager")' sh-4.4# Steps followed to test the build with out fix: ==================================================== 1. create a 4.11 cluster 2. Login to the cluster, run command 'oc debug node/<masternode>'; chroot /host 3. Run less /var/log/kube-apiserver/audit.log | jq -c 'select(.verb=="watch" and .objectRef.resource=="events" and .user.username=="system:kube-controller-manager")' 4. Verify that there is output that gets displayed which has logs related to events. Results: =============== sh-4.4# less /var/log/kube-apiserver/audit.log | jq -c 'select(.verb=="watch" and .objectRef.resource=="events" and .user.username=="system:kube-controller-manager")' {"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"f5221016-fe1f-4c54-b559-819223a8ea4f","stage":"ResponseComplete","requestURI":"/apis/events.k8s.io/v1/events?allowWatchBookmarks=true&resourceVersion=38712&timeout=5m27s&timeoutSeconds=327&watch=true","verb":"watch","user":{"username":"system:kube-controller-manager","groups":["system:authenticated"]},"sourceIPs":["10.0.214.219"],"userAgent":"cluster-policy-controller/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"events","apiGroup":"events.k8s.io","apiVersion":"v1"},"responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2022-10-06T07:10:41.197122Z","stageTimestamp":"2022-10-06T07:16:08.201028Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by ClusterRoleBinding \"system:kube-controller-manager\" of ClusterRole \"system:kube-controller-manager\" to User \"system:kube-controller-manager\""}} {"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"20af4be9-4a0f-4a49-84ca-db145f9a017a","stage":"ResponseStarted","requestURI":"/apis/events.k8s.io/v1/events?allowWatchBookmarks=true&resourceVersion=41527&timeout=5m17s&timeoutSeconds=317&watch=true","verb":"watch","user":{"username":"system:kube-controller-manager","groups":["system:authenticated"]},"sourceIPs":["10.0.214.219"],"userAgent":"cluster-policy-controller/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"events","apiGroup":"events.k8s.io","apiVersion":"v1"},"responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2022-10-06T07:16:08.202558Z","stageTimestamp":"2022-10-06T07:16:08.203681Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by ClusterRoleBinding \"system:kube-controller-manager\" of ClusterRole \"system:kube-controller-manager\" to User \"system:kube-controller-manager\""}} {"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"20af4be9-4a0f-4a49-84ca-db145f9a017a","stage":"ResponseComplete","requestURI":"/apis/events.k8s.io/v1/events?allowWatchBookmarks=true&resourceVersion=41527&timeout=5m17s&timeoutSeconds=317&watch=true","verb":"watch","user":{"username":"system:kube-controller-manager","groups":["system:authenticated"]},"sourceIPs":["10.0.214.219"],"userAgent":"cluster-policy-controller/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"events","apiGroup":"events.k8s.io","apiVersion":"v1"},"responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2022-10-06T07:16:08.202558Z","stageTimestamp":"2022-10-06T07:21:25.204262Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by ClusterRoleBinding \"system:kube-controller-manager\" of ClusterRole \"system:kube-controller-manager\" to User \"system:kube-controller-manager\""}} {"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"41deb121-a103-43cf-ad64-54e9315ce7ac","stage":"ResponseStarted","requestURI":"/apis/events.k8s.io/v1/events?allowWatchBookmarks=true&resourceVersion=43433&timeout=6m17s&timeoutSeconds=377&watch=true","verb":"watch","user":{"username":"system:kube-controller-manager","groups":["system:authenticated"]},"sourceIPs":["10.0.214.219"],"userAgent":"cluster-policy-controller/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"events","apiGroup":"events.k8s.io","apiVersion":"v1"},"responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2022-10-06T07:21:25.205570Z","stageTimestamp":"2022-10-06T07:21:25.206518Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by ClusterRoleBinding \"system:kube-controller-manager\" of ClusterRole \"system:kube-controller-manager\" to User \"system:kube-controller-manager\""}} {"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"41deb121-a103-43cf-ad64-54e9315ce7ac","stage":"ResponseComplete","requestURI":"/apis/events.k8s.io/v1/events?allowWatchBookmarks=true&resourceVersion=43433&timeout=6m17s&timeoutSeconds=377&watch=true","verb":"watch","user":{"username":"system:kube-controller-manager","groups":["system:authenticated"]},"sourceIPs":["10.0.214.219"],"userAgent":"cluster-policy-controller/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"events","apiGroup":"events.k8s.io","apiVersion":"v1"},"responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2022-10-06T07:21:25.205570Z","stageTimestamp":"2022-10-06T07:27:42.207260Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by ClusterRoleBinding \"system:kube-controller-manager\" of ClusterRole \"system:kube-controller-manager\" to User \"system:kube-controller-manager\""}} {"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"c8e3efda-a8da-4c46-8fcb-55382ed9e9cb","stage":"ResponseStarted","requestURI":"/apis/events.k8s.io/v1/events?allowWatchBookmarks=true&resourceVersion=45164&timeout=9m50s&timeoutSeconds=590&watch=true","verb":"watch","user":{"username":"system:kube-controller-manager","groups":["system:authenticated"]},"sourceIPs":["10.0.214.219"],"userAgent":"cluster-policy-controller/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"events","apiGroup":"events.k8s.io","apiVersion":"v1"},"responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2022-10-06T07:27:42.208567Z","stageTimestamp":"2022-10-06T07:27:42.209242Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by ClusterRoleBinding \"system:kube-controller-manager\" of ClusterRole \"system:kube-controller-manager\" to User \"system:kube-controller-manager\""}} Based on the above setting Verified flag to tested to indicate the fix actually works fine.