Description of problem: The ClusterOperator lastTransitionTime field for conditions is meant to be updated on status field changes. Presently, CCO updates on status or reason or message changes. Version-Release number of selected component (if applicable): How reproducible: 100% Steps to Reproduce: 1. Put CCO into a degraded state. For exmaple make a CredentialsRequest that specifies a .spec.secretRef.namespace that doesn't exist. 2. Wait for Degraded=True, and notice the lastTransitionTime 3. Create a second CredentailsRequest that also references a non-existent namespace. Actual results: the Degraded condition's lastTransitionTime will be updated when the message changes from "1 of X CredentailsRequests failing" to "2 of X CredentailsRequests failing" Expected results: the lastTransitionTime should only change when Degraded status changes from true-to-false or false-to-true. Additional info:
Verified with 4.10.0-0.nightly-2021-11-16-212854. 1. Check cloud-credential status.conditions jianpingshu@jshu-mac mycluster % oc get co cloud-credential -o json | jq -r ".status.conditions" [ { "lastTransitionTime": "2021-11-17T00:28:18Z", "status": "True", "type": "Available" }, { "lastTransitionTime": "2021-11-17T00:28:22Z", "status": "False", "type": "Degraded" }, { "lastTransitionTime": "2021-11-17T00:36:15Z", "status": "False", "type": "Progressing" }, { "lastTransitionTime": "2021-11-17T00:28:18Z", "status": "True", "type": "Upgradeable" } ] 2.Create 1st CredentialsRequest that specifies a .spec.secretRef.namespace that doesn't exist (as below template) apiVersion: cloudcredential.openshift.io/v1 kind: CredentialsRequest metadata: name: my-cred-request1 namespace: openshift-cloud-credential-operator spec: secretRef: name: my-cred-request1-secret namespace: namespace-does-not-exist providerSpec: apiVersion: cloudcredential.openshift.io/v1 kind: AWSProviderSpec statementEntries: - effect: Allow action: - s3:DeleteBucket resource: "*" 3.Check cloud-credential status.conditions again jianpingshu@jshu-mac mycluster % oc get co cloud-credential -o json | jq -r ".status.conditions" [ { "lastTransitionTime": "2021-11-17T00:28:18Z", "status": "True", "type": "Available" }, { "lastTransitionTime": "2021-11-17T02:35:20Z", "message": "1 of 6 credentials requests are failing to sync.", "reason": "CredentialsFailing", "status": "True", "type": "Degraded" }, { "lastTransitionTime": "2021-11-17T02:35:18Z", "message": "5 of 6 credentials requests provisioned, 1 reporting errors.", "reason": "Reconciling", "status": "True", "type": "Progressing" }, { "lastTransitionTime": "2021-11-17T00:28:18Z", "status": "True", "type": "Upgradeable" } ] 4.Wait for minutes, create 2nd CredentialsRequest that specifies a .spec.secretRef.namespace that doesn't exist 5.Check cloud-credential status.conditions again. The lastTransitionTime of Degraded/Progressing kept no change and message updated. jianpingshu@jshu-mac mycluster % oc get co cloud-credential -o json | jq -r ".status.conditions" [ { "lastTransitionTime": "2021-11-17T00:28:18Z", "status": "True", "type": "Available" }, { "lastTransitionTime": "2021-11-17T02:35:20Z", "message": "2 of 7 credentials requests are failing to sync.", "reason": "CredentialsFailing", "status": "True", "type": "Degraded" }, { "lastTransitionTime": "2021-11-17T02:35:18Z", "message": "5 of 7 credentials requests provisioned, 2 reporting errors.", "reason": "Reconciling", "status": "True", "type": "Progressing" }, { "lastTransitionTime": "2021-11-17T00:28:18Z", "status": "True", "type": "Upgradeable" } ]
Reproduced with 4.10.0-0.nightly-2021-11-15-034648. 1. Check cloud-credential status.conditions jianpingshu@jshu-mac ~ % oc get co cloud-credential -o json | jq -r ".status.conditions" [ { "lastTransitionTime": "2021-11-16T23:47:07Z", "status": "True", "type": "Available" }, { "lastTransitionTime": "2021-11-16T23:47:07Z", "status": "False", "type": "Degraded" }, { "lastTransitionTime": "2021-11-16T23:53:45Z", "status": "False", "type": "Progressing" }, { "lastTransitionTime": "2021-11-16T23:47:07Z", "status": "True", "type": "Upgradeable" } ] 2.Create 1st CredentialsRequest that specifies a .spec.secretRef.namespace that doesn't exist (as below template) apiVersion: cloudcredential.openshift.io/v1 kind: CredentialsRequest metadata: name: my-cred-request1 namespace: openshift-cloud-credential-operator spec: secretRef: name: my-cred-request1-secret namespace: namespace-does-not-exist providerSpec: apiVersion: cloudcredential.openshift.io/v1 kind: AWSProviderSpec statementEntries: - effect: Allow action: - s3:DeleteBucket resource: "*" 3.Check cloud-credential status.conditions again jianpingshu@jshu-mac ~ % oc get co cloud-credential -o json | jq -r ".status.conditions" [ { "lastTransitionTime": "2021-11-16T23:47:07Z", "status": "True", "type": "Available" }, { "lastTransitionTime": "2021-11-17T02:23:44Z", "message": "1 of 6 credentials requests are failing to sync.", "reason": "CredentialsFailing", "status": "True", "type": "Degraded" }, { "lastTransitionTime": "2021-11-17T02:23:44Z", "message": "5 of 6 credentials requests provisioned, 1 reporting errors.", "reason": "Reconciling", "status": "True", "type": "Progressing" }, { "lastTransitionTime": "2021-11-16T23:47:07Z", "status": "True", "type": "Upgradeable" } ] 4.Wait for minutes, create 2nd CredentialsRequest that specifies a .spec.secretRef.namespace that doesn't exist 5.Check cloud-credential status.conditions again. The lastTransitionTime of Degraded/Progressing changed agin and message updated. jianpingshu@jshu-mac ~ % oc get co cloud-credential -o json | jq -r ".status.conditions" [ { "lastTransitionTime": "2021-11-16T23:47:07Z", "status": "True", "type": "Available" }, { "lastTransitionTime": "2021-11-17T02:30:41Z", "message": "2 of 7 credentials requests are failing to sync.", "reason": "CredentialsFailing", "status": "True", "type": "Degraded" }, { "lastTransitionTime": "2021-11-17T02:30:41Z", "message": "5 of 7 credentials requests provisioned, 2 reporting errors.", "reason": "Reconciling", "status": "True", "type": "Progressing" }, { "lastTransitionTime": "2021-11-16T23:47:07Z", "status": "True", "type": "Upgradeable" } ]
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:0056