Bug 1883412
| Summary: | cloud-credential upgradeable status didn't immediately respond when we remove or recreate the root creds in mint/passthrough mode | |||
|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | wang lin <lwan> | |
| Component: | Cloud Credential Operator | Assignee: | Devan Goodwin <dgoodwin> | |
| Status: | CLOSED ERRATA | QA Contact: | wang lin <lwan> | |
| Severity: | medium | Docs Contact: | ||
| Priority: | medium | |||
| Version: | 4.5 | CC: | gshereme, lwan, yunjiang | |
| Target Milestone: | --- | |||
| Target Release: | 4.5.z | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | ||
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1883644 (view as bug list) | Environment: | ||
| Last Closed: | 2020-12-01 10:48:48 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | 1883644 | |||
| Bug Blocks: | ||||
Sorry for that I put a wrong description info in "Actual results" above Actual results: Cloud-credential doesn’t immediately update the upgradeable status when we remove or re-add the root creds until we force a cco reconcile via adding an annotations in CloudCredential object or deleting the cco pod The issue has fixed.
test payload:4.5.0-0.nightly-2020-11-22-160319
1. Run "oc edit cloudcredential cluster" and modify .spec.credentialsMode to "Mint" or "Passthrough"
2. remove root creds
oc delete secret aws-creds -n kube-system
3. check co cloud-credential status, Upgradeable=False
$ oc get co cloud-credential -o json | jq -r ".status.conditions"
[
{
"lastTransitionTime": "2020-11-23T02:31:48Z",
"message": "No credentials requests reporting errors.",
"reason": "NoCredentialsFailing",
"status": "False",
"type": "Degraded"
},
{
"lastTransitionTime": "2020-11-23T02:50:15Z",
"message": "4 of 4 credentials requests provisioned and reconciled.",
"reason": "ReconcilingComplete",
"status": "False",
"type": "Progressing"
},
{
"lastTransitionTime": "2020-11-23T02:31:48Z",
"status": "True",
"type": "Available"
},
{
"lastTransitionTime": "2020-11-23T03:35:37Z",
"message": "Parent credential secret must be restored prior to upgrade: kube-system/aws-creds",
"reason": "MissingRootCredential",
"status": "False",
"type": "Upgradeable"
}
]
4. re-create root credential
5. check co cloud-credential status, Upgradeable=True
$ oc get co cloud-credential -o json | jq -r ".status.conditions"
[
{
"lastTransitionTime": "2020-11-23T02:31:48Z",
"message": "No credentials requests reporting errors.",
"reason": "NoCredentialsFailing",
"status": "False",
"type": "Degraded"
},
{
"lastTransitionTime": "2020-11-23T02:50:15Z",
"message": "4 of 4 credentials requests provisioned and reconciled.",
"reason": "ReconcilingComplete",
"status": "False",
"type": "Progressing"
},
{
"lastTransitionTime": "2020-11-23T02:31:48Z",
"status": "True",
"type": "Available"
},
{
"lastTransitionTime": "2020-11-23T03:36:22Z",
"status": "True",
"type": "Upgradeable"
}
]
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.5.21 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:5194 |
Description of problem: A backport PR to 4.5 involves a new issue. This PR adds a function that if cluster is in mint/passthrough mode, and at the same time root creds being removed, it will set cloud-credential upgradeable=false. But the issue now is the state won't change immediately when we remove or re-add the root creds, it maybe need to wait for several hours. The same issue occurred when re-creating root creds while upgradeable=false, the state won't change immediately to upgradeable=true. Version-Release number of selected component (if applicable): 4.5.0-0.nightly-2020-09-28-124031 How reproducible: always Steps to Reproduce: 1. Launch an IPI with latest 4.5 2. Check the root creds is in mint mode $oc get secret aws-creds -n kube-system -o json | jq .metadata.annotations { "cloudcredential.openshift.io/mode": "mint" } 3.Check the cloud-credential upgradeable=true $oc get co cloud-credential -o json | jq -r ".status.conditions" [ { "lastTransitionTime": "2020-09-28T04:24:19Z", "status": "True", "type": "Upgradeable" } ] 4. Remove the root creds ,check the cloud-credential upgradeable status again，the status is still upgradeable=true 5. The same situation when we re-add root creds, check the upgradeable=false now, and recreate root creds, verify the cloud-credential condiftion is still upgradeable=false Actual results: 1. Launch an IPI with latest 4.5 2. Check the root creds is in mint mode $oc get secret aws-creds -n kube-system -o json | jq .metadata.annotations { "cloudcredential.openshift.io/mode": "mint" } 3.Check the cloud-credential upgradeable=true $oc get co cloud-credential -o json | jq -r ".status.conditions" [ { "lastTransitionTime": "2020-09-28T04:24:19Z", "status": "True", "type": "Upgradeable" } ] 4. Remove the root creds ,check the cloud-credential upgradeable status again，the status is still upgradeable=true 5. The same situation when we re-add root creds, check the upgradeable=false now, and recreate root creds, verify the cloud-credential condiftion is still upgradeable=false Expected results: It can immediately update the upgradeable status. Additional info: [1] in version 4.6, it works well [2] related bug https://bugzilla.redhat.com/show_bug.cgi?id=1871713