Bug 1922187
| Summary: | [build-watch] release-openshift-origin-installer-e2e-aws-upgrade 4.6-stable-to-4.7-ci failing | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Kevin Pouget <kpouget> |
| Component: | Machine Config Operator | Assignee: | Antonio Murdaca <amurdaca> |
| Status: | CLOSED DUPLICATE | QA Contact: | Michael Nguyen <mnguyen> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 4.7 | CC: | aos-bugs, behoward, danw, mfojtik, vrutkovs, wlewis |
| Target Milestone: | --- | Keywords: | Reopened, Upgrades |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: |
[sig-node] Managed cluster should report ready nodes the entire duration of the test run [Late]
|
|
| Last Closed: | 2021-02-03 18:17:04 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Kevin Pouget
2021-01-29 12:31:52 UTC
Lets track this in existing bug *** This bug has been marked as a duplicate of bug 1921198 *** Sorry, I spoke too soon on Slack on Friday; this is not the same as 1921198; in that bug, the upgrade succeeds, but the e2e job complains because the apiserver was unreachable at one point (because of an OVS problem). Whereas in this one the apiserver is actually failing and marking itself degraded. The openshift-apiserver pod
"name": "apiserver-d559dcf48-b7n2b",
"namespace": "openshift-apiserver",
shows:
"status": {
"conditions": [
{
"lastProbeTime": null,
"lastTransitionTime": "2021-01-29T09:18:58Z",
"message": "0/6 nodes are available: 2 node(s) didn't match Pod's node affinity, 2 node(s) didn't match pod affinity/anti-affinity, 2 node(s) didn't match pod anti-affinity rules, 2 node(s) were unschedulable.",
"reason": "Unschedulable",
"status": "False",
"type": "PodScheduled"
}
],
"phase": "Pending",
"qosClass": "Burstable"
}
The node ip-10-0-146-224.ec2.internal has suspicious annotations:
"machineconfiguration.openshift.io/reason": "unexpected on-disk state validating against rendered-master-94090a3396a605db226f3f20b4c8e931: content mismatch for file \"/etc/systemd/system/pivot.service.d/10-mco-default-env.conf\"",
"machineconfiguration.openshift.io/state": "Degraded",
with
"spec": {
"providerID": "aws:///us-east-1b/i-0862a1b8ba235bbf9",
"taints": [
{
"effect": "NoSchedule",
"key": "node-role.kubernetes.io/master"
},
{
"effect": "NoSchedule",
"key": "node.kubernetes.io/unschedulable",
"timeAdded": "2021-01-29T09:18:56Z"
}
],
"unschedulable": true
},
https://prow.ci.openshift.org/?job=release-openshift-origin-installer-e2e-aws-upgrade shows better test history since first reported. Is this still an UpgradeBlocker? This should have been fixed with the following PRs: https://github.com/openshift/machine-config-operator/commit/44d95f3dd112a2a56935edb93a61953b83ab5f79 https://github.com/openshift/machine-config-operator/commit/3837e3812207d09a76c31bd1f1abb23e8fbfc1fd *** This bug has been marked as a duplicate of bug 1920027 *** Removing UpgradeBlocker from this older bug, to remove it from the suspect queue described in [1]. If you feel like this bug still needs to be a suspect, please add keyword again. [1]: https://github.com/openshift/enhancements/pull/475 |