Backport the Upgradeable=False blocking to allow bumps within a z stream, since folks are running into trouble with 4.3 -> 4.3 updates on clusters with DefaultSecurityContextConstraints_Mutated firing.
*** Bug 1821905 has been marked as a duplicate of this bug. ***
Backport from bz1797624, and considering #1821905, qe will verify it with scc trigger of upgradeable=false condition to cover more scenarios. versions: 4.3.12-4.3.13 Steps: 1. install ocp v4.3.12 2. trigger upgradeable=false by mutating default scc # ./oc get scc privileged -o json|jq .users [ "system:admin", "system:serviceaccount:openshift-infra:build-controller", "e2e-user" ] # ./oc get clusterversion version -o json|jq .status.conditions[-1] { "lastTransitionTime": "2020-04-16T02:29:59Z", "message": "Cluster operator kube-apiserver cannot be upgraded: DefaultSecurityContextConstraintsUpgradeable: Default SecurityContextConstraints object(s) have mutated [privileged]", "reason": "DefaultSecurityContextConstraints_Mutated", "status": "False", "type": "Upgradeable" } 3. change to candidate channel and do upgrade # ./oc adm upgrade --to 4.3.13 Updating to 4.3.13 # ./oc adm upgrade info: An upgrade is in progress. Unable to apply 4.3.13: it may not be safe to apply this update I0416 02:37:52.139229 1 sync_worker.go:471] Running sync 4.3.13 (force=false) on generation 3 in state Updating at attempt 1 I0416 02:37:52.139263 1 sync_worker.go:477] Loading payload I0416 02:37:52.139323 1 payload.go:210] Loading updatepayload from "/etc/cvo/updatepayloads/JO9H_J5oBM13JfFeCnU7hQ" E0416 02:37:52.435507 1 precondition.go:49] Precondition "ClusterVersionUpgradeable" failed: Cluster operator kube-apiserver cannot be upgraded: DefaultSecurityContextConstraintsUpgradeable: Default SecurityContextConstraints object(s) have mutated [privileged] E0416 02:37:52.435558 1 sync_worker.go:329] unable to synchronize image (waiting 2m52.525702462s): Precondition "ClusterVersionUpgradeable" failed because of "DefaultSecurityContextConstraints_Mutated": Cluster operator kube-apiserver cannot be upgraded: DefaultSecurityContextConstraintsUpgradeable: Default SecurityContextConstraints object(s) have mutated [privileged] The bump from 4.3.z to 4.3.(z+1) still blocked from above result. So assign it back. Please correct me if any important thing missing here.
> The bump from 4.3.z to 4.3.(z+1) still blocked from above result. It's the source release that matters. So 4.3.12 -> 4.3.13 is blocked by the source release (4.3.12) not having the CVO patch. To verify this, you'll need to wait until 4.3.14 gets cut next week, or use an alternative Cincinnati upstream that suggests an edge from 4.3.13 to some other, signed, 4.3.z release (could even be suggesting 4.3.13 -> 4.3.12).
(In reply to W. Trevor King from comment #9) > > The bump from 4.3.z to 4.3.(z+1) still blocked from above result. > > It's the source release that matters. So 4.3.12 -> 4.3.13 is blocked by the > source release (4.3.12) not having the CVO patch. Thx, @King, you're right, pr#315 was not included in 4.3.12 after a deep check. I checked it before, but just have a quick check from commit time and build time. To verify this, you'll > need to wait until 4.3.14 gets cut next week, or use an alternative > Cincinnati upstream that suggests an edge from 4.3.13 to some other, signed, > 4.3.z release (could even be suggesting 4.3.13 -> 4.3.12). I will use a dummy server to verify the bug with 4.3.13-4.3.12 later.
versions: 4.3.13-4.3.12 Steps: 1. install ocp v4.3.13 2. trigger upgradeable=false by mutating default scc # ./oc get scc privileged -o json|jq .users [ "system:admin", "system:serviceaccount:openshift-infra:build-controller", "e2e-user" ] # ./oc get clusterversion version -o json|jq .status.conditions[-1] { "lastTransitionTime": "2020-04-16T07:13:32Z", "message": "Cluster operator kube-apiserver cannot be upgraded: DefaultSecurityContextConstraintsUpgradeable: Default SecurityContextConstraints object(s) have mutated [privileged]", "reason": "DefaultSecurityContextConstraints_Mutated", "status": "False", "type": "Upgradeable" } 3. set upstream to a dummy server with upgrade path 4.3.13-4.3.12 and do upgrade # ./oc adm upgrade --to 4.3.12 Updating to 4.3.12 # ./oc get clusterversion -o json|jq -r '.items[0].status.history[]'{ "completionTime": "2020-04-16T09:25:19Z", "image": "quay.io/openshift-release-dev/ocp-release@sha256:75e8f20e9d5a8fcf5bba4b8f7d17057463e222e350bcfc3cf7ea2c47f7d8ba5d", "startedTime": "2020-04-16T08:45:06Z", "state": "Completed", "verified": true, "version": "4.3.12" } { "completionTime": "2020-04-16T06:54:36Z", "image": "quay.io/openshift-release-dev/ocp-release@sha256:e1ebc7295248a8394afb8d8d918060a7cc3de12c491283b317b80b26deedfe61", "startedTime": "2020-04-16T06:30:40Z", "state": "Completed", "verified": false, "version": "4.3.13" }
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:1482