Description of problem: Version-Release number of the following components: Client Version: 4.3.5 Server Version: 4.3.9 Kubernetes Version: v1.16.2 Channel: stable-4.2 How reproducible: Steps to Reproduce: 1. oc adm upgrade --to=4.3.10 Actual results: $ oc get clusterversion version NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.3.9 True True 7h44m Unable to apply 4.3.10: it may not be safe to apply this update $ oc describe clusterversion version Name: version Namespace: Labels: <none> Annotations: <none> API Version: config.openshift.io/v1 Kind: ClusterVersion Metadata: Creation Timestamp: 2020-03-16T17:53:57Z Generation: 3 Resource Version: 3558789 Self Link: /apis/config.openshift.io/v1/clusterversions/version UID: d5c29c0c-1e50-4411-a7e6-fdd8c4c48f0a Spec: Channel: stable-4.3 Cluster ID: 9d18a202-2a8a-4258-86fb-4319e20c8080 Desired Update: Force: false Image: quay.io/openshift-release-dev/ocp-release@sha256:edb4364367cff4f751ffdc032bc830a469548f998127b523047a8dd518c472cd Version: 4.3.10 Upstream: https://api.openshift.com/api/upgrades_info/v1/graph Status: Available Updates: Force: false Image: quay.io/openshift-release-dev/ocp-release@sha256:edb4364367cff4f751ffdc032bc830a469548f998127b523047a8dd518c472cd Version: 4.3.10 Conditions: Last Transition Time: 2020-03-16T18:37:42Z Message: Done applying 4.3.9 Status: True Type: Available Last Transition Time: 2020-04-14T02:51:40Z Message: Precondition "ClusterVersionUpgradeable" failed because of "DefaultSecurityContextConstraints_Mutated": Cluster operator kube-apiserver cannot be upgraded: DefaultSecurityContextConstraintsUpgradeable: Default SecurityContextConstraints object(s) have mutated [anyuid hostmount-anyuid] Reason: UpgradePreconditionCheckFailed Status: True Type: Failing Last Transition Time: 2020-04-13T19:08:21Z Message: Unable to apply 4.3.10: it may not be safe to apply this update Reason: UpgradePreconditionCheckFailed Status: True Type: Progressing Last Transition Time: 2020-04-13T16:46:02Z Status: True Type: RetrievedUpdates Last Transition Time: 2020-04-11T07:33:55Z Message: Cluster operator kube-apiserver cannot be upgraded: DefaultSecurityContextConstraintsUpgradeable: Default SecurityContex tConstraints object(s) have mutated [anyuid hostmount-anyuid] Reason: DefaultSecurityContextConstraints_Mutated Status: False Type: Upgradeable Desired: Force: false Image: quay.io/openshift-release-dev/ocp-release@sha256:edb4364367cff4f751ffdc032bc830a469548f998127b523047a8dd518c472cd Version: 4.3.10 History: Completion Time: <nil> Image: quay.io/openshift-release-dev/ocp-release@sha256:edb4364367cff4f751ffdc032bc830a469548f998127b523047a8dd518c472cd Started Time: 2020-04-13T19:08:21Z State: Partial Verified: true Version: 4.3.10 Completion Time: 2020-04-12T16:30:21Z Image: quay.io/openshift-release-dev/ocp-release@sha256:f0fada3c8216dc17affdd3375ff845b838ef9f3d67787d3d42a88dcd0f328eea Started Time: 2020-04-11T07:29:45Z State: Completed Verified: true Version: 4.3.9 Completion Time: 2020-03-16T18:37:42Z Image: quay.io/openshift-release-dev/ocp-release@sha256:64320fbf95d968fc6b9863581a92d373bc75f563a13ae1c727af37450579f61a Started Time: 2020-03-16T17:53:57Z State: Completed Verified: false Version: 4.3.5 Observed Generation: 3 Version Hash: CNrYpBZUUr8= Events: <none> Expected results: $ oc adm upgrade $ oc get clusterversion version should be work Additional info: Please attach logs from ansible-playbook with the -vvv flag
I learn some news from Bug 1818893 Provide more informations from SCC anyuid and hostmount-anyuid $ oc get scc anyuid -o yaml allowHostDirVolumePlugin: false allowHostIPC: false allowHostNetwork: false allowHostPID: false allowHostPorts: false allowPrivilegeEscalation: true allowPrivilegedContainer: false allowedCapabilities: null apiVersion: security.openshift.io/v1 defaultAddCapabilities: null fsGroup: type: RunAsAny groups: - system:cluster-admins kind: SecurityContextConstraints metadata: annotations: kubernetes.io/description: anyuid provides all features of the restricted SCC but allows users to run with any UID and any GID. creationTimestamp: "2020-03-16T18:23:04Z" generation: 4 name: anyuid resourceVersion: "268421" selfLink: /apis/security.openshift.io/v1/securitycontextconstraints/anyuid uid: a66c7a1d-eda5-42ee-97e0-a814caee19cf priority: 10 readOnlyRootFilesystem: false requiredDropCapabilities: - MKNOD runAsUser: type: RunAsAny seLinuxContext: type: MustRunAs supplementalGroups: type: RunAsAny users: - system:serviceaccount:debugging:runasanyuid volumes: - configMap - downwardAPI - emptyDir - persistentVolumeClaim - projected - secret $ oc get scc hostmount-anyuid -o yaml allowHostDirVolumePlugin: true allowHostIPC: false allowHostNetwork: false allowHostPID: false allowHostPorts: false allowPrivilegeEscalation: true allowPrivilegedContainer: false allowedCapabilities: null apiVersion: security.openshift.io/v1 defaultAddCapabilities: null fsGroup: type: RunAsAny groups: [] kind: SecurityContextConstraints metadata: annotations: kubernetes.io/description: 'hostmount-anyuid provides all the features of the restricted SCC but allows host mounts and any UID by a pod. This is primarily used by the persistent volume recycler. WARNING: this SCC allows host file system access as any UID, including UID 0. Grant with caution.' creationTimestamp: "2020-03-16T18:23:04Z" generation: 2 name: hostmount-anyuid resourceVersion: "2291577" selfLink: /apis/security.openshift.io/v1/securitycontextconstraints/hostmount-anyuid uid: 853871a0-535f-4c2b-a462-8ce2b8682369 priority: null readOnlyRootFilesystem: false requiredDropCapabilities: - MKNOD runAsUser: type: RunAsAny seLinuxContext: type: MustRunAs supplementalGroups: type: RunAsAny users: - system:serviceaccount:openshift-infra:pv-recycler-controller - system:serviceaccount:default:nfs-client-provisioner volumes: - configMap - downwardAPI - emptyDir - hostPath - nfs - persistentVolumeClaim - projected - secret
In SCC hostmount-anyuid, I try to remove users `- system:serviceaccount:default:nfs-client-provisioner`, and it is work. But I don't have any idea in SCC anyuid, any comment are welcome
OK... I dont known what happened $ oc get clusterversion version NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.3.10 True False 2m46s Cluster version is 4.3.10 $ oc edit scc hostmount-anyuid and put `- system:serviceaccount:default:nfs-client-provisioner` back to users lists I think it is not a reality to change SCC on the production environment when upgrading OCP, and it should provide a more clear and specific solution with users.
*** This bug has been marked as a duplicate of bug 1821905 ***