https://github.com/openshift/machine-config-operator/pull/887 I'll provide steps to reproduce and verify next
To verify this BZ on AWS: - bring up a 4.2 cluster on AWS that already contains https://github.com/openshift/machine-config-operator/pull/887 (anything already since it's already in the latest payloads) - run the following for i in $(oc get pods -n openshift-machine-config-operator -l k8s-app=machine-config-daemon -o go-template --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}' | xargs); do oc rsh -n openshift-machine-config-operator $i chroot /rootfs mkdir -p /run/pivot; oc rsh -n openshift-machine-config-operator $i chroot /rootfs touch /run/pivot/reboot-needed; done - disable the CVO oc scale --replicas=0 -n openshift-cluster-version deploy/cluster-version-operator - edit the osimageurl ConfigMap and put a newer oscontainer pullspec in there from https://releases-art-jenkins.cloud.paas.upshift.redhat.com/, releases/rhcos-4.2 from the dropdown (an old oscontainer should work as well by downgrading): oc edit cm/machine-config-osimageurl -nopenshift-machine-config-operartor - the pools should now go through an os update (equal to a real upgrade for the sake of verifying and testing this BZ) - everything reconciles NOTE: make sure you have creds in the pullsecret for the cluster to pull oscontainers from https://releases-art-jenkins.cloud.paas.upshift.redhat.com/ - releases/rhcos-4.2
Verified on 4.2.0-0.nightly-2019-06-27-041730 using the steps above to upgrade to quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d0fc729885d9986f3846198a5b8dc3b2603a2fb273c4b5282c9d32ac0f84654b $ for i in $(oc get pods -n openshift-machine-config-operator -l k8s-app=machine-config-daemon -o go-template --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}' | xargs); do oc rsh -n openshift-machine-config-operator $i chroot /rootfs rpm-ostree status; done State: idle AutomaticUpdates: disabled Deployments: * pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d0fc729885d9986f3846198a5b8dc3b2603a2fb273c4b5282c9d32ac0f84654b CustomOrigin: Managed by pivot tool Version: 420.8.20190626.0 (2019-06-26T18:35:16Z) pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d9c4793f2b5a279f0ee0066c474b0d2b0d6f2cb5730d377a7ae144b9f6e1a25 CustomOrigin: Managed by pivot tool Version: 410.8.20190604.0 (2019-06-04T20:37:00Z) State: idle AutomaticUpdates: disabled Deployments: * pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d0fc729885d9986f3846198a5b8dc3b2603a2fb273c4b5282c9d32ac0f84654b CustomOrigin: Managed by pivot tool Version: 420.8.20190626.0 (2019-06-26T18:35:16Z) pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d9c4793f2b5a279f0ee0066c474b0d2b0d6f2cb5730d377a7ae144b9f6e1a25 CustomOrigin: Managed by pivot tool Version: 410.8.20190604.0 (2019-06-04T20:37:00Z) State: idle AutomaticUpdates: disabled Deployments: * pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d0fc729885d9986f3846198a5b8dc3b2603a2fb273c4b5282c9d32ac0f84654b CustomOrigin: Managed by pivot tool Version: 420.8.20190626.0 (2019-06-26T18:35:16Z) pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dd3bd07c9a4fd39d8039b137fa161c3a5b38a2855e8df9f6c16946ccdf7b3f31 CustomOrigin: Provisioned from oscontainer Version: 420.8.20190624.0 (2019-06-24T00:25:32Z) State: idle AutomaticUpdates: disabled Deployments: * pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d0fc729885d9986f3846198a5b8dc3b2603a2fb273c4b5282c9d32ac0f84654b CustomOrigin: Managed by pivot tool Version: 420.8.20190626.0 (2019-06-26T18:35:16Z) pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d9c4793f2b5a279f0ee0066c474b0d2b0d6f2cb5730d377a7ae144b9f6e1a25 CustomOrigin: Managed by pivot tool Version: 410.8.20190604.0 (2019-06-04T20:37:00Z) State: idle AutomaticUpdates: disabled Deployments: * pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d0fc729885d9986f3846198a5b8dc3b2603a2fb273c4b5282c9d32ac0f84654b CustomOrigin: Managed by pivot tool Version: 420.8.20190626.0 (2019-06-26T18:35:16Z) pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d9c4793f2b5a279f0ee0066c474b0d2b0d6f2cb5730d377a7ae144b9f6e1a25 CustomOrigin: Managed by pivot tool Version: 410.8.20190604.0 (2019-06-04T20:37:00Z) State: idle AutomaticUpdates: disabled Deployments: * pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d0fc729885d9986f3846198a5b8dc3b2603a2fb273c4b5282c9d32ac0f84654b CustomOrigin: Managed by pivot tool Version: 420.8.20190626.0 (2019-06-26T18:35:16Z) pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d9c4793f2b5a279f0ee0066c474b0d2b0d6f2cb5730d377a7ae144b9f6e1a25 CustomOrigin: Managed by pivot tool Version: 410.8.20190604.0 (2019-06-04T20:37:00Z)
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:2922