Bug 1724155

Summary: Upgrade failure with "Marking Degraded due to: during bootstrap: unexpected on-disk state validating" error.
Product: OpenShift Container Platform Reporter: Antonio Murdaca <amurdaca>
Component: Machine Config OperatorAssignee: Antonio Murdaca <amurdaca>
Status: CLOSED ERRATA QA Contact: Micah Abbott <miabbott>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.2.0CC: amurdaca, dmoessne, jmathian, miabbott, mnguyen, mzali
Target Milestone: ---   
Target Release: 4.2.0   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1723327 Environment:
Last Closed: 2019-10-16 06:32:41 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1723327, 1729345    
Bug Blocks:    

Comment 1 Antonio Murdaca 2019-06-26 11:12:03 UTC
https://github.com/openshift/machine-config-operator/pull/887

I'll provide steps to reproduce and verify next

Comment 2 Antonio Murdaca 2019-06-26 13:37:37 UTC
To verify this BZ on AWS:

- bring up a 4.2 cluster on AWS that already contains https://github.com/openshift/machine-config-operator/pull/887 (anything already since it's already in the latest payloads)
- run the following

for i in $(oc get pods -n openshift-machine-config-operator -l k8s-app=machine-config-daemon -o go-template --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}' | xargs); do oc rsh -n openshift-machine-config-operator $i chroot /rootfs mkdir -p /run/pivot; oc rsh -n openshift-machine-config-operator $i chroot /rootfs touch /run/pivot/reboot-needed; done

- disable the CVO

oc scale --replicas=0 -n openshift-cluster-version deploy/cluster-version-operator

- edit the osimageurl ConfigMap and put a newer oscontainer pullspec in there from https://releases-art-jenkins.cloud.paas.upshift.redhat.com/, releases/rhcos-4.2 from the dropdown (an old oscontainer should work as well by downgrading):

oc edit cm/machine-config-osimageurl -nopenshift-machine-config-operartor

- the pools should now go through an os update (equal to a real upgrade for the sake of verifying and testing this BZ)
- everything reconciles


NOTE: make sure you have creds in the pullsecret for the cluster to pull oscontainers from https://releases-art-jenkins.cloud.paas.upshift.redhat.com/ - releases/rhcos-4.2

Comment 4 Michael Nguyen 2019-06-27 18:21:38 UTC
Verified on 4.2.0-0.nightly-2019-06-27-041730 using the steps above to upgrade to quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d0fc729885d9986f3846198a5b8dc3b2603a2fb273c4b5282c9d32ac0f84654b

$ for i in $(oc get pods -n openshift-machine-config-operator -l k8s-app=machine-config-daemon -o go-template --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}' | xargs); do oc rsh -n openshift-machine-config-operator $i chroot /rootfs rpm-ostree status; done
State: idle
AutomaticUpdates: disabled
Deployments:
* pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d0fc729885d9986f3846198a5b8dc3b2603a2fb273c4b5282c9d32ac0f84654b
              CustomOrigin: Managed by pivot tool
                   Version: 420.8.20190626.0 (2019-06-26T18:35:16Z)

  pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d9c4793f2b5a279f0ee0066c474b0d2b0d6f2cb5730d377a7ae144b9f6e1a25
              CustomOrigin: Managed by pivot tool
                   Version: 410.8.20190604.0 (2019-06-04T20:37:00Z)
State: idle
AutomaticUpdates: disabled
Deployments:
* pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d0fc729885d9986f3846198a5b8dc3b2603a2fb273c4b5282c9d32ac0f84654b
              CustomOrigin: Managed by pivot tool
                   Version: 420.8.20190626.0 (2019-06-26T18:35:16Z)

  pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d9c4793f2b5a279f0ee0066c474b0d2b0d6f2cb5730d377a7ae144b9f6e1a25
              CustomOrigin: Managed by pivot tool
                   Version: 410.8.20190604.0 (2019-06-04T20:37:00Z)
State: idle
AutomaticUpdates: disabled
Deployments:
* pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d0fc729885d9986f3846198a5b8dc3b2603a2fb273c4b5282c9d32ac0f84654b
              CustomOrigin: Managed by pivot tool
                   Version: 420.8.20190626.0 (2019-06-26T18:35:16Z)

  pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dd3bd07c9a4fd39d8039b137fa161c3a5b38a2855e8df9f6c16946ccdf7b3f31
              CustomOrigin: Provisioned from oscontainer
                   Version: 420.8.20190624.0 (2019-06-24T00:25:32Z)
State: idle
AutomaticUpdates: disabled
Deployments:
* pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d0fc729885d9986f3846198a5b8dc3b2603a2fb273c4b5282c9d32ac0f84654b
              CustomOrigin: Managed by pivot tool
                   Version: 420.8.20190626.0 (2019-06-26T18:35:16Z)

  pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d9c4793f2b5a279f0ee0066c474b0d2b0d6f2cb5730d377a7ae144b9f6e1a25
              CustomOrigin: Managed by pivot tool
                   Version: 410.8.20190604.0 (2019-06-04T20:37:00Z)
State: idle
AutomaticUpdates: disabled
Deployments:
* pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d0fc729885d9986f3846198a5b8dc3b2603a2fb273c4b5282c9d32ac0f84654b
              CustomOrigin: Managed by pivot tool
                   Version: 420.8.20190626.0 (2019-06-26T18:35:16Z)

  pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d9c4793f2b5a279f0ee0066c474b0d2b0d6f2cb5730d377a7ae144b9f6e1a25
              CustomOrigin: Managed by pivot tool
                   Version: 410.8.20190604.0 (2019-06-04T20:37:00Z)
State: idle
AutomaticUpdates: disabled
Deployments:
* pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d0fc729885d9986f3846198a5b8dc3b2603a2fb273c4b5282c9d32ac0f84654b
              CustomOrigin: Managed by pivot tool
                   Version: 420.8.20190626.0 (2019-06-26T18:35:16Z)

  pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d9c4793f2b5a279f0ee0066c474b0d2b0d6f2cb5730d377a7ae144b9f6e1a25
              CustomOrigin: Managed by pivot tool
                   Version: 410.8.20190604.0 (2019-06-04T20:37:00Z)

Comment 5 errata-xmlrpc 2019-10-16 06:32:41 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922