This is a regression, we'll work on it.
Possibly a regression from https://github.com/openshift/machine-config-operator/pull/1766 though we do have the proxy config stuff in the unit: {{if .Proxy -}}
(In reply to Colin Walters from comment #6) > Possibly a regression from > https://github.com/openshift/machine-config-operator/pull/1766 > though we do have the proxy config stuff in the unit: > {{if .Proxy -}} I think changing the unit name here https://github.com/openshift/machine-config-operator/pull/1766/files#diff-95e83e4216073d5ba6d128c764d05756R323 is what makes it _not_ use the proxy variables. Could that be? I don't see anything that specifies proxy for "mco-pivot"
> I think changing the unit name here https://github.com/openshift/machine-config-operator/pull/1766/files#diff-95e83e4216073d5ba6d128c764d05756R323 is what makes it _not_ use the proxy variables. Could that be? I don't see anything that specifies proxy for "mco-pivot" Ah yes, sounds like the bug indeed! Hmm. I guess we could ship a static unit for this instead of using `systemd-run`. Or we could inject the proxy variables. The annoying thing is that `systemd-run` has clearer semantics here than blocking on `systemctl start` on a statically defined unit (e.g. what happens if the MCD pod dies in the middle of an update). In the end we probably want to use the systemd DBus API via bindings, it's just a lot more code.
Verified on 4.6.0-0.nightly-2020-09-17-113547. Created 4.5.1 cluster with same profile (UPI on Azure with http_proxy, fips, etcd_encryption) with which I was able to reproduce this issue. It got successfully upgraded to 4.6.0-0.nightly-2020-09-17-113547 $ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.5.1 True False 59m Cluster version is 4.5.1 $ oc get nodes NAME STATUS ROLES AGE VERSION sunilc-bz-09171420-master-0 Ready master 84m v1.18.3+6025c28 sunilc-bz-09171420-master-1 Ready master 83m v1.18.3+6025c28 sunilc-bz-09171420-master-2 Ready master 84m v1.18.3+6025c28 sunilc-bz-09171420-worker-centralus-1 Ready worker 68m v1.18.3+6025c28 sunilc-bz-09171420-worker-centralus-2 Ready worker 67m v1.18.3+6025c28 sunilc-bz-09171420-worker-centralus-3 Ready worker 68m v1.18.3+6025c28 $ oc adm upgrade --to-image=registry.svc.ci.openshift.org/ocp/release:4.6.0-0.nightly-2020-09-17-113547 --force --allow-explicit-upgrade warning: Using by-tag pull specs is dangerous, and while we still allow it in combination with --force for backward compatibility, it would be much safer to pass a by-digest pull spec instead warning: The requested upgrade image is not one of the available updates. You have used --allow-explicit-upgrade to the update to preceed anyway warning: --force overrides cluster verification of your supplied release image and waives any update precondition failures. Updating to release image registry.svc.ci.openshift.org/ocp/release:4.6.0-0.nightly-2020-09-17-113547 $ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.6.0-0.nightly-2020-09-17-113547 True False 2m49s Cluster version is 4.6.0-0.nightly-2020-09-17-113547 $ oc get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME sunilc-bz-09171420-master-0 Ready master 159m v1.19.0+b4ffb45 10.0.0.6 <none> Red Hat Enterprise Linux CoreOS 46.82.202009170240-0 (Ootpa) 4.18.0-193.19.1.el8_2.x86_64 cri-o://1.19.0-18.rhaos4.6.gitd802e19.el8 sunilc-bz-09171420-master-1 Ready master 159m v1.19.0+b4ffb45 10.0.0.8 <none> Red Hat Enterprise Linux CoreOS 46.82.202009170240-0 (Ootpa) 4.18.0-193.19.1.el8_2.x86_64 cri-o://1.19.0-18.rhaos4.6.gitd802e19.el8 sunilc-bz-09171420-master-2 Ready master 159m v1.19.0+b4ffb45 10.0.0.7 <none> Red Hat Enterprise Linux CoreOS 46.82.202009170240-0 (Ootpa) 4.18.0-193.19.1.el8_2.x86_64 cri-o://1.19.0-18.rhaos4.6.gitd802e19.el8 sunilc-bz-09171420-worker-centralus-1 Ready worker 143m v1.19.0+b4ffb45 10.0.1.5 <none> Red Hat Enterprise Linux CoreOS 46.82.202009170240-0 (Ootpa) 4.18.0-193.19.1.el8_2.x86_64 cri-o://1.19.0-18.rhaos4.6.gitd802e19.el8 sunilc-bz-09171420-worker-centralus-2 Ready worker 142m v1.19.0+b4ffb45 10.0.1.6 <none> Red Hat Enterprise Linux CoreOS 46.82.202009170240-0 (Ootpa) 4.18.0-193.19.1.el8_2.x86_64 cri-o://1.19.0-18.rhaos4.6.gitd802e19.el8 sunilc-bz-09171420-worker-centralus-3 Ready worker 143m v1.19.0+b4ffb45 10.0.1.4 <none> Red Hat Enterprise Linux CoreOS 46.82.202009170240-0 (Ootpa) 4.18.0-193.19.1.el8_2.x86_64 cri-o://1.19.0-18.rhaos4.6.gitd802e19.el8 $ oc describe clusterversion Name: version Namespace: Labels: <none> Annotations: <none> API Version: config.openshift.io/v1 Kind: ClusterVersion Metadata: Creation Timestamp: 2020-09-17T14:46:48Z Generation: 2 Managed Fields: API Version: config.openshift.io/v1 Fields Type: FieldsV1 fieldsV1: f:spec: .: f:channel: f:clusterID: f:upstream: Manager: cluster-bootstrap Operation: Update Time: 2020-09-17T14:46:48Z API Version: config.openshift.io/v1 Fields Type: FieldsV1 fieldsV1: f:spec: f:desiredUpdate: .: f:force: f:image: f:version: Manager: oc Operation: Update Time: 2020-09-17T16:16:36Z API Version: config.openshift.io/v1 Fields Type: FieldsV1 fieldsV1: f:status: .: f:availableUpdates: f:conditions: f:desired: .: f:force: f:image: f:version: f:history: f:observedGeneration: f:versionHash: Manager: cluster-version-operator Operation: Update Time: 2020-09-17T17:26:08Z Resource Version: 146814 Self Link: /apis/config.openshift.io/v1/clusterversions/version UID: fa1053d2-b077-43ce-acfc-f67b65428b90 Spec: Channel: stable-4.5 Cluster ID: 00548598-816a-4846-860b-257c6a811102 Desired Update: Force: true Image: registry.svc.ci.openshift.org/ocp/release:4.6.0-0.nightly-2020-09-17-113547 Version: Upstream: https://api.openshift.com/api/upgrades_info/v1/graph Status: Available Updates: <nil> Conditions: Last Transition Time: 2020-09-17T15:15:30Z Message: Done applying 4.6.0-0.nightly-2020-09-17-113547 Status: True Type: Available Last Transition Time: 2020-09-17T17:21:22Z Status: False Type: Failing Last Transition Time: 2020-09-17T17:26:08Z Message: Cluster version is 4.6.0-0.nightly-2020-09-17-113547 Status: False Type: Progressing Last Transition Time: 2020-09-17T16:16:59Z Message: Unable to retrieve available updates: currently reconciling cluster version 4.6.0-0.nightly-2020-09-17-113547 not found in the "stable-4.5" channel Reason: VersionNotFound Status: False Type: RetrievedUpdates Last Transition Time: 2020-09-17T15:33:06Z Message: Cluster operator marketplace cannot be upgraded between minor versions: The cluster has custom OperatorSource, which is deprecated in future versions. Please visit this link for further details: https://docs.openshift.com/container-platform/4.4/release_notes/ocp-4-4-release-notes.html#ocp-4-4-marketplace-apis-deprecated Reason: DeprecatedAPIsInUse Status: False Type: Upgradeable Desired: Image: registry.svc.ci.openshift.org/ocp/release:4.6.0-0.nightly-2020-09-17-113547 Version: 4.6.0-0.nightly-2020-09-17-113547 History: Completion Time: 2020-09-17T17:26:08Z Image: registry.svc.ci.openshift.org/ocp/release:4.6.0-0.nightly-2020-09-17-113547 Started Time: 2020-09-17T16:16:45Z State: Completed Verified: false Version: 4.6.0-0.nightly-2020-09-17-113547 Completion Time: 2020-09-17T15:15:30Z Image: quay.io/openshift-release-dev/ocp-release@sha256:a656048696e79a30f0536f5acd5a1e8ec5ae331d4c7d21ca62bc8de412c79dc4 Started Time: 2020-09-17T14:46:52Z State: Completed Verified: false Version: 4.5.1 Observed Generation: 2 Version Hash: HIbzcE_ZWqc= Events: <none>
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:4196
Removing UpgradeBlocker from this older bug, to remove it from the suspect queue described in [1]. If you feel like this bug still needs to be a suspect, please add keyword again. [1]: https://github.com/openshift/enhancements/pull/475