Description of problem: I am upgrading AWS cluster from 4.1 to 4.6 but seeing a panic in machine-config-operator when upgrading to 4.6 Version-Release number of selected component (if applicable): 4.6.0-0.nightly-2020-07-25-091217 How reproducible: 2/2 Steps to Reproduce: 1.Upgrade IPI AWS cluster from 4.1 to 4.6, like this: 4.1.0-0.nightly-2020-05-28-040321, 4.2.0-0.nightly-2020-07-18-024525, 4.3.0-0.nightly-2020-07-22-001315, 4.4.0-0.nightly-2020-07-22-132117, 4.5.0-0.nightly-2020-07-22-163134, 4.6.0-0.nightly-2020-07-25-091217 2. 3. Actual results: Upgrading to 4.6 failed and panic is in MCO version 4.5.0-0.nightly-2020-07-22-163134 True True 118m Unable to apply 4.6.0-0.nightly-2020-07-25-091217: the cluster operator machine-config has not yet successfully rolled out # oc logs machine-config-operator-67fdf89c85-q84kn -n openshift-machine-config-operator I0728 01:51:52.463753 1 start.go:46] Version: 4.6.0-0.nightly-2020-07-25-091217 (Raw: v4.6.0-202007250017.p0-dirty, Hash: ab2673986646c62bc6599931d22f050ffd5871db) I0728 01:51:52.466810 1 leaderelection.go:242] attempting to acquire leader lease openshift-machine-config-operator/machine-config... I0728 01:53:50.286343 1 leaderelection.go:252] successfully acquired lease openshift-machine-config-operator/machine-config I0728 01:53:50.915868 1 operator.go:270] Starting MachineConfigOperator E0728 01:53:53.093372 1 runtime.go:78] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference) goroutine 224 [running]: k8s.io/apimachinery/pkg/util/runtime.logPanic(0x1814240, 0x2a35120) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:74 +0xa3 k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:48 +0x82 panic(0x1814240, 0x2a35120) /opt/rh/go-toolset-1.14/root/usr/lib/go-toolset-1.14-golang/src/runtime/panic.go:969 +0x166 github.com/openshift/machine-config-operator/lib/resourcemerge.ensureControllerConfigSpec(0xc002c5980f, 0xc00003c618, 0xc0029f3d20, 0xb, 0x0, 0x0, 0xc0029f3e48, 0x3, 0xc002c329c0, 0x29, ...) /go/src/github.com/openshift/machine-config-operator/lib/resourcemerge/machineconfig.go:79 +0x19f github.com/openshift/machine-config-operator/lib/resourcemerge.EnsureControllerConfig(0xc002c5980f, 0xc00003c500, 0x16c2301, 0x10, 0xc002c32a50, 0x24, 0xc002c1d4e0, 0x19, 0x0, 0x0, ...) /go/src/github.com/openshift/machine-config-operator/lib/resourcemerge/machineconfig.go:19 +0xd4 github.com/openshift/machine-config-operator/lib/resourceapply.ApplyControllerConfig(0x7f9b99923210, 0xc000140880, 0xc00003c280, 0x7f9b99923210, 0xc000140880, 0x5ac9, 0x5b42) /go/src/github.com/openshift/machine-config-operator/lib/resourceapply/machineconfig.go:67 +0x185 github.com/openshift/machine-config-operator/pkg/operator.(*Operator).syncMachineConfigController(0xc000354000, 0xc000a26000, 0xc0021cc80e, 0x29bfde5178e6) /go/src/github.com/openshift/machine-config-operator/pkg/operator/sync.go:468 +0x438 github.com/openshift/machine-config-operator/pkg/operator.(*Operator).syncAll(0xc000354000, 0xc002c59ca8, 0x6, 0x6, 0xc0007b1c01, 0x413893) /go/src/github.com/openshift/machine-config-operator/pkg/operator/sync.go:69 +0x177 github.com/openshift/machine-config-operator/pkg/operator.(*Operator).sync(0xc000354000, 0xc0001548a0, 0x30, 0x0, 0x0) /go/src/github.com/openshift/machine-config-operator/pkg/operator/operator.go:362 +0x40a github.com/openshift/machine-config-operator/pkg/operator.(*Operator).processNextWorkItem(0xc000354000, 0x203000) /go/src/github.com/openshift/machine-config-operator/pkg/operator/operator.go:318 +0xd2 github.com/openshift/machine-config-operator/pkg/operator.(*Operator).worker(0xc000354000) /go/src/github.com/openshift/machine-config-operator/pkg/operator/operator.go:307 +0x2b k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc0007f20a0) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155 +0x5f k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc0007f20a0, 0x1cbf1c0, 0xc000912180, 0xc000908001, 0xc0004d00c0) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156 +0xa3 k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc0007f20a0, 0x3b9aca00, 0x0, 0x1, 0xc0004d00c0) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0xe2 k8s.io/apimachinery/pkg/util/wait.Until(0xc0007f20a0, 0x3b9aca00, 0xc0004d00c0) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:90 +0x4d created by github.com/openshift/machine-config-operator/pkg/operator.(*Operator).Run /go/src/github.com/openshift/machine-config-operator/pkg/operator/operator.go:276 +0x3dc panic: runtime error: invalid memory address or nil pointer dereference [recovered] panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x1a8 pc=0x13aac6f] goroutine 224 [running]: k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:55 +0x105 panic(0x1814240, 0x2a35120) /opt/rh/go-toolset-1.14/root/usr/lib/go-toolset-1.14-golang/src/runtime/panic.go:969 +0x166 github.com/openshift/machine-config-operator/lib/resourcemerge.ensureControllerConfigSpec(0xc002c5980f, 0xc00003c618, 0xc0029f3d20, 0xb, 0x0, 0x0, 0xc0029f3e48, 0x3, 0xc002c329c0, 0x29, ...) /go/src/github.com/openshift/machine-config-operator/lib/resourcemerge/machineconfig.go:79 +0x19f github.com/openshift/machine-config-operator/lib/resourcemerge.EnsureControllerConfig(0xc002c5980f, 0xc00003c500, 0x16c2301, 0x10, 0xc002c32a50, 0x24, 0xc002c1d4e0, 0x19, 0x0, 0x0, ...) /go/src/github.com/openshift/machine-config-operator/lib/resourcemerge/machineconfig.go:19 +0xd4 github.com/openshift/machine-config-operator/lib/resourceapply.ApplyControllerConfig(0x7f9b99923210, 0xc000140880, 0xc00003c280, 0x7f9b99923210, 0xc000140880, 0x5ac9, 0x5b42) /go/src/github.com/openshift/machine-config-operator/lib/resourceapply/machineconfig.go:67 +0x185 github.com/openshift/machine-config-operator/pkg/operator.(*Operator).syncMachineConfigController(0xc000354000, 0xc000a26000, 0xc0021cc80e, 0x29bfde5178e6) /go/src/github.com/openshift/machine-config-operator/pkg/operator/sync.go:468 +0x438 github.com/openshift/machine-config-operator/pkg/operator.(*Operator).syncAll(0xc000354000, 0xc002c59ca8, 0x6, 0x6, 0xc0007b1c01, 0x413893) /go/src/github.com/openshift/machine-config-operator/pkg/operator/sync.go:69 +0x177 github.com/openshift/machine-config-operator/pkg/operator.(*Operator).sync(0xc000354000, 0xc0001548a0, 0x30, 0x0, 0x0) /go/src/github.com/openshift/machine-config-operator/pkg/operator/operator.go:362 +0x40a github.com/openshift/machine-config-operator/pkg/operator.(*Operator).processNextWorkItem(0xc000354000, 0x203000) /go/src/github.com/openshift/machine-config-operator/pkg/operator/operator.go:318 +0xd2 github.com/openshift/machine-config-operator/pkg/operator.(*Operator).worker(0xc000354000) /go/src/github.com/openshift/machine-config-operator/pkg/operator/operator.go:307 +0x2b k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc0007f20a0) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155 +0x5f k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc0007f20a0, 0x1cbf1c0, 0xc000912180, 0xc000908001, 0xc0004d00c0) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156 +0xa3 k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc0007f20a0, 0x3b9aca00, 0x0, 0x1, 0xc0004d00c0) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0xe2 k8s.io/apimachinery/pkg/util/wait.Until(0xc0007f20a0, 0x3b9aca00, 0xc0004d00c0) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:90 +0x4d created by github.com/openshift/machine-config-operator/pkg/operator.(*Operator).Run /go/src/github.com/openshift/machine-config-operator/pkg/operator/operator.go:276 +0x3dc Expected results: Upgrade is successful. Additional info:
Must gather tarball is online. https://drive.google.com/file/d/1NhdY8_tz8aNgZCJUYxc5Kz_TJFrhprkB/view?usp=sharing
4.6.0-0.nightly-2020-07-25-091217 is old, and since then things like [1] have landed. Although from the attached must-gather, this isn't a vSphere cluster: $ tar -xOz must-gather.local.3184524102915563670/quay-io-openshift-release-dev-ocp-v4-0-art-dev-sha256-611f823bcbd1b627ba7c3b4f558094dd08b4625567a49d56e8687295ae454d26/cluster-scoped-resources/config.openshift.io/infrastructures/cluster.yaml <must-gather.tar.gz | grep -A5 platform platform: AWS platformStatus: aws: region: us-east-2 type: AWS So I'm not sure what's going on there. [1]: https://github.com/openshift/machine-config-operator/pull/1951
This particular panic was already fixed (line 79 in machineconfig.go) https://github.com/openshift/machine-config-operator/pull/1935/files @yangyang since that was an old nightly (that doesn't exist anymore) can you verify that you still see this in your 4.1->...-> 4.6 upgrades using a recent nightly? If not, I will dupe this to the old closed BZ.
I still can see the bug with 4.6.0-0.nightly-2020-08-18-055142 which has some must-gather info here: https://bugzilla.redhat.com/show_bug.cgi?id=1866554#c12
@Wenjing that must gather doesn't have any MCO logs..
I still found this issue recently when upgrading from 4.1.0-0.nightly-2020-07-29-210856 -> 4.2.0-0.nightly-2020-08-06-223716 -> 4.3.0-0.nightly-2020-08-10-122110 -> 4.4.0-0.nightly-2020-08-10-180247 -> 4.5.0-0.nightly-2020-08-10-150345 -> 4.6.0-0.nightly-2020-08-10-150008 It's different from bz1858026. In this issue, mco upgrades to 4.5 successfully but upgrade to 4.6 failed. NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.5.0-0.nightly-2020-08-10-150345 True True 119m Unable to apply 4.6.0-0.nightly-2020-08-10-150008: the cluster operator machine-config has not yet successfully rolled out https://mastern-jenkins-csb-openshift-qe.cloud.paas.psi.redhat.com/job/upgrade_CI/4281/console
How often are you seeing this? Have you reproduced this in any of the recent nightlies available from https://openshift-release.apps.ci.l2s4.p1.openshiftapps.com/#4.6.0-0.nightly
I'm seeing: 2020-08-11T03:08:25.484223874Z I0811 03:08:25.484186 1 kubelet_config_controller.go:318] Error syncing kubeletconfig cluster: GenerateMachineConfigsforRole failed with error cannot generate MachineConfigs when no platformStatus.type is set 2020-08-11T03:08:25.773143916Z I0811 03:08:25.773102 1 template_controller.go:366] Error syncing controllerconfig machine-config-controller: failed to create MachineConfig for role master: cannot generate MachineConfigs when no platformStatus.type is set 2020-08-11T03:08:25.806605111Z I0811 03:08:25.806563 1 container_runtime_config_controller.go:369] Error syncing image config openshift-config: could not Create/Update MachineConfig: could not generate origin ContainerRuntime Configs: generateMachineConfigsforRole failed with error cannot generate MachineConfigs when no platformStatus.type is set 2020-08-11T03:08:26.099569478Z I0811 03:08:26.099504 1 kubelet_config_controller.go:318] Error syncing kubeletconfig cluster: GenerateMachineConfigsforRole failed with error cannot generate MachineConfigs when no platformStatus.type is set But I'm also seeing: platformStatus: aws: region: us-east-2 type: AWS
It's reproducible when upgrading from 4.1.38-x86_64 -> 4.2.36-x86_64 -> 4.3.33-x86_64 -> 4.4.17-x86_64 -> 4.5.6-x86_64 -> 4.6.0-0.nightly-2020-08-18-165040. https://mastern-jenkins-csb-openshift-qe.cloud.paas.psi.redhat.com/job/upgrade_CI/4403/console
*** Bug 1870548 has been marked as a duplicate of this bug. ***
This is caused by our ControllerConfig not syncing the infra object coming from 4.1 unfortunately (
Antonio, I'm trying to decode your last comment. Do we understand the problem? Do we know what a fix might look like?
@Nick Yes to both: we understand the problem and understand the path a fix might take.
Related: https://bugzilla.redhat.com/show_bug.cgi?id=1812179
Verified with 4.6.0-0.nightly-2020-09-19-004228 (6+ hrs to go from 4.1 -> 4.6!) ``` $ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.1.41 True False 41m Cluster version is 4.1.41 $ oc adm upgrade --allow-explicit-upgrade=true --allow-upgrade-with-warnings=true --force=true --to-image=quay.io/openshift-release-dev/ocp-release:4.2.36-x86_64 warning: Using by-tag pull specs is dangerous, and while we still allow it in combination with --force for backward compatibility, it would be much safer to pass a by-digest pull spec instead warning: The requested upgrade image is not one of the available updates. You have used --allow-explicit-upgrade to the update to preceed anyway warning: --force overrides cluster verification of your supplied release image and waives any update precondition failures. Updating to release image quay.io/openshift-release-dev/ocp-release:4.2.36-x86_64 $ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.2.36 True False 6s Cluster version is 4.2.36 $ oc adm upgrade --allow-explicit-upgrade=true --allow-upgrade-with-warnings=true --force=true --to-image=quay.io/openshift-release-dev/ocp-release:4.3.35-x86_64 warning: Using by-tag pull specs is dangerous, and while we still allow it in combination with --force for backward compatibility, it would be much safer to pass a by-digest pull spec instead warning: The requested upgrade image is not one of the available updates. You have used --allow-explicit-upgrade to the update to preceed anyway warning: --force overrides cluster verification of your supplied release image and waives any update precondition failures. Updating to release image quay.io/openshift-release-dev/ocp-release:4.3.35-x86_64 $ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.3.35 True False 116s Cluster version is 4.3.35 $ oc adm upgrade --allow-explicit-upgrade=true --allow-upgrade-with-warnings=true --force=true --to-image=quay.io/openshift-release-dev/ocp-release:4.4.23-x86_64 warning: Using by-tag pull specs is dangerous, and while we still allow it in combination with --force for backward compatibility, it would be much safer to pass a by-digest pull spec instead warning: The requested upgrade image is not one of the available updates. You have used --allow-explicit-upgrade to the update to preceed anyway warning: --force overrides cluster verification of your supplied release image and waives any update precondition failures. Updating to release image quay.io/openshift-release-dev/ocp-release:4.4.23-x86_64 $ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.4.23 True False 25m Cluster version is 4.4.23 $ oc adm upgrade --allow-explicit-upgrade=true --allow-upgrade-with-warnings=true --force=true --to-image=quay.io/openshift-release-dev/ocp-release:4.5.10-x86_64 warning: Using by-tag pull specs is dangerous, and while we still allow it in combination with --force for backward compatibility, it would be much safer to pass a by-digest pull spec instead warning: The requested upgrade image is not one of the available updates. You have used --allow-explicit-upgrade to the update to preceed anyway warning: --force overrides cluster verification of your supplied release image and waives any update precondition failures. Updating to release image quay.io/openshift-release-dev/ocp-release:4.5.10-x86_64 $ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.5.10 True False 17m Cluster version is 4.5.10 $ oc adm upgrade --allow-explicit-upgrade=true --allow-upgrade-with-warnings=true --force=true --to-image=registry.svc.ci.openshift.org/ocp/release@sha256:2f6222aecdfe27eae59131d1e698e06e48bb8bcefec336a39c4d2cd761a621ac warning: The requested upgrade image is not one of the available updates. You have used --allow-explicit-upgrade to the update to preceed anyway warning: --force overrides cluster verification of your supplied release image and waives any update precondition failures. Updating to release image registry.svc.ci.openshift.org/ocp/release@sha256:2f6222aecdfe27eae59131d1e698e06e48bb8bcefec336a39c4d2cd761a621ac $ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.6.0-0.nightly-2020-09-19-004228 True False 34s Cluster version is 4.6.0-0.nightly-2020-09-19-004228 $ oc describe co/machine-config Name: machine-config Namespace: Labels: <none> Annotations: <none> API Version: config.openshift.io/v1 Kind: ClusterOperator Metadata: Creation Timestamp: 2020-09-19T15:50:31Z Generation: 1 Resource Version: 230034 Self Link: /apis/config.openshift.io/v1/clusteroperators/machine-config UID: d938ac7b-fa8f-11ea-97bf-022c5ecfda63 Spec: Status: Conditions: Last Transition Time: 2020-09-19T21:55:43Z Message: Cluster has deployed 4.6.0-0.nightly-2020-09-19-004228 Status: True Type: Available Last Transition Time: 2020-09-19T21:55:43Z Message: Cluster version is 4.6.0-0.nightly-2020-09-19-004228 Status: False Type: Progressing Last Transition Time: 2020-09-19T21:31:53Z Status: False Type: Degraded Last Transition Time: 2020-09-19T17:27:43Z Reason: AsExpected Status: True Type: Upgradeable Extension: Master: all 3 nodes are at latest configuration rendered-master-ea3d50e6005eb46bd31d38c7af0e1890 Worker: all 3 nodes are at latest configuration rendered-worker-89b4196319e54095cad4b003daa21e96 Related Objects: Group: Name: openshift-machine-config-operator Resource: namespaces Group: machineconfiguration.openshift.io Name: Resource: machineconfigpools Group: machineconfiguration.openshift.io Name: Resource: controllerconfigs Group: machineconfiguration.openshift.io Name: Resource: kubeletconfigs Group: machineconfiguration.openshift.io Name: Resource: containerruntimeconfigs Group: machineconfiguration.openshift.io Name: Resource: machineconfigs Group: Name: Resource: nodes Versions: Name: operator Version: 4.6.0-0.nightly-2020-09-19-004228 Events: <none> $ oc get nodes NAME STATUS ROLES AGE VERSION ip-10-0-129-165.us-west-2.compute.internal Ready master 6h9m v1.19.0+7f9e863 ip-10-0-135-149.us-west-2.compute.internal Ready worker 6h4m v1.19.0+7f9e863 ip-10-0-147-43.us-west-2.compute.internal Ready worker 6h3m v1.19.0+7f9e863 ip-10-0-154-82.us-west-2.compute.internal Ready master 6h10m v1.19.0+7f9e863 ip-10-0-171-208.us-west-2.compute.internal Ready master 6h10m v1.19.0+7f9e863 ip-10-0-173-169.us-west-2.compute.internal Ready worker 6h3m v1.19.0+7f9e863 ```
``` $ oc -n openshift-machine-config-operator logs machine-config-operator-7b58dc6cff-fkvbd I0919 21:46:37.540011 1 start.go:43] Version: 4.6.0-0.nightly-2020-09-19-004228 (Raw: v4.6.0-202009181332.p0-dirty, Hash: c08c048584ef0bf18ab2dd88fdddd93279e1c6a1) I0919 21:46:37.543321 1 leaderelection.go:243] attempting to acquire leader lease openshift-machine-config-operator/machine-config... I0919 21:48:33.252997 1 leaderelection.go:253] successfully acquired lease openshift-machine-config-operator/machine-config W0919 21:48:33.366368 1 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition W0919 21:48:33.376676 1 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition W0919 21:48:33.406841 1 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition I0919 21:48:33.780030 1 operator.go:253] Starting MachineConfigOperator W0919 21:48:33.803280 1 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition W0919 21:54:40.888815 1 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition I0919 21:55:43.170205 1 event.go:282] Event(v1.ObjectReference{Kind:"", Namespace:"", Name:"machine-config", UID:"d938ac7b-fa8f-11ea-97bf-022c5ecfda63", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorVersionChanged' clusteroperator/machine-config-operator version changed from [{operator 4.5.10}] to [{operator 4.6.0-0.nightly-2020-09-19-004228}] W0919 21:55:43.972427 1 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition W0919 21:56:05.332981 1 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition W0919 21:57:25.219700 1 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition W0919 21:57:43.308575 1 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition W0919 21:58:13.600420 1 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition W0919 21:58:20.218655 1 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition W0919 22:00:19.114058 1 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition W0919 22:00:20.570039 1 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition W0919 22:00:20.814491 1 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition W0919 22:00:25.672157 1 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition W0919 22:00:32.262745 1 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition ```
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:4196
Removing UpgradeBlocker from this older bug, to remove it from the suspect queue described in [1]. If you feel like this bug still needs to be a suspect, please add keyword again. [1]: https://github.com/openshift/enhancements/pull/475