Bug 1988414

Summary: OpenShift kube-apiserver remains degraded after installing OCP 4.8.2 using single-node-developer profile
Product: OpenShift Container Platform Reporter: Ales Nosek <anosek>
Component: crcAssignee: Praveen Kumar <prkumar>
Status: CLOSED DEFERRED QA Contact: Tomáš Sedmík <tsedmik>
Severity: low Docs Contact: Kevin Owen <kowen>
Priority: low    
Version: 4.8CC: aos-bugs, cfergeau, dhellmann, gbraad, kowen, mfojtik, prkumar, sttts, tsedmik, veillard, xxia
Target Milestone: ---   
Target Release: 4.9.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-03-09 01:05:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Ales Nosek 2021-07-30 13:55:15 UTC
Description of problem:

Installed OCP 4.8.2 using the single-node-developer profile:

$ export OPENSHIFT_INSTALL_EXPERIMENTAL_CLUSTER_PROFILE=single-node-developer
$ oc create manifests
$ oc create ignition-configs

Version-Release number of selected component (if applicable):

4.8.2, link https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.8.2/

How reproducible:

Just install OpenShift 4.8.2 using the single-node-developer profile

Actual results:

The kube-apiserver operator remains in a degraded state:

$ oc get clusteroperator
NAME                                       VERSION   AVAILABLE   PROGRESSING   DEGRADED   SINCE
authentication                             4.8.2     True        False         False      23m
baremetal
cluster-autoscaler                         4.8.2     True        False         False      39m
config-operator                            4.8.2     True        False         False      40m
console                                    4.8.2     False       False         False      32m
csi-snapshot-controller                    4.8.2     True        False         False      40m
dns                                        4.8.2     True        False         False      40m
etcd                                       4.8.2     True        False         False      39m
image-registry                             4.8.2     True        False         False      33m
ingress                                    4.8.2     True        False         False      34m
insights                                   4.8.2     True        False         False      33m
kube-apiserver                             4.8.2     True        False         True       35m
kube-controller-manager                    4.8.2     True        False         False      39m
kube-scheduler                             4.8.2     True        False         False      36m
machine-api                                4.8.2     True        False         False      39m
machine-approver                           4.8.2     True        False         False      40m
machine-config                             4.8.2     True        False         False      39m
marketplace                                4.8.2     True        False         False      39m
monitoring                                 4.8.2     True        False         False      31m
network                                    4.8.2     True        False         False      41m
node-tuning                                4.8.2     True        False         False      39m
openshift-apiserver                        4.8.2     True        False         False      33m
openshift-controller-manager               4.8.2     True        False         False      39m
openshift-samples                          4.8.2     True        False         False      32m
operator-lifecycle-manager                 4.8.2     True        False         False      40m
operator-lifecycle-manager-catalog         4.8.2     True        False         False      40m
operator-lifecycle-manager-packageserver   4.8.2     True        False         False      33m
service-ca                                 4.8.2     True        False         False      40m
storage                                    4.8.2     True        False         False      40m

Expected results:

Kube-apiserver operator should not remain in a degraded state.

Additional info:

This issue is caused by the cluster-version operator not deploying
kubestorageversionmigrators.operator.openshift.io CRD.

This is the command that fixes the issue for me:

$ oc rsh -n openshift-cluster-version cluster-version-operator-789d46ff9d-jqbgt cat /release-manifests/0000_40_kube-storage-version-migrator-operator_01_storage_migration_crd.yaml | oc apply -f -

This issue is similar to https://bugzilla.redhat.com/show_bug.cgi?id=1986129

Comment 1 Stefan Schimanski 2021-08-02 12:12:17 UTC
This is CRC. We don't own this.

Comment 2 Praveen Kumar 2021-08-02 12:40:50 UTC
@Alex What is the use case you have? if you just want to have single node cluster then you don't need to export this profile but in the install-config just use 0 for compute section and 1 for control plane.

Comment 3 Christophe Fergeau 2021-08-02 13:14:05 UTC
cluster-kube-storage-version-migrator-operator was intentionally not added to the single-node-developer profile as it did not seem needed for our use case (and I'm quite sure this was ok to do in 4.7). Did something change in 4.8 which made it mandatory?

Comment 4 Ales Nosek 2021-08-02 18:55:48 UTC
(In reply to Praveen Kumar from comment #2)
> @Alex What is the use case you have? if you just want to have single node
> cluster then you don't need to export this profile but in the install-config
> just use 0 for compute section and 1 for control plane.

Praveen, I just wanted to install a single node cluster. Didn't know that it was possible to achieve this just by setting the number of masters to 1, thanks! I removed the OPENSHIFT_INSTALL_EXPERIMENTAL_CLUSTER_PROFILE=single-node-developer setting and the cluster came up nicely:

$ oc get clusteroperator
NAME                                       VERSION   AVAILABLE   PROGRESSING   DEGRADED   SINCE
authentication                             4.8.2     True        True          False      78s
baremetal                                  4.8.2     True        False         False      74m
cloud-credential                           4.8.2     True        False         False      78m
cluster-autoscaler                         4.8.2     True        False         False      75m
config-operator                            4.8.2     True        False         False      76m
console                                    4.8.2     True        False         False      2m20s
csi-snapshot-controller                    4.8.2     True        False         False      65m
dns                                        4.8.2     True        False         False      74m
etcd                                       4.8.2     True        False         False      74m
image-registry                             4.8.2     True        False         False      6m48s
ingress                                    4.8.2     True        False         False      69m
insights                                   4.8.2     True        False         False      69m
kube-apiserver                             4.8.2     True        False         False      69m
kube-controller-manager                    4.8.2     True        False         False      74m
kube-scheduler                             4.8.2     True        False         False      69m
kube-storage-version-migrator              4.8.2     True        False         False      76m
machine-api                                4.8.2     True        False         False      75m
machine-approver                           4.8.2     True        False         False      76m
machine-config                             4.8.2     True        False         False      75m
marketplace                                4.8.2     True        False         False      75m
monitoring                                 4.8.2     True        False         False      2m51s
network                                    4.8.2     True        False         False      76m
node-tuning                                4.8.2     True        False         False      76m
openshift-apiserver                        4.8.2     True        False         False      2m12s
openshift-controller-manager               4.8.2     True        False         False      75m
openshift-samples                          4.8.2     True        False         False      66m
operator-lifecycle-manager                 4.8.2     True        False         False      75m
operator-lifecycle-manager-catalog         4.8.2     True        False         False      75m
operator-lifecycle-manager-packageserver   4.8.2     True        False         False      7m25s
service-ca                                 4.8.2     True        False         False      76m
storage                                    4.8.2     True        False         False      76m

Comment 5 Shiftzilla 2023-03-09 01:05:01 UTC
OpenShift has moved to Jira for its defect tracking! This bug can now be found in the OCPBUGS project in Jira.

https://issues.redhat.com/browse/OCPBUGS-8929