Description of problem: Cluster operator image-registry can't create when launch Version-Release number of selected component (if applicable): OCP 4.5.6 How reproducible: Steps to Reproduce: Create a OCP 4.5 cluster on OpenStack disconnect environment via UPI installation. Login to the cluster and check operators status. Actual results: $ oc get clusteroperator NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE authentication 4.5.6 True False False 38d cloud-credential 4.5.6 True False False 38d cluster-autoscaler 4.5.6 True False False 38d config-operator 4.5.6 True False False 38d console 4.5.6 True False False 35d csi-snapshot-controller 4.5.6 True False False 35d dns 4.5.6 True False False 38d etcd 4.5.6 True False False 38d image-registry ingress 4.5.6 True False False 38d insights 4.5.6 True False False 38d kube-apiserver 4.5.6 True False False 38d kube-controller-manager 4.5.6 True False False 38d kube-scheduler 4.5.6 True False False 38d kube-storage-version-migrator 4.5.6 True False False 35d machine-api 4.5.6 True False False 38d machine-approver 4.5.6 True False False 38d machine-config 4.5.6 True False False 35d marketplace 4.5.6 True False False 35d monitoring 4.5.6 True False False 38d network 4.5.6 True False False 38d node-tuning 4.5.6 True False False 38d openshift-apiserver 4.5.6 True False False 38d openshift-controller-manager 4.5.6 True False False 38d openshift-samples 4.5.6 True False False 38d operator-lifecycle-manager 4.5.6 True False False 38d operator-lifecycle-manager-catalog 4.5.6 True False False 38d operator-lifecycle-manager-packageserver 4.5.6 True False False 35d service-ca 4.5.6 True False False 38d storage 4.5.6 True False False 38d Expected results: All operators in available state Additional info: Errors observed: E1020 06:54:06.631909 13 nodecadaemon.go:82] NodeCADaemonController: unable to sync: config.imageregistry.operator.openshift.io "cluster" not found, requeuing E1020 06:54:06.631932 13 clusteroperator.go:96] unable to sync ClusterOperatorStatusController: config.imageregistry.operator.openshift.io "cluster" not found, requeuing I1020 06:54:06.631978 13 caconfig.go:75] unable to get the service name to add service-ca.crt E1020 06:54:06.632063 13 imageregistrycertificates.go:94] ImageRegistryCertificatesController: unable to sync: config.imageregistry.operator.openshift.io "cluster" not found, requeuing E1020 06:54:06.636007 13 imageconfig.go:109] ImageConfigController: unable to sync: config.imageregistry.operator.openshift.io "cluster" not found, requeuing E1020 06:54:06.637069 13 nodecadaemon.go:82] NodeCADaemonController: unable to sync: config.imageregistry.operator.openshift.io "cluster" not found, requeuing E1020 06:54:06.637072 13 clusteroperator.go:96] unable to sync ClusterOperatorStatusController: config.imageregistry.operator.openshift.io "cluster" not found, requeuing I1020 06:54:06.637156 13 caconfig.go:75] unable to get the service name to add service-ca.crt E1020 06:54:06.637200 13 imageregistrycertificates.go:94] ImageRegistryCertificatesController: unable to sync: config.imageregistry.operator.openshift.io "cluster" not found, requeuing E1020 06:54:06.639683 13 controllerimagepruner.go:306] (image pruner) unable to sync: unable to apply objects: failed to create object *v1beta1.CronJob, Namespace=openshift-image-registry, Name=image-pruner: config.imageregistry.operator.openshift.io "cluster" not found, requeuing E1020 06:54:06.644046 13 imageconfig.go:109] ImageConfigController: unable to sync: config.imageregistry.operator.openshift.io "cluster" not found, requeuing E1020 06:54:06.647779 13 clusteroperator.go:96] unable to sync ClusterOperatorStatusController: config.imageregistry.operator.openshift.io "cluster" not found, requeuing I1020 06:54:06.647968 13 caconfig.go:75] unable to get the service name to add service-ca.crt E1020 06:54:06.648041 13 nodecadaemon.go:82] NodeCADaemonController: unable to sync: config.imageregistry.operator.openshift.io "cluster" not found, requeuing E1020 06:54:06.648093 13 imageregistrycertificates.go:94] ImageRegistryCertificatesController: unable to sync: config.imageregistry.operator.openshift.io "cluster" not found, requeuing E1020 06:54:06.657326 13 imageconfig.go:109] ImageConfigController: unable to sync: config.imageregistry.operator.openshift.io "cluster" not found, requeuing E1020 06:54:06.660002 13 controllerimagepruner.go:306] (image pruner) unable to sync: unable to apply objects: failed to create object *v1beta1.CronJob, Namespace=openshift-image-registry, Name=image-pruner: config.imageregistry.operator.openshift.io "cluster" not found, requeuing E1020 06:54:06.660391 13 controller.go:311] unable to sync: Failed to authenticate provider client: Post https://X.X.X.X:YYYY//v3/auth/tokens: x509: certificate signed by unknown authority, requeuing E1020 06:54:06.667932 13 clusteroperator.go:96] unable to sync ClusterOperatorStatusController: config.imageregistry.operator.openshift.io "cluster" not found, requeuing E1020 06:54:06.668198 13 nodecadaemon.go:82] NodeCADaemonController: unable to sync: config.imageregistry.operator.openshift.io "cluster" not found, requeuing I1020 06:54:06.668231 13 caconfig.go:75] unable to get the service name to add service-ca.crt
Dhruv, can you check the configmap cloud-provider-config in the openshift-config namespace? It should contain ca-bundle.pem with CA for Swift.
Hello Oleg Find details below. $ oc get cm cloud-provider-config -n openshift-config -o yaml apiVersion: v1 data: config: | [Global] secret-name = openstack-credentials secret-namespace = kube-system region = regionOne kind: ConfigMap metadata: creationTimestamp: "2020-09-09T11:51:56Z" managedFields: - apiVersion: v1 fieldsType: FieldsV1 fieldsV1: f:data: .: {} f:config: {} manager: cluster-bootstrap operation: Update time: "2020-09-09T11:51:56Z" name: cloud-provider-config namespace: openshift-config resourceVersion: "1098" selfLink: /api/v1/namespaces/openshift-config/configmaps/cloud-provider-config uid: ca5c3c71-f83b-40d3-8d42-e614bbd0e4ab $ oc get cm -n openshift-config NAME DATA AGE admin-kubeconfig-client-ca 1 41d cloud-provider-config 1 41d custom-ca 1 40d etcd-ca-bundle 1 41d etcd-metric-serving-ca 1 41d etcd-serving-ca 1 41d initial-etcd-ca 1 41d initial-kube-apiserver-server-ca 1 41d openshift-install-manifests 2 41d user-ca-bundle 1 41d
Ok, so they don't have it there. Do they have `cacert` in their `clouds.yaml` file? It's the recommended way to provide the CA bundle for OpenStack if it uses self-signed certificates [1]. [1]: https://github.com/openshift/installer/tree/master/docs/user/openstack#self-signed-openstack-ca-certificates
Edit cloud-provider-config manually and add something like this: data: ca-bundle.pem: | -----BEGIN CERTIFICATE ----- ... The operator should notice this field and continue with it. Please let me know if it helps.