Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1825948

Summary:	Read cloudconfig from openshift-managed-config/kube-cloud-config configmap
Product:	OpenShift Container Platform	Reporter:	Sinny Kumari <skumari>
Component:	Machine Config Operator	Assignee:	Sinny Kumari <skumari>
Status:	CLOSED ERRATA	QA Contact:	Michael Nguyen <mnguyen>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	4.5	CC:	adahiya, amurdaca, jerzhang, jima, smilner, yanyang
Target Milestone:	---
Target Release:	4.5.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-07-13 17:29:07 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Sinny Kumari 2020-04-20 14:51:52 UTC

Description of problem:
With recent changes in 4.5 (https://github.com/openshift/api/pull/621 ) OpenShift api, all clients are supposed to read cloud config from cloud.conf present in openshift-managed-config/kube-cloud-config configmap. MCO uses cloud.conf key to configure the kubelet. So, MCO should adapt to this new change.

Version-Release number of selected component (if applicable): 4.5+


Expected results: MCO should be able to launch cluster on all supported platform without any error and use cloud.conf on the platforms which supports cloud config.

Comment 9 Michael Nguyen 2020-05-06 22:59:23 UTC

$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.5.0-0.nightly-2020-05-05-205255   True        False         21m     Cluster version is 4.5.0-0.nightly-2020-05-05-205255
$ oc get node
NAME                                                    STATUS   ROLES    AGE   VERSION
mnguye-7vjph-m-0.c.openshift-gce-devel.internal         Ready    master   51m   v1.18.0-rc.1
mnguye-7vjph-m-1.c.openshift-gce-devel.internal         Ready    master   51m   v1.18.0-rc.1
mnguye-7vjph-m-2.c.openshift-gce-devel.internal         Ready    master   51m   v1.18.0-rc.1
mnguye-7vjph-w-a-jl2m4.c.openshift-gce-devel.internal   Ready    worker   35m   v1.18.0-rc.1
mnguye-7vjph-w-b-lh5ml.c.openshift-gce-devel.internal   Ready    worker   35m   v1.18.0-rc.1
mnguye-7vjph-w-c-pkh7w.c.openshift-gce-devel.internal   Ready    worker   35m   v1.18.0-rc.1
$ oc projects | grep managed
openshift-config-managed
$ oc -n openshift-config-managed get cm
NAME                                                  DATA   AGE
bound-sa-token-signing-certs                          1      54m
console-public                                        1      51m
csr-controller-ca                                     1      54m
default-ingress-cert                                  1      44m
grafana-dashboard-cluster-total                       1      27m
grafana-dashboard-etcd                                1      27m
grafana-dashboard-k8s-resources-cluster               1      27m
grafana-dashboard-k8s-resources-namespace             1      26m
grafana-dashboard-k8s-resources-node                  1      26m
grafana-dashboard-k8s-resources-pod                   1      26m
grafana-dashboard-k8s-resources-workload              1      26m
grafana-dashboard-k8s-resources-workloads-namespace   1      26m
grafana-dashboard-node-cluster-rsrc-use               1      26m
grafana-dashboard-node-rsrc-use                       1      26m
grafana-dashboard-prometheus                          1      26m
kube-apiserver-aggregator-client-ca                   1      54m
kube-apiserver-client-ca                              1      54m
kube-apiserver-server-ca                              1      54m
kube-cloud-config                                     1      43m
kubelet-bootstrap-kubeconfig                          1      58m
kubelet-serving-ca                                    1      54m
monitoring-shared-config                              4      34m
oauth-openshift                                       1      39m
release-verification                                  3      27m
sa-token-signing-certs                                2      57m
service-ca                                            1      54m
signatures-managed                                    0      52m
trusted-ca-bundle                                     1      56m
$ oc -n openshift-config-managed describe cm/kube-cloud-config
Name:         kube-cloud-config
Namespace:    openshift-config-managed
Labels:       <none>
Annotations:  <none>

Data
====
cloud.conf:
----
[global]
project-id      = openshift-gce-devel
regional        = true
multizone       = true
node-tags       = mnguye-7vjph-master
node-tags       = mnguye-7vjph-worker
node-instance-prefix = mnguye-7vjph
external-instance-groups-prefix = mnguye-7vjph
subnetwork-name = mnguye-7vjph-worker-subnet


Events:  <none>
$ oc get controllerconfig
NAME                        AGE
machine-config-controller   4h22m
$ oc get controllerconfig/machine-config-controller -o yaml
apiVersion: machineconfiguration.openshift.io/v1
kind: ControllerConfig
metadata:
  creationTimestamp: "2020-05-06T18:14:49Z"
  generation: 1
  name: machine-config-controller
  resourceVersion: "15250"
  selfLink: /apis/machineconfiguration.openshift.io/v1/controllerconfigs/machine-config-controller
  uid: bc27f9b5-5682-4c85-8f8a-18e696299562
spec:
  additionalTrustBundle: null
  cloudProviderCAData: null
  cloudProviderConfig: |+
    [global]
    project-id      = openshift-gce-devel
    regional        = true
    multizone       = true
    node-tags       = mnguye-7vjph-master
    node-tags       = mnguye-7vjph-worker
    node-instance-prefix = mnguye-7vjph
    external-instance-groups-prefix = mnguye-7vjph
    subnetwork-name = mnguye-7vjph-worker-subnet

  clusterDNSIP: 172.30.0.10
  etcdCAData: --SNIP--
  etcdDiscoveryDomain: mnguyen45.gcp.devcluster.openshift.com
  etcdMetricCAData: --SNIP--
  images:
    baremetalRuntimeCfgImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1cd6dfed1c2178e4811b02c9a735cae09898d8ea66abbfebf50de715fa343a1
    clusterEtcdOperatorImageKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:418c8fe15f2b9391d07b49a730264a396bc2979a37ed8ed0641542d041a00eaa
    corednsImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15a47f1b950f3c8ae8446e02fd5f98813d5664233361f9f04fc1e9164793e04a
    etcdKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:01106cb6d8c2825f5ccd96038be81bc444adf5385cdb27cdd79295197d4c81d3
    gcpRoutesControllerKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b2d56eee850b10df07bc273bd0061c9a6bdba86d2c2e3ccbaaeb3dea97dd5475
    haproxyImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:31e518340773eda69f7510ab0eeb64f2121b583f58c71d9621c899826895622e
    infraImageKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d54892b888e0839a42c00fec023fd9ac80ea02d7214c59609688bdc2064457d2
    keepalivedImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:100f9518f67b20260cb75a5d2e1f2d3d44e0346bb366410fa70c0d35229731ac
    kubeClientAgentImageKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:14420911c35d5d2939298d7c1f545b9a2d1c05890e0991afc7aabfc80dc1d3ea
    mdnsPublisherImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32c7aeaec4d965606fd7dfec34ed3375879b19f85a2ce933a9a2828da6ad8ff6
    setupEtcdEnvKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b2d56eee850b10df07bc273bd0061c9a6bdba86d2c2e3ccbaaeb3dea97dd5475
  infra:
    apiVersion: config.openshift.io/v1
    kind: Infrastructure
    metadata: {}
    spec:
      cloudConfig:
        key: config
        name: cloud-provider-config
    status:
      apiServerInternalURI: https://api-int.mnguyen45.gcp.devcluster.openshift.com:6443
      apiServerURL: https://api.mnguyen45.gcp.devcluster.openshift.com:6443
      etcdDiscoveryDomain: mnguyen45.gcp.devcluster.openshift.com
      infrastructureName: mnguye-7vjph
      platform: GCP
      platformStatus:
        gcp:
          projectID: openshift-gce-devel
          region: us-central1
        type: GCP
  kubeAPIServerServingCAData: --SNIP--
  osImageURL: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:776ba70c38bcd2772061bdba1fe7c088b7115ad2190ef159d3505f730e89a117
  platform: gcp
  proxy: null
  pullSecret:
    name: pull-secret
    namespace: openshift-config
  rootCAData: --SNIP--
status:
  conditions:
  - lastTransitionTime: "2020-05-06T18:14:49Z"
    message: sync completed towards (1) generation using controller version v4.5.0-202005041917-dirty
    status: "True"
    type: TemplateControllerCompleted
  - lastTransitionTime: "2020-05-06T18:14:49Z"
    status: "False"
    type: TemplateControllerRunning
  - lastTransitionTime: "2020-05-06T18:14:49Z"
    status: "False"
    type: TemplateControllerFailing
  observedGeneration: 1

Comment 13 Michael Nguyen 2020-05-11 15:10:35 UTC

AZURE:

$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.5.0-0.nightly-2020-05-05-205255   True        False         35m     Cluster version is 4.5.0-0.nightly-2020-05-05-205255
$ oc get nodes
NAME                                                STATUS   ROLES    AGE   VERSION
ci-ln-gfznqqt-002ac-kgpnj-master-0                  Ready    master   56m   v1.18.0-rc.1
ci-ln-gfznqqt-002ac-kgpnj-master-1                  Ready    master   56m   v1.18.0-rc.1
ci-ln-gfznqqt-002ac-kgpnj-master-2                  Ready    master   56m   v1.18.0-rc.1
ci-ln-gfznqqt-002ac-kgpnj-worker-centralus1-hvsjk   Ready    worker   40m   v1.18.0-rc.1
ci-ln-gfznqqt-002ac-kgpnj-worker-centralus2-ss99p   Ready    worker   40m   v1.18.0-rc.1
ci-ln-gfznqqt-002ac-kgpnj-worker-centralus3-54ms9   Ready    worker   41m   v1.18.0-rc.1
$ oc -n openshift-config-managed get cm
NAME                                                  DATA   AGE
bound-sa-token-signing-certs                          1      55m
console-public                                        1      50m
csr-controller-ca                                     1      55m
default-ingress-cert                                  1      48m
grafana-dashboard-cluster-total                       1      32m
grafana-dashboard-etcd                                1      32m
grafana-dashboard-k8s-resources-cluster               1      32m
grafana-dashboard-k8s-resources-namespace             1      32m
grafana-dashboard-k8s-resources-node                  1      32m
grafana-dashboard-k8s-resources-pod                   1      32m
grafana-dashboard-k8s-resources-workload              1      32m
grafana-dashboard-k8s-resources-workloads-namespace   1      32m
grafana-dashboard-node-cluster-rsrc-use               1      32m
grafana-dashboard-node-rsrc-use                       1      32m
grafana-dashboard-prometheus                          1      32m
kube-apiserver-aggregator-client-ca                   1      55m
kube-apiserver-client-ca                              1      55m
kube-apiserver-server-ca                              1      55m
kube-cloud-config                                     1      48m
kubelet-bootstrap-kubeconfig                          1      59m
kubelet-serving-ca                                    1      55m
monitoring-shared-config                              4      43m
oauth-openshift                                       1      40m
release-verification                                  3      32m
sa-token-signing-certs                                2      59m
service-ca                                            1      55m
signatures-managed                                    0      54m
trusted-ca-bundle                                     1      57m
$ oc -n openshift-config-managed describe cm/kube-cloud-config
Name:         kube-cloud-config
Namespace:    openshift-config-managed
Labels:       <none>
Annotations:  <none>

Data
====
cloud.conf:
----
{
  "cloud": "AzurePublicCloud",
  "tenantId": "6047c7e9-b2ad-488d-a54e-dc3f6be6a7ee",
  "aadClientId": "",
  "aadClientSecret": "",
  "aadClientCertPath": "",
  "aadClientCertPassword": "",
  "useManagedIdentityExtension": true,
  "userAssignedIdentityID": "",
  "subscriptionId": "d38f1e38-4bed-438e-b227-833f997adf6a",
  "resourceGroup": "ci-ln-gfznqqt-002ac-kgpnj-rg",
  "location": "centralus",
  "vnetName": "ci-ln-gfznqqt-002ac-kgpnj-vnet",
  "vnetResourceGroup": "ci-ln-gfznqqt-002ac-kgpnj-rg",
  "subnetName": "ci-ln-gfznqqt-002ac-kgpnj-worker-subnet",
  "securityGroupName": "ci-ln-gfznqqt-002ac-kgpnj-node-nsg",
  "routeTableName": "ci-ln-gfznqqt-002ac-kgpnj-node-routetable",
  "primaryAvailabilitySetName": "",
  "vmType": "",
  "primaryScaleSetName": "",
  "cloudProviderBackoff": true,
  "cloudProviderBackoffRetries": 0,
  "cloudProviderBackoffExponent": 0,
  "cloudProviderBackoffDuration": 6,
  "cloudProviderBackoffJitter": 0,
  "cloudProviderRateLimit": true,
  "cloudProviderRateLimitQPS": 6,
  "cloudProviderRateLimitBucket": 10,
  "cloudProviderRateLimitQPSWrite": 6,
  "cloudProviderRateLimitBucketWrite": 10,
  "useInstanceMetadata": true,
  "loadBalancerSku": "standard",
  "excludeMasterFromStandardLB": null,
  "disableOutboundSNAT": null,
  "maximumLoadBalancerRuleCount": 0
}

Events:  <none>
$ oc get controllerconfig/machine-config-controller -o yaml
apiVersion: machineconfiguration.openshift.io/v1
kind: ControllerConfig
metadata:
  creationTimestamp: "2020-05-11T14:20:02Z"
  generation: 1
  name: machine-config-controller
  resourceVersion: "14936"
  selfLink: /apis/machineconfiguration.openshift.io/v1/controllerconfigs/machine-config-controller
  uid: 0576ddf7-d430-4b25-bb9e-7bbdd50e85bd
spec:
  additionalTrustBundle: null
  cloudProviderCAData: null
  cloudProviderConfig: "{\n\t\"cloud\": \"AzurePublicCloud\",\n\t\"tenantId\": \"6047c7e9-b2ad-488d-a54e-dc3f6be6a7ee\",\n\t\"aadClientId\":
    \"\",\n\t\"aadClientSecret\": \"\",\n\t\"aadClientCertPath\": \"\",\n\t\"aadClientCertPassword\":
    \"\",\n\t\"useManagedIdentityExtension\": true,\n\t\"userAssignedIdentityID\":
    \"\",\n\t\"subscriptionId\": \"d38f1e38-4bed-438e-b227-833f997adf6a\",\n\t\"resourceGroup\":
    \"ci-ln-gfznqqt-002ac-kgpnj-rg\",\n\t\"location\": \"centralus\",\n\t\"vnetName\":
    \"ci-ln-gfznqqt-002ac-kgpnj-vnet\",\n\t\"vnetResourceGroup\": \"ci-ln-gfznqqt-002ac-kgpnj-rg\",\n\t\"subnetName\":
    \"ci-ln-gfznqqt-002ac-kgpnj-worker-subnet\",\n\t\"securityGroupName\": \"ci-ln-gfznqqt-002ac-kgpnj-node-nsg\",\n\t\"routeTableName\":
    \"ci-ln-gfznqqt-002ac-kgpnj-node-routetable\",\n\t\"primaryAvailabilitySetName\":
    \"\",\n\t\"vmType\": \"\",\n\t\"primaryScaleSetName\": \"\",\n\t\"cloudProviderBackoff\":
    true,\n\t\"cloudProviderBackoffRetries\": 0,\n\t\"cloudProviderBackoffExponent\":
    0,\n\t\"cloudProviderBackoffDuration\": 6,\n\t\"cloudProviderBackoffJitter\":
    0,\n\t\"cloudProviderRateLimit\": true,\n\t\"cloudProviderRateLimitQPS\": 6,\n\t\"cloudProviderRateLimitBucket\":
    10,\n\t\"cloudProviderRateLimitQPSWrite\": 6,\n\t\"cloudProviderRateLimitBucketWrite\":
    10,\n\t\"useInstanceMetadata\": true,\n\t\"loadBalancerSku\": \"standard\",\n\t\"excludeMasterFromStandardLB\":
    null,\n\t\"disableOutboundSNAT\": null,\n\t\"maximumLoadBalancerRuleCount\": 0\n}\n"
  clusterDNSIP: 172.30.0.10
  etcdCAData: --SNIP--
  etcdDiscoveryDomain: ci-ln-gfznqqt-002ac.ci.azure.devcluster.openshift.com
  etcdMetricCAData: --SNIP--
  images:
    baremetalRuntimeCfgImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1cd6dfed1c2178e4811b02c9a735cae09898d8ea66abbfebf50de715fa343a1
    clusterEtcdOperatorImageKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:418c8fe15f2b9391d07b49a730264a396bc2979a37ed8ed0641542d041a00eaa
    corednsImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15a47f1b950f3c8ae8446e02fd5f98813d5664233361f9f04fc1e9164793e04a
    etcdKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:01106cb6d8c2825f5ccd96038be81bc444adf5385cdb27cdd79295197d4c81d3
    gcpRoutesControllerKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b2d56eee850b10df07bc273bd0061c9a6bdba86d2c2e3ccbaaeb3dea97dd5475
    haproxyImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:31e518340773eda69f7510ab0eeb64f2121b583f58c71d9621c899826895622e
    infraImageKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d54892b888e0839a42c00fec023fd9ac80ea02d7214c59609688bdc2064457d2
    keepalivedImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:100f9518f67b20260cb75a5d2e1f2d3d44e0346bb366410fa70c0d35229731ac
    kubeClientAgentImageKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:14420911c35d5d2939298d7c1f545b9a2d1c05890e0991afc7aabfc80dc1d3ea
    mdnsPublisherImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32c7aeaec4d965606fd7dfec34ed3375879b19f85a2ce933a9a2828da6ad8ff6
    setupEtcdEnvKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b2d56eee850b10df07bc273bd0061c9a6bdba86d2c2e3ccbaaeb3dea97dd5475
  infra:
    apiVersion: config.openshift.io/v1
    kind: Infrastructure
    metadata: {}
    spec:
      cloudConfig:
        key: config
        name: cloud-provider-config
    status:
      apiServerInternalURI: https://api-int.ci-ln-gfznqqt-002ac.ci.azure.devcluster.openshift.com:6443
      apiServerURL: https://api.ci-ln-gfznqqt-002ac.ci.azure.devcluster.openshift.com:6443
      etcdDiscoveryDomain: ci-ln-gfznqqt-002ac.ci.azure.devcluster.openshift.com
      infrastructureName: ci-ln-gfznqqt-002ac-kgpnj
      platform: Azure
      platformStatus:
        azure:
          networkResourceGroupName: ci-ln-gfznqqt-002ac-kgpnj-rg
          resourceGroupName: ci-ln-gfznqqt-002ac-kgpnj-rg
        type: Azure
  kubeAPIServerServingCAData: --SNIP--
  osImageURL: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:776ba70c38bcd2772061bdba1fe7c088b7115ad2190ef159d3505f730e89a117
  platform: azure
  proxy: null
  pullSecret:
    name: pull-secret
    namespace: openshift-config
  rootCAData: --SNIP--
status:
  conditions:
  - lastTransitionTime: "2020-05-11T14:20:03Z"
    message: sync completed towards (1) generation using controller version v4.5.0-202005041917-dirty
    status: "True"
    type: TemplateControllerCompleted
  - lastTransitionTime: "2020-05-11T14:20:03Z"
    status: "False"
    type: TemplateControllerRunning
  - lastTransitionTime: "2020-05-11T14:20:03Z"
    status: "False"
    type: TemplateControllerFailing
  observedGeneration: 1

Comment 14 Sinny Kumari 2020-05-11 16:36:37 UTC

On aws, I tried to add custom iam endpoint and it seems to create kube-cloud-config configmap.

Steps I used:
1. Generated install-config
$ env OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE=registry.svc.ci.openshift.org/ocp/release:4.5.0-0.ci-2020-05-11-114006 ./bin/openshift-install --dir asset  create install-config  --log-level=debug

2. Edited platform section in generated install-config to add custoim iam endpoint
$ cat asset/install-config.yaml
...
platform:
  aws:
    serviceEndpoints:
      - name: iam
        url: https://iam.amazonaws.com
    region: us-east-1
...

3. Create cluster
$ env OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE=registry.svc.ci.openshift.org/ocp/release:4.5.0-0.ci-2020-05-11-114006 ./bin/openshift-install --dir asset  create cluster  --log-level=debug

4. Once, cluster is up check kube-cloud-config configmap
$ oc -n openshift-config-managed describe cm/kube-cloud-config
Name:         kube-cloud-config
Namespace:    openshift-config-managed
Labels:       <none>
Annotations:  <none>

Data
====
cloud.conf:
----

[ServiceOverride "0"]
  Service = iam
  Region = us-east-1
  URL = https://iam.amazonaws.com
  SigningRegion = us-east-1

Events:  <none

Comment 15 Michael Nguyen 2020-05-11 18:22:25 UTC

AWS:
$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.5.0-0.nightly-2020-05-05-205255   True        False         5m23s   Cluster version is 4.5.0-0.nightly-2020-05-05-205255
$ oc -n openshift-config-managed describe cm/kube-cloud-config
Name:         kube-cloud-config
Namespace:    openshift-config-managed
Labels:       <none>
Annotations:  <none>

Data
====
cloud.conf:
----

[ServiceOverride "0"]
  Service = ec2
  Region = us-east-1
  URL = https://ec2.amazonaws.com
  SigningRegion = us-east-1

[ServiceOverride "1"]
  Service = iam
  Region = us-east-1
  URL = https://iam.amazonaws.com
  SigningRegion = us-east-1

Events:  <none>
$ oc get controllerconfig/machine-config-controller -o yaml
apiVersion: machineconfiguration.openshift.io/v1
kind: ControllerConfig
metadata:
  creationTimestamp: "2020-05-11T17:53:57Z"
  generation: 2
  name: machine-config-controller
  resourceVersion: "15214"
  selfLink: /apis/machineconfiguration.openshift.io/v1/controllerconfigs/machine-config-controller
  uid: 88f46212-88d8-41d3-8b9a-dc096e482217
spec:
  additionalTrustBundle: null
  cloudProviderCAData: null
  cloudProviderConfig: "\n[ServiceOverride \"0\"]\n\tService = ec2\n\tRegion = us-east-1\n\tURL
    = https://ec2.amazonaws.com\n\tSigningRegion = us-east-1\n\n[ServiceOverride \"1\"]\n\tService
    = iam\n\tRegion = us-east-1\n\tURL = https://iam.amazonaws.com\n\tSigningRegion
    = us-east-1\n"
  clusterDNSIP: 172.30.0.10
  etcdCAData: --SNIP--
  etcdDiscoveryDomain: mnguyen45.devcluster.openshift.com
  etcdMetricCAData: --SNIP--
  images:
    baremetalRuntimeCfgImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1cd6dfed1c2178e4811b02c9a735cae09898d8ea66abbfebf50de715fa343a1
    clusterEtcdOperatorImageKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:418c8fe15f2b9391d07b49a730264a396bc2979a37ed8ed0641542d041a00eaa
    corednsImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15a47f1b950f3c8ae8446e02fd5f98813d5664233361f9f04fc1e9164793e04a
    etcdKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:01106cb6d8c2825f5ccd96038be81bc444adf5385cdb27cdd79295197d4c81d3
    gcpRoutesControllerKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b2d56eee850b10df07bc273bd0061c9a6bdba86d2c2e3ccbaaeb3dea97dd5475
    haproxyImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:31e518340773eda69f7510ab0eeb64f2121b583f58c71d9621c899826895622e
    infraImageKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d54892b888e0839a42c00fec023fd9ac80ea02d7214c59609688bdc2064457d2
    keepalivedImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:100f9518f67b20260cb75a5d2e1f2d3d44e0346bb366410fa70c0d35229731ac
    kubeClientAgentImageKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:14420911c35d5d2939298d7c1f545b9a2d1c05890e0991afc7aabfc80dc1d3ea
    mdnsPublisherImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32c7aeaec4d965606fd7dfec34ed3375879b19f85a2ce933a9a2828da6ad8ff6
    setupEtcdEnvKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b2d56eee850b10df07bc273bd0061c9a6bdba86d2c2e3ccbaaeb3dea97dd5475
  infra:
    apiVersion: config.openshift.io/v1
    kind: Infrastructure
    metadata: {}
    spec:
      cloudConfig:
        name: ""
    status:
      apiServerInternalURI: https://api-int.mnguyen45.devcluster.openshift.com:6443
      apiServerURL: https://api.mnguyen45.devcluster.openshift.com:6443
      etcdDiscoveryDomain: mnguyen45.devcluster.openshift.com
      infrastructureName: mnguyen45-kh6cj
      platform: AWS
      platformStatus:
        aws:
          region: us-east-1
        type: AWS
  kubeAPIServerServingCAData: --SNIP--
  osImageURL: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:776ba70c38bcd2772061bdba1fe7c088b7115ad2190ef159d3505f730e89a117
  platform: aws
  proxy: null
  pullSecret:
    name: pull-secret
    namespace: openshift-config
  rootCAData: --SNIP--
status:
  conditions:
  - lastTransitionTime: "2020-05-11T17:53:58Z"
    status: "False"
    type: TemplateControllerFailing
  - lastTransitionTime: "2020-05-11T18:02:03Z"
    message: sync completed towards (2) generation using controller version v4.5.0-202005041917-dirty
    status: "True"
    type: TemplateControllerCompleted
  - lastTransitionTime: "2020-05-11T18:02:03Z"
    status: "False"
    type: TemplateControllerRunning
  observedGeneration: 2

Comment 18 Michael Nguyen 2020-05-20 00:35:13 UTC

VSPHERE!

$ oc get nodes
NAME              STATUS   ROLES    AGE   VERSION
compute-0         Ready    worker   46m   v1.18.2
compute-1         Ready    worker   47m   v1.18.2
compute-2         Ready    worker   46m   v1.18.2
control-plane-0   Ready    master   59m   v1.18.2
control-plane-1   Ready    master   59m   v1.18.2
control-plane-2   Ready    master   59m   v1.18.2
$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.5.0-0.nightly-2020-05-19-041951   True        False         38m     Cluster version is 4.5.0-0.nightly-2020-05-19-041951
$ oc -n openshift-config-managed describe cm/kube-cloud-config
Name:         kube-cloud-config
Namespace:    openshift-config-managed
Labels:       <none>
Annotations:  <none>

Data
====
cloud.conf:
----
[Global]
secret-name = "vsphere-creds"
secret-namespace = "kube-system"
insecure-flag = "1"

[Workspace]
server = "vcsa-ci.vmware.devcluster.openshift.com"
datacenter = "dc1"
default-datastore = "nvme-ds1"
folder = "ci-ln-vvp2xkb-8c773"

[VirtualCenter "vcsa-ci.vmware.devcluster.openshift.com"]
datacenters = "dc1"

Events:  <none>
$ oc get controllerconfig/machine-config-controller -o yaml
apiVersion: machineconfiguration.openshift.io/v1
kind: ControllerConfig
metadata:
  creationTimestamp: "2020-05-19T23:41:47Z"
  generation: 1
  managedFields:
  - apiVersion: machineconfiguration.openshift.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        .: {}
        f:conditions: {}
        f:observedGeneration: {}
    manager: machine-config-controller
    operation: Update
    time: "2020-05-19T23:41:47Z"
  - apiVersion: machineconfiguration.openshift.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:spec:
        .: {}
        f:additionalTrustBundle: {}
        f:cloudProviderCAData: {}
        f:cloudProviderConfig: {}
        f:clusterDNSIP: {}
        f:etcdCAData: {}
        f:etcdDiscoveryDomain: {}
        f:etcdMetricCAData: {}
        f:images:
          .: {}
          f:baremetalRuntimeCfgImage: {}
          f:clusterEtcdOperatorImageKey: {}
          f:corednsImage: {}
          f:etcdKey: {}
          f:gcpRoutesControllerKey: {}
          f:haproxyImage: {}
          f:infraImageKey: {}
          f:keepalivedImage: {}
          f:kubeClientAgentImageKey: {}
          f:mdnsPublisherImage: {}
          f:setupEtcdEnvKey: {}
        f:infra:
          .: {}
          f:apiVersion: {}
          f:kind: {}
          f:metadata: {}
          f:spec:
            .: {}
            f:cloudConfig:
              .: {}
              f:key: {}
              f:name: {}
          f:status:
            .: {}
            f:apiServerInternalURI: {}
            f:apiServerURL: {}
            f:etcdDiscoveryDomain: {}
            f:infrastructureName: {}
            f:platform: {}
            f:platformStatus:
              .: {}
              f:type: {}
        f:kubeAPIServerServingCAData: {}
        f:osImageURL: {}
        f:platform: {}
        f:proxy: {}
        f:pullSecret:
          .: {}
          f:name: {}
          f:namespace: {}
        f:rootCAData: {}
    manager: machine-config-operator
    operation: Update
    time: "2020-05-19T23:41:47Z"
  name: machine-config-controller
  resourceVersion: "13997"
  selfLink: /apis/machineconfiguration.openshift.io/v1/controllerconfigs/machine-config-controller
  uid: ce01723c-e30d-4ddc-89e7-18ab7a16dec3
spec:
  additionalTrustBundle: null
  cloudProviderCAData: null
  cloudProviderConfig: |
    [Global]
    secret-name = "vsphere-creds"
    secret-namespace = "kube-system"
    insecure-flag = "1"

    [Workspace]
    server = "vcsa-ci.vmware.devcluster.openshift.com"
    datacenter = "dc1"
    default-datastore = "nvme-ds1"
    folder = "ci-ln-vvp2xkb-8c773"

    [VirtualCenter "vcsa-ci.vmware.devcluster.openshift.com"]
    datacenters = "dc1"
  clusterDNSIP: 172.30.0.10
  etcdCAData: --SNIP--
  etcdDiscoveryDomain: ci-ln-vvp2xkb-8c773.origin-ci-int-aws.dev.rhcloud.com
  etcdMetricCAData: --SNIP--
  images:
    baremetalRuntimeCfgImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:823c36590e8029679f03800006fab52c06be42dc2dacdf57533f174ee2a2df14
    clusterEtcdOperatorImageKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:08493bd36be521fc2057ac891e640ba82fc3dd8d564ee71be44bf1dc14ddfc6a
    corednsImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0246d160d94d8a2d49610038b15b1cf538364fb0a733ea06092a55685ab7bac1
    etcdKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:481365c00594d914759480590a3df47414e1f2233f2a344e418c89043ba9245b
    gcpRoutesControllerKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:557f0fd87f307469cf3fcca41741b2491966ab7b5fbb94720ccdce6a4de8b34f
    haproxyImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:430cf1bc3b7506e935578db1a9a193f21c12ffa23534ece6f5d826d02112d061
    infraImageKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e298a789160492e9f0d799549abee201b64a56922e554d81d5ad8eeb6ac5e4d
    keepalivedImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0e39f28287ba4d17c17734456b2cede5425d2b67476058fa24ab5f56c9b02b9e
    kubeClientAgentImageKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:49eadc3c77c3864a827c0066cd27f85fc5bcf0d203aed591a6560b11a056da74
    mdnsPublisherImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:93e96aef3081faf5a79794db855bb030d62f226aeb0316caa28bd5dd044600ac
    setupEtcdEnvKey: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:557f0fd87f307469cf3fcca41741b2491966ab7b5fbb94720ccdce6a4de8b34f
  infra:
    apiVersion: config.openshift.io/v1
    kind: Infrastructure
    metadata: {}
    spec:
      cloudConfig:
        key: config
        name: cloud-provider-config
    status:
      apiServerInternalURI: https://api-int.ci-ln-vvp2xkb-8c773.origin-ci-int-aws.dev.rhcloud.com:6443
      apiServerURL: https://api.ci-ln-vvp2xkb-8c773.origin-ci-int-aws.dev.rhcloud.com:6443
      etcdDiscoveryDomain: ci-ln-vvp2xkb-8c773.origin-ci-int-aws.dev.rhcloud.com
      infrastructureName: ci-ln-vvp2xkb-8c773-dh245
      platform: VSphere
      platformStatus:
        type: VSphere
  kubeAPIServerServingCAData: --SNIP--
  osImageURL: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c7b3125e5cfbc8b84495669f27e80c3f18ec3a39ed455709fcca91d5721f528
  platform: vsphere
  proxy: null
  pullSecret:
    name: pull-secret
    namespace: openshift-config
  rootCAData: --SNIP--
status:
  conditions:
  - lastTransitionTime: "2020-05-19T23:41:47Z"
    message: sync completed towards (1) generation using controller version v4.5.0-202005190117-dirty
    status: "True"
    type: TemplateControllerCompleted
  - lastTransitionTime: "2020-05-19T23:41:47Z"
    status: "False"
    type: TemplateControllerRunning
  - lastTransitionTime: "2020-05-19T23:41:47Z"
    status: "False"
    type: TemplateControllerFailing
  observedGeneration: 1
$

Comment 19 errata-xmlrpc 2020-07-13 17:29:07 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2409