Red Hat Bugzilla – Bug 1491202
[Federation] Failed to create load balancer for service federation-system/apiserver on GCE
Last modified: 2017-11-28 17:10:32 EST
Description of problem: The apiserver service is pending and blocks subsequent process during federation control panel. The error is "Failed to create load balancer for service federation-system/qwangfed-apiserver: GCECloud.ClusterID is not ready. Call Initialize() before using" This problem occurs in OCP 3.7.0-0.125.0 with image ose-federation:v3.7.0-0.125.0 and v3.6.173.0.30, not in OCP 3.6.173.0.30 with the same images. Version-Release number of selected component (if applicable): openshift v3.7.0-0.125.0 kubernetes v1.7.0+695f48a16f etcd 3.2.1 registry.ops.openshift.com/openshift3/ose-federation:v3.7.0-0.125.0 How reproducible: Always Steps to Reproduce: 1. Initialize federation control panel 2. Check whether federation control panel works Actual results: 1. [root@preserve-910-qe-qwang-37-federation-master-etcd-nfs-1 ~]# kubefed init qwangfed --dns-provider=google-clouddns --dns-zone-name=federation.ocpqe.com. --etcd-persistent-storage=true --image=registry.ops.openshift.com/openshift3/ose-federation:v3.7.0-0.125.0 Creating a namespace federation-system for federation system components... done Creating federation control plane service........................................... 2. [root@preserve-910-qe-qwang-37-federation-master-etcd-nfs-1 ~]# oc get all -n federation-system NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE svc/qwangfed36173-apiserver 172.30.138.23 <pending> 443:32539/TCP 56m [root@preserve-910-qe-qwang-37-federation-master-etcd-nfs-1 ~]# oc describe svc/qwangfed36173-apiserver -n federation-system Name: qwangfed36173-apiserver Namespace: federation-system Labels: app=federated-cluster Annotations: federation.alpha.kubernetes.io/federation-name=qwangfed36173 Selector: app=federated-cluster,module=federation-apiserver Type: LoadBalancer IP: 172.30.138.23 Port: https 443/TCP NodePort: https 32539/TCP Endpoints: <none> Session Affinity: None Events: FirstSeen LastSeen Count From SubObjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 57m 2m 17 service-controller Normal CreatingLoadBalancer Creating load balancer 57m 2m 17 service-controller Warning CreatingLoadBalancerFailed Error creating load balancer (will retry): Failed to create load balancer for service federation-system/qwangfed36173-apiserver: GCECloud.ClusterID is not ready. Call Initialize() before using. Expected results: The load balancer can be created. Additional info:
This looks like an error creating a Service type LoadBalancer on GCE (not specific to federation).
Can you please provide the master logs when this happen? Also is this permanently broken or it fixes itself (as there should be retry). To me this seems like the GCE is lagging in setting the cluster ID as ready.
Created attachment 1326376 [details] atomic-openshift-master-controllers
Created attachment 1326377 [details] atomic-openshift-master-api
It's permanently broken. Attached the master logs.
It blocks the federation cluster setup for OpenShift 3.7 on GCP.
The https://github.com/openshift/origin/pull/16089 was merged, setting this ON_QA.
The fix has not been in OCP 3.7.0-0.126.4 yet.
Moving back to modified.
Tested on openshift v3.7.0-0.143.2 (kubernetes v1.7.0+80709908fd, etcd 3.2.1, registry.ops.openshift.com/openshift3/ose-federation:v3.7.0-0.147.1), the bug has been fixed, thanks. [root@preserve-qw-master-etcd-nfs-1 ~]# oc get all -n federation-system NAME READY STATUS RESTARTS AGE po/qwangfed-apiserver-2147565502-7b76l 2/2 Running 0 36m po/qwangfed-controller-manager-1909511936-s5kqx 1/1 Running 1 38m NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE svc/qwangfed-apiserver 172.30.0.140 172.29.147.174,172.29.147.174 443:32697/TCP 38m NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE deploy/qwangfed-apiserver 1 1 1 1 38m deploy/qwangfed-controller-manager 1 1 1 1 38m NAME DESIRED CURRENT READY AGE rs/qwangfed-apiserver-2147565502 1 1 1 36m rs/qwangfed-apiserver-265993101 0 0 0 38m rs/qwangfed-controller-manager-1909511936 1 1 1 38m
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:3188