Description of problem: On OpenStack instances, configure master to enable cloud provider, master can not be started because of error 'Config error: unrecognised lb-version "v2"' Version-Release number of selected component (if applicable): atomic-openshift-3.4.0.16-1.git.0.cc70b72.el7.x86_64 cockpit-kubernetes-0.114-2.el7.x86_64 Error logs: /usr/bin/openshift start master controllers --config=/etc/origin/master/master-config.yaml --loglevel=10 --listen=https://0.0.0.0:8444 fails with: I1104 03:46:11.044004 92992 round_trippers.go:318] GET https://test-ocp34-master-0.example.com:8443/api/v1/watch/pods?fieldSelector=status.phase%21%3DPending%2Cstatus.phase%21%3DRunning%2Cstatus.phase%21%3DUnknown&resourceVersion=7&timeoutSeconds=551 200 OK in 34 milliseconds I1104 03:46:11.044019 92992 round_trippers.go:324] Response Headers: I1104 03:46:11.044024 92992 round_trippers.go:327] Cache-Control: no-store I1104 03:46:11.044028 92992 round_trippers.go:327] Content-Type: application/vnd.kubernetes.protobuf;stream=watch I1104 03:46:11.044038 92992 round_trippers.go:327] Date: Fri, 04 Nov 2016 07:46:11 GMT I1104 03:46:11.118536 92992 openstack.go:410] Using LBaaS extension v2 I1104 03:46:11.118547 92992 openstack.go:413] Claiming to support LoadBalancer W1104 03:46:11.118552 92992 openstack.go:420] Config error: unrecognised lb-version "v2" F1104 03:46:11.118558 92992 master.go:453] Unable to start service controller: the cloud provider does not support external load balancers. Additional info: In this block in pkg/cloudprovider/providers/openstack/openstack.go: if os.lbOpts.LBVersion == "v2" { return &LbaasV2{LoadBalancer{network, compute, os.lbOpts}}, true } else if lbversion == "v1" { return &LbaasV1{LoadBalancer{network, compute, os.lbOpts}}, true } else { glog.Warningf("Config error: unrecognised lb-version \"%v\"", lbversion) return nil, false } the check: if os.lbOpts.LBVersion == "v2" should be probably replaced with: if lbversion == "v2"
https://github.com/openshift/origin/pull/11789
Worth noting the severity of this bug (i.e. it is fatal to the master) is mitigated by https://bugzilla.redhat.com/show_bug.cgi?id=1389205 Kube PR: https://github.com/kubernetes/kubernetes/pull/36249 Origin cherry pick PR: https://github.com/openshift/origin/pull/11789
Thanks for the quick fix Seth. BZ 1389205 is fatal for openstack envs where LBaaS is not available, in opposite this BZ is fatal for openstack envs where LBaaS v2 is available (supposing LBaaSv1 is deprecated, it would be majority), so AIUI fixing of 1389205 has no impact on severity of this bug.
The call chain is MasterConfig RunServiceLoadBalancerController() ServiceController.New() ServiceController.init() cloudprovider.LoadBalancer() PR: https://github.com/openshift/origin/pull/11648/files Before the PR, if LoadBalancer() returned an error, it would result in a glog.Fatalf() in RunServiceLoadBalancerController(). After the PR, LoadBalancer() failure only does glog.Errorf() and prevents the start of the service controller, rather than killing the whole master. In other words it prevents this from being fatal: F1104 03:46:11.118558 92992 master.go:453] Unable to start service controller: the cloud provider does not support external load balancers.
Ah, right, thanks.
This has been merged into ose and is in OSE v3.4.0.24 or newer.
@Jan, because QE do not have a openstack with LbaasV2 installed, we only have a OSP9 + LbaasV1, could you help verify this bug?
Unfortunately I don't have an env with LBaasV2 at the moment too, we hit this when we manually enabled LBaaSv2 in the scale lab as a workaround for BZ 1389205 but disabled it again when BZ 1389205 was fixed.
Is it possible to enable LBaasV2 on OSP9, if yes, could you tell me how to enable it?
Supposing neutron-lbaas packages are available in OSP9 then yes (I think these are avail, but I can not confirm - I used OSP10). You can follow http://docs.openstack.org/mitaka/networking-guide/config-lbaas.html
Testing in OSP9 which enables LBaaSv2.(cloudprovider is enabled in OpenShift) Verified with openshift v3.4.0.26 1.Reproduced with openshift v3.4.0.16+cc70b72 kubernetes v1.4.0+776c994 Master can't start due to "getsockopt: connection refused". Start the atomic-openshift-master service manually, there's a similar error threw out like Jan's report: #/usr/bin/openshift start master --config=/etc/origin/master/master-config.yaml --loglevel=10 I1116 02:56:19.056420 3650 openstack.go:410] Using LBaaS extension v2 I1116 02:56:19.056440 3650 openstack.go:413] Claiming to support LoadBalancer W1116 02:56:19.056448 3650 openstack.go:420] Config error: unrecognised lb-version "v2" F1116 02:56:19.056457 3650 master.go:453] Unable to start service controller: the cloud provider does not support external load balancers. 2. Trigger a fresh installation with openshift v3.4.0.26+f7e109e kubernetes v1.4.0+776c994 Installation succeed, Master and Node services both are working well, S2I build succeed. Move to verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:0066