If the kuryr-controller gets restarted at certain point when creating a service of LoadBalancer type it may happen that the associated floating IP is not yet annotated into the services, this leads to the next: - When kuryr-controller starts again it created a new floating ip - when trying to assing the floating IP to the loadbalancer VIP is fails due to the port already having a FIP: 2018-12-04 12:34:12.848 1 ERROR kuryr_kubernetes.handlers.logging [-] Failed to handle event {u'object': {u'kind': u'Endpoints', u'subsets': [{u'addresses': [{u'ip': u'10.1.0.207', u'targetRef': {u'kind': u'Pod', u'resourceVersion': u'1554', u'namespace': u'default', u'name': u'kuryr-pod-404655504', u'uid': u'4c9fce4a-f7ba-11e8-b317-fa163ecbea25'}, u'nodeName': u'ubuntu-xenial-limestone-regionone-0000902971'}, {u'ip': u'10.1.0.249', u'targetRef': {u'kind': u'Pod', u'resourceVersion': u'1637', u'namespace': u'default', u'name': u'kuryr-pod-788029385', u'uid': u'79a8703a-f7ba-11e8-b317-fa163ecbea25'}, u'nodeName': u'ubuntu-xenial-limestone-regionone-0000902971'}], u'ports': [{u'protocol': u'TCP', u'port': 8080}]}], u'apiVersion': u'v1', u'metadata': {u'name': u'kuryr-service-768175562', u'namespace': u'default', u'resourceVersion': u'1647', u'creationTimestamp': u'2018-12-04T11:48:26Z', u'annotations': {u'openstack.org/kuryr-lbaas-spec': u'{"versioned_object.data": {"ip": "10.1.0.148", "lb_ip": null, "ports": [{"versioned_object.data": {"name": null, "port": 80, "protocol": "TCP"}, "versioned_object.name": "LBaaSPortSpec", "versioned_object.namespace": "kuryr_kubernetes", "versioned_object.version": "1.0"}], "project_id": "a4d8fd0afb164986bd692d2d0fb57229", "security_groups_ids": ["e4580fd6-a9e3-4e59-a088-d883410fc92e", "f092fde7-db20-407f-ad12-fb5d5a093569"], "subnet_id": "68bb17ae-ca6a-4cee-bc50-8334514361ee", "type": "LoadBalancer"}, "versioned_object.name": "LBaaSServiceSpec", "versioned_object.namespace": "kuryr_kubernetes", "versioned_object.version": "1.0"}'}, u'selfLink': u'/api/v1/namespaces/default/endpoints/kuryr-service-768175562', u'uid': u'82b03730-f7ba-11e8-b317-fa163ecbea25'}}, u'type': u'ADDED'}: Conflict: Cannot associate floating IP 172.24.5.31 (47847532-c29b-4443-b270-7c8d0593de5a) with port d2b1add7-d3c9-4b7b-9f78-092ef24bf8be using fixed IP 10.1.0.148, as that fixed IP already has a floating IP on external network 1b513b3f-e6c4-4183-89a4-b0df353e22d5. - After several retries, the kuryr-controller will crash and restart again, generating and extra floating ip - After a certain number of restarts, there user quota for FIPs is reached, therefore leading the kuryr-controller to a NotReady status
Will verify when we have the proper image (https://bugzilla.redhat.com/show_bug.cgi?id=1673799)
Checked with: openstack-kuryr-kubernetes-0.5.3-0.20190121111334
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0591