Bug 1896696

Summary: OCP Service missing endpoint annotation openstack.org/kuryr-lbaas-state: causing kuryr controller to constantly crash
Product: OpenShift Container Platform Reporter: Mohammad <mahmad>
Component: NetworkingAssignee: Maysa Macedo <mdemaced>
Networking sub component: kuryr QA Contact: Itzik Brown <itbrown>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: medium CC: itbrown
Version: 3.11.0   
Target Milestone: ---   
Target Release: 3.11.z   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-12-16 12:35:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
oc get svc myservice -o yaml
none
oc get endpoints myservice -o yaml none

Description Mohammad 2020-11-11 10:13:53 UTC
Created attachment 1728287 [details]
oc get svc myservice -o yaml

Description of problem:

OCP Service missing endpoint openstack.org/kuryr-lbaas-state: causing kuryr controller to constantly crash

Version-Release number of selected component (if applicable): 3.11.306


How reproducible: We haven't seen this before. It seems completely random at this stage.


Steps to Reproduce:
Unsure at this stage, normal service creation process was used.

Actual results:

Creating a service would result in all the annotations being created.

Comment 1 Mohammad 2020-11-11 10:14:36 UTC
Created attachment 1728288 [details]
oc get endpoints myservice -o yaml

Comment 2 Mohammad 2020-11-11 10:32:39 UTC
Further information:

$ openstack loadbalancer list |grep XX.XX.145.109
| 14e18ac8-f68e-4b02-98e0-bee3aae5fcec | mynamespace/myservice-hotrod                     | c2958d528ceb43c29520f4a965c4ea45 | XX.XX.145.109 | ACTIVE              | octavia  |


$ openstack loadbalancer listener list --loadbalancer 14e18ac8-f68e-4b02-98e0-bee3aae5fcec

{ no output }

$ openstack loadbalancer pool list --loadbalancer 14e18ac8-f68e-4b02-98e0-bee3aae5fcec

{ no output }

Comment 3 Mohammad 2020-11-11 10:36:26 UTC
Clarifying, myservice-hotrod and myservice in this case are the same, this was my mistake.

Comment 5 Mohammad 2020-11-11 10:50:47 UTC
(In reply to Mohammad from comment #3)
> Clarifying, myservice-hotrod and myservice in this case are the same, this
> was my mistake.

This would be the service/loadbalancer with IP XX.XX.145.109

Comment 11 Itzik Brown 2020-12-10 16:09:58 UTC
Verified:
OCP v3.11.319
OSP 2020-10-06.2

Ran two scenarios:
1) Created a service and deleted the controller before that service got the openstack.org/kuryr-lbaas-state annotation and verified the service connectivity

2) Created a LB with tag 'openshift' with the service subnet, Deleted the Kuryr controller and then verified that the LB is deleted when the controller was up again

Comment 13 errata-xmlrpc 2020-12-16 12:35:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 3.11.343 security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5363