Bug 1596696
| Summary: | Kuryr-controller crashes if Octavia amphoras take too long or fail to be created | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Luis Tomas Bolivar <ltomasbo> |
| Component: | openstack-octavia | Assignee: | Luis Tomas Bolivar <ltomasbo> |
| Status: | CLOSED ERRATA | QA Contact: | Jon Uriarte <juriarte> |
| Severity: | urgent | Docs Contact: | |
| Priority: | urgent | ||
| Version: | 13.0 (Queens) | CC: | asegurap, ihrachys, joflynn, jschluet, lpeer, ltomasbo, majopela, nyechiel, tsedovic |
| Target Milestone: | z2 | Keywords: | Triaged, ZStream |
| Target Release: | 13.0 (Queens) | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | openstack-octavia-2.0.1-6.ddaf235git.el7ost | Doc Type: | Bug Fix |
| Doc Text: |
Previously, filtering load balancers by VIP data raised an internal server error (500 HTTP) in octavia-api. As a result, the Kuryr-controller did not locate previously created load balancers upon restart and crashed.
In this update, the fields translation on filtering has been corrected and pool filtering by healthmonitor_id has been added. Now filtering load balancers by VIP data is possible and the Kuryr-controller can find previously created load balancers.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-08-29 16:20:28 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Luis Tomas Bolivar
2018-06-29 13:22:46 UTC
https://review.openstack.org/#/c/579144/ is not needed anymore as the problem was fixed on Octavia (and backported): https://review.openstack.org/#/c/579559/ https://review.openstack.org/#/c/579846/ This bug is marked for inclusion in the errata but does not currently contain draft documentation text. To ensure the timely release of this advisory please provide draft documentation text for this bug as soon as possible. If you do not think this bug requires errata documentation, set the requires_doc_text flag to "-". To add draft documentation text: * Select the documentation type from the "Doc Type" drop down field. * A template will be provided in the "Doc Text" field based on the "Doc Type" value selected. Enter draft text in the "Doc Text" field. Verified on OSP 13 2018-08-16.1 puddle / openshift-ansible-3.10.34-1, and: openstack-kuryr-kubernetes-cni-0.4.3-2.el7ost.noarch openstack-kuryr-kubernetes-controller-0.4.3-2.el7ost.noarch Verification steps: 1. Deploy OCP on OSP with kuryr 2. From master node: $ oc new-project test $ oc run --image=kuryr/demo demo -n test $ oc scale -n test dc/demo --replicas=2 $ oc get pods NAME READY STATUS RESTARTS AGE demo-1-t24lc 1/1 Running 0 2m demo-1-xx2mx 1/1 Running 0 2m $ oc expose -n test dc/demo --port 80 --target-port 8080 $ oc get pods -n openshift-infra NAME READY STATUS RESTARTS AGE bootstrap-autoapprover-0 1/1 Running 0 2h kuryr-cni-ds-85kmz 2/2 Running 0 2h kuryr-cni-ds-fhfcz 2/2 Running 0 2h kuryr-cni-ds-qq8pq 2/2 Running 0 2h kuryr-cni-ds-xzrd4 2/2 Running 0 2h kuryr-controller-59fc7f478b-swpsm 1/1 Running 3 58m 3. Delete kuryr-controller pod, from master node: $ oc -n openshift-infra delete pod kuryr-controller-59fc7f478b-swpsm 4. Check a new kuryr-controller pod is deployed, with no error messages, from master node: $ oc get pods -n openshift-infra NAME READY STATUS RESTARTS AGE bootstrap-autoapprover-0 1/1 Running 0 2h kuryr-cni-ds-85kmz 2/2 Running 0 2h kuryr-cni-ds-fhfcz 2/2 Running 0 2h kuryr-cni-ds-qq8pq 2/2 Running 0 2h kuryr-cni-ds-xzrd4 2/2 Running 0 2h kuryr-controller-59fc7f478b-jfz2b 1/1 Running 0 13s $ oc -n openshift-infra logs -f kuryr-controller-59fc7f478b-2gdrr 2018-08-27 14:26:53.632 1 INFO kuryr_kubernetes.config [-] Logging enabled! 2018-08-27 14:26:53.633 1 INFO kuryr_kubernetes.config [-] /usr/bin/kuryr-k8s-controller version 0.4.3 2018-08-27 14:26:55.154 1 INFO os_vif [-] Loaded VIF plugins: noop, ovs, linux_bridge 2018-08-27 14:26:55.241 1 INFO kuryr_kubernetes.controller.service [-] Service 'KuryrK8sService' stopped 2018-08-27 14:26:55.250 1 INFO kuryr_kubernetes.controller.service [-] Service 'KuryrK8sService' starting 2018-08-27 14:26:55.250 1 INFO kuryr_kubernetes.controller.managers.health [-] Starting health check server. 2018-08-27 14:26:55.251 1 INFO werkzeug [-] * Running on http://localhost:8082/ 2018-08-27 14:26:55.257 1 INFO kuryr_kubernetes.watcher [-] Started watching '/api/v1/pods' 2018-08-27 14:26:55.259 1 INFO kuryr_kubernetes.watcher [-] Started watching '/api/v1/services' 2018-08-27 14:26:55.262 1 INFO kuryr_kubernetes.watcher [-] Started watching '/api/v1/endpoints' 2018-08-27 14:26:56.914 1 INFO kuryr_kubernetes.controller.drivers.vif_pool [-] PORTS POOL: pools updated with pre-created ports 5. Curl to the service (load balancer) IP, and check it is routed to each pod in a round robin basis, from the master node: $ oc get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE demo ClusterIP 172.30.1.140 <none> 80/TCP 1m $ curl 172.30.1.140 demo-1-t24lc: HELLO! I AM ALIVE!!! $ curl 172.30.1.140 demo-1-xx2mx: HELLO! I AM ALIVE!!! 6. Check kuryr-controller logs, no exception should be raised. 7. Delete the project: $ oc delete project test Forgot to add a step in the verification: After creating the service (oc expose -n test dc/demo --port 80 --target-port 8080) a new scale-out must be performed: $ oc scale -n test dc/demo --replicas=3 The next steps remain equal. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:2596 |