Bug 2054145
| Summary: | Server on port X did not begin passing traffic within the timeout period | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Omer Schwartz <oschwart> |
| Component: | openstack-octavia | Assignee: | Gregory Thiemonge <gthiemon> |
| Status: | CLOSED ERRATA | QA Contact: | Bruna Bonguardo <bbonguar> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 17.0 (Wallaby) | CC: | gthiemon, ihrachys, lpeer, majopela, scohen |
| Target Milestone: | Alpha | Keywords: | Triaged |
| Target Release: | 17.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | openstack-octavia-8.0.2-0.20220329110858.a16f516.el9ost | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-09-21 12:18:59 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Omer Schwartz
2022-02-14 09:52:56 UTC
It looks like all the requests were forwarded to the same member:
2022-02-11 14:03:06,693 318518 DEBUG [octavia_tempest_plugin.tests.validators] Loadbalancer wait for load balancer response totals: {'1': 25636}
Need to investigate, but this open review https://review.opendev.org/c/openstack/octavia/+/828606 fixes a similar issue with ipv6 members
One potential related issue that I reproduced in my env: Load balancer loses connectivity to its members when adding a new member (observed in the IPv6 Scenario tests) The IPv6 tests trigger a race condition in the management of the member ports in Octavia. I. Reproduction steps: Using the same load balancer: test 1: 1. Create member on subnet A A port on subnet A is attached to the amphora 2. Create member on subnet B A port on subnet B is attached to the amphora 3. Delete members Ports are not updated test 2: 4. Create member on subnet A Port is already attached, but Octavia notices that a port on subnet B is not used, so it unplugs this port 5. Create member on subnet B Octavia gets the list of ports attached to the amphora, it should create a port on subnet B but the port on subnet B is still in the list, so it doesn't update the ports 6. Port is removed from the list of attached ports and is missing from the amphora, the load balancer cannot create a connection to subnet B II. Details Unplugging a port from a server can take many seconds (the deletion of the port from the server is fast, but it takes up to 8 seconds on OSP for the changes to be committed in the DB) and Octavia doesn't wait for the removal to be completed. So if a new member is added between the unplug API call and the effective deletion from the DB, it may leave the amphora with a bad network configuration. Waiting for the DB update in Octavia would fix this issue, but it would also increase the duration of the member create flow. Change proposed in https://review.opendev.org/c/openstack/octavia/+/829805 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Release of components for Red Hat OpenStack Platform 17.0 (Wallaby)), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2022:6543 |