Bug 2075118
| Summary: | Octavia fails when enabling TLS-e in existing setup | |||
|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Shailesh Chhabdiya <schhabdi> | |
| Component: | openstack-tripleo-heat-templates | Assignee: | Jakub Libosvar <jlibosva> | |
| Status: | CLOSED ERRATA | QA Contact: | Fiorella Yanac <fyanac> | |
| Severity: | high | Docs Contact: | ||
| Priority: | high | |||
| Version: | 16.2 (Train) | CC: | averdagu, bcafarel, beagles, ebarrera, gthiemon, jlibosva, mburns, pgodwin, rcarrier, rrubins, vkoul | |
| Target Milestone: | z4 | Keywords: | Triaged | |
| Target Release: | 16.2 (Train on RHEL 8.4) | |||
| Hardware: | All | |||
| OS: | All | |||
| Whiteboard: | ||||
| Fixed In Version: | openstack-tripleo-heat-templates-11.6.1-2.20220821010130.b1e9bfe.el8ost | Doc Type: | No Doc Update | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 2100906 2100907 (view as bug list) | Environment: | ||
| Last Closed: | 2022-12-07 19:22:17 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | 2100906 | |||
| Bug Blocks: | 2100907 | |||
|
Description
Shailesh Chhabdiya
2022-04-13 16:12:15 UTC
Hello Team, Details are present in description Issue is reproduced twice and at same setup Here are detailed steps we performed 1. deploying stack form scratch with SSL on public endpoints only 2. Running an update with TLS-e The initial deployment (2) finished successfully. The TLS-e update (3) failed at exactly the same step as previously, that is, at Octavia. The overcloud was not functional: (overcloud) [stack@dtr01 ~]$ openstack network list HttpException: 503: Server Error for url: https://xxxx:13696/v2.0/networks, 503 Service Unavailable: No server is available to handle this request. Restarting ovn and octavia containers on all controllers with sudo podman restart $(sudo podman ps -f name=ovn -f name=octavia --format="{{.Names}}") didn't bring the overcloud back to being functional and above error when listing networks persisted. Also the same ovn errors as previously were present: ----- ctl01 ----- 2022-04-12T15:19:36Z|00001|stream_ssl|ERR|Private key must be configured to use SSL 2022-04-12T15:19:36Z|00002|stream_ssl|ERR|Certificate must be configured to use SSL 2022-04-12T15:19:36Z|00003|stream_ssl|ERR|CA certificate must be configured to use SSL ovn-nbctl: ssl:192.168.211.240:6641: database connection failed (Protocol not available) ----- ctl02 ----- 2022-04-12T15:19:37Z|00001|stream_ssl|ERR|Private key must be configured to use SSL 2022-04-12T15:19:37Z|00002|stream_ssl|ERR|Certificate must be configured to use SSL 2022-04-12T15:19:37Z|00003|stream_ssl|ERR|CA certificate must be configured to use SSL ovn-nbctl: ssl:192.168.211.240:6641: database connection failed (Protocol not available) ----- ctl03 ----- 2022-04-12T15:19:38Z|00001|stream_ssl|ERR|Private key must be configured to use SSL 2022-04-12T15:19:38Z|00002|stream_ssl|ERR|Certificate must be configured to use SSL 2022-04-12T15:19:38Z|00003|stream_ssl|ERR|CA certificate must be configured to use SSL ovn-nbctl: ssl:192.168.211.240:6641: database connection failed (Protocol not available) Having restarted the containers, I ran the update again, and it finished successfully: 2022-04-12 17:06:34.076894 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ End Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Ansible passed.Overcloud configuration completed The overcloud look operational. Verified creating and deleting a network, subnet and port successfully. However, the ovn errors still persist: ----- ctl01 ----- 2022-04-13T07:26:41Z|00001|stream_ssl|ERR|Private key must be configured to use SSL 2022-04-13T07:26:41Z|00002|stream_ssl|ERR|Certificate must be configured to use SSL 2022-04-13T07:26:41Z|00003|stream_ssl|ERR|CA certificate must be configured to use SSL ovn-nbctl: ssl:192.168.211.240:6641: database connection failed (Protocol not available) ----- ctl02 ----- 2022-04-13T07:26:42Z|00001|stream_ssl|ERR|Private key must be configured to use SSL 2022-04-13T07:26:42Z|00002|stream_ssl|ERR|Certificate must be configured to use SSL 2022-04-13T07:26:42Z|00003|stream_ssl|ERR|CA certificate must be configured to use SSL ovn-nbctl: ssl:192.168.211.240:6641: database connection failed (Protocol not available) ----- ctl03 ----- 2022-04-13T07:26:43Z|00001|stream_ssl|ERR|Private key must be configured to use SSL 2022-04-13T07:26:43Z|00002|stream_ssl|ERR|Certificate must be configured to use SSL 2022-04-13T07:26:43Z|00003|stream_ssl|ERR|CA certificate must be configured to use SSL ovn-nbctl: ssl:192.168.211.240:6641: database connection failed (Protocol not available) Apparently, there is an issue in the TLS-e update process related to certificates (sort of race condition?), which can be recreated. So ideally we have a workaround but a permanent fix is needed Thank You Well, the update fails when the Octavia playbook executes an OpenStack command: "openstack network show lb-mgmt-net" It doesn't seem related to Octavia because the customer also reports: (overcloud) [stack@dtr01 ~]$ openstack network list HttpException: 503: Server Error for url: https://xxxx:13696/v2.0/networks, 503 Service Unavailable: No server is available to handle this request. Adding Squad:Neutron Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Release of components for Red Hat OpenStack Platform 16.2.4), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:8794 |