Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1658688

Summary:	Octavia health manager doesn't receive heartbeat message from amp on composable role
Product:	Red Hat OpenStack	Reporter:	Carlos Goncalves <cgoncalves>
Component:	openstack-tripleo-heat-templates	Assignee:	Carlos Goncalves <cgoncalves>
Status:	CLOSED ERRATA	QA Contact:	Alexander Stafeyev <astafeye>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	14.0 (Rocky)	CC:	gfidente, mariel, mburns, rheslop, sputhenp
Target Milestone:	z1	Keywords:	Triaged, ZStream
Target Release:	14.0 (Rocky)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-tripleo-heat-templates-9.0.1-0.20181013060909.el7ost	Doc Type:	Bug Fix
Doc Text:	Previously, the Octavia Health Manager did not receive heartbeat messages from amphorae in Octavia composable role deployments. This caused the firewall on Network nodes to drop packets, and the the load balancers did not return an ONLINE operating status. After this fix, the Octavia Health Manager now receives heartbeat message from amphorae, and the load balancers can go online.	Story Points:	---
Clone Of:
Clones:	1659077 (view as bug list)		Environment:
Last Closed:	2019-03-18 13:03:25 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1659077

Description Carlos Goncalves 2018-12-12 16:38:12 UTC

Description of problem: the Octavia health manager service does not receive heartbeat messages sent from amphorae. Firewall on networker node drops packets. Firewall rule (protocol UDP, port 5555, interface o-hm0) is missing.

How reproducible:
100%

Steps to Reproduce:
1. create new load balancer
2. wait for provisioning_status to go ACTIVE
3. observe that operating_status will never go ONLINE

Actual results:
operating_status stays OFFLINE forever

Expected results:
operating_status to flip to ONLINE

Additional info:
This only affects composable Octavia deployments

Comment 3 Carlos Goncalves 2018-12-12 16:55:05 UTC

With missing firewall rule in network node:

(overcloud) [stack@undercloud-0 ~]$ openstack loadbalancer show lb2
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| admin_state_up      | True                                 |
| created_at          | 2018-12-12T15:51:55                  |
| description         |                                      |
| flavor              |                                      |
| id                  | 4417aca4-9252-46eb-b9a0-e64c85d6b69c |
| listeners           |                                      |
| name                | lb2                                  |
| operating_status    | OFFLINE                              |
| pools               |                                      |
| project_id          | 0bde3075a41d45ef95566863121fec69     |
| provider            | octavia                              |
| provisioning_status | ACTIVE                               |
| updated_at          | 2018-12-12T15:53:26                  |
| vip_address         | 10.0.10.18                           |
| vip_network_id      | 1d128f7e-f180-4c47-81ab-252196d0be8e |
| vip_port_id         | 26461b79-3c8f-4659-b294-7d102896ff8f |
| vip_qos_policy_id   | None                                 |
| vip_subnet_id       | b4b66a6a-ecf9-4851-87dd-fdbea352d4fc |
+---------------------+--------------------------------------+


After manually creating firewall rule in network node:

(overcloud) [stack@undercloud-0 ~]$ openstack loadbalancer show lb2
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| admin_state_up      | True                                 |
| created_at          | 2018-12-12T15:51:55                  |
| description         |                                      |
| flavor              |                                      |
| id                  | 4417aca4-9252-46eb-b9a0-e64c85d6b69c |
| listeners           |                                      |
| name                | lb2                                  |
| operating_status    | ONLINE                               |
| pools               |                                      |
| project_id          | 0bde3075a41d45ef95566863121fec69     |
| provider            | octavia                              |
| provisioning_status | ACTIVE                               |
| updated_at          | 2018-12-12T16:02:41                  |
| vip_address         | 10.0.10.18                           |
| vip_network_id      | 1d128f7e-f180-4c47-81ab-252196d0be8e |
| vip_port_id         | 26461b79-3c8f-4659-b294-7d102896ff8f |
| vip_qos_policy_id   | None                                 |
| vip_subnet_id       | b4b66a6a-ecf9-4851-87dd-fdbea352d4fc |
+---------------------+--------------------------------------+

Octavia tempest tests ran on an OSP 13 composable Octavia deployment caught this issue [1]. For example, octavia_tempest_plugin.tests.api.v2.test_listener.ListenerAPITest.test_listener_create failed because of this bug.

[1] https://cgoncalves.pt/trash/openstack/octavia/bf02560a-95b2-4b5e-afa6-17d0e3150903.html

Comment 4 Carlos Goncalves 2018-12-12 21:32:23 UTC

OSP 13 composable Octavia with fix applied locally: https://cgoncalves.pt/trash/openstack/octavia/cc8b80a3-8be4-4807-8183-d9324b5415a4.html
Way better now :-)

Comment 9 errata-xmlrpc 2019-03-18 13:03:25 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0446