1816657 – [OSP16][ml2-ovs] neutron-keepalived-qrouter containers are still exist after deletion of associated router and network.

Bug 1816657 - [OSP16][ml2-ovs] neutron-keepalived-qrouter containers are still exist after deletion of associated router and network.

Summary: [OSP16][ml2-ovs] neutron-keepalived-qrouter containers are still exist after ...

Keywords:
Status:	CLOSED EOL
Alias:	None
Product:	Red Hat OpenStack
Classification:	Red Hat
Component:	openstack-tripleo-heat-templates
Sub Component:
Version:	16.0 (Train)
Hardware:	Unspecified
OS:	Unspecified
Priority:	low
Severity:	low
Target Milestone:	---
Target Release:	---
Assignee:	Bernard Cafarelli
QA Contact:	Eran Kuris
Docs Contact:
URL:
Whiteboard:
Duplicates (1):	1829757 (view as bug list)
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2020-03-24 13:32 UTC by Pradipta Kumar Sahoo
Modified:	2023-12-15 17:33 UTC (History)
CC List:	10 users (show)
Fixed In Version:	openstack-tripleo-heat-templates-11.3.2-0.20200405044626.ec9970c.el8ost
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2020-11-04 08:20:11 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
OpenStack gerrit	730744	0	None	MERGED	Fix sending SIGTERM to the sidecar containers	2020-11-03 17:56:44 UTC

Description Pradipta Kumar Sahoo 2020-03-24 13:32:51 UTC

Test: Migration test from ml2-ovs to ml2-ovn in scale lab

Description of problem:
During the below rally workload test in the ml2-ovs environment, we noticed "neutron-keepalived-qrouter-xxx" containers are still existing in controller nodes after cleaned up all associated neutron tenant VMs, routers, and networks.

Version-Release number of selected component (if applicable):

Red Hat OpenStack Platform release 16.0.1 (Train)
Red Hat Enterprise Linux release 8.1 (Ootpa)

5a131f8863b5  undercloud.ctlplane.localdomain:8787/rh-osbs/rhosp16-openstack-neutron-l3-agent:20200226.1           /usr/sbin/keepali...  11 minutes ago  Up 11 minutes ago         neutron-keepalived-qrouter-5d171f79-c4fd-4270-ae6b-f354b8313bd4
1b38bec4dda0  undercloud.ctlplane.localdomain:8787/rh-osbs/rhosp16-openstack-neutron-openvswitch-agent:20200226.1  kolla_start           12 days ago     Up 13 minutes ago         neutron_ovs_agent
5f702294ea40  undercloud.ctlplane.localdomain:8787/rh-osbs/rhosp16-openstack-neutron-metadata-agent:20200226.1     kolla_start           12 days ago     Up 13 minutes ago         neutron_metadata_agent
f086e7ac1c40  undercloud.ctlplane.localdomain:8787/rh-osbs/rhosp16-openstack-neutron-l3-agent:20200226.1           kolla_start           12 days ago     Up 13 minutes ago         neutron_l3_agent
97ade9a5d87b  undercloud.ctlplane.localdomain:8787/rh-osbs/rhosp16-openstack-neutron-dhcp-agent:20200226.1         kolla_start           12 days ago     Up 13 minutes ago         neutron_dhcp
21672d199775  undercloud.ctlplane.localdomain:8787/rh-osbs/rhosp16-openstack-neutron-server:20200226.1             kolla_start           12 days ago     Up 13 minutes ago         neutron_api


How reproducible:
100% reproduced in OpenStack performance scale lab

Steps to Reproduce:
1. Ran the following brobeat scenario:

2020-03-24 11:24:58,793 - browbeat.rally -    INFO - Running Scenario: netcreate-boot-ping
2020-03-24 11:24:58,793 - browbeat.rally -    INFO - Running with scenario_args: {"image_name": "rhel8", "flavor_name": "m1.small", "num_vms": 10, "ext_net_id": "c0925254-7729-43cd-ae70-3f60d53361d8", "times": 100, "concurrency": 16}


2. Browbeat "rally" test result:

+-----------------------------------------------------------------------------------------------------------------------------------------+
|                                                          Response Times (sec)                                                           |
+--------------------------------------+-----------+--------------+--------------+--------------+-----------+-----------+---------+-------+
| Action                               | Min (sec) | Median (sec) | 90%ile (sec) | 95%ile (sec) | Max (sec) | Avg (sec) | Success | Count |
+--------------------------------------+-----------+--------------+--------------+--------------+-----------+-----------+---------+-------+
| neutron.create_router                | 6.818     | 7.631        | 10.868       | 17.609       | 20.8      | 8.713     | 100.0%  | 100   |
| neutron.create_network               | 0.16      | 0.227        | 0.405        | 0.472        | 0.688     | 0.272     | 100.0%  | 100   |
| neutron.create_subnet                | 0.682     | 0.985        | 1.306        | 1.689        | 2.377     | 1.036     | 100.0%  | 100   |
| neutron.add_interface_router         | 3.059     | 3.553        | 4.766        | 5.189        | 5.91      | 3.726     | 100.0%  | 100   |
| nova.boot_server (x10)               | 83.388    | 112.93       | 140.5        | 143.066      | 149.501   | 114.561   | 100.0%  | 99    |
| vm.attach_floating_ip (x10)          | 53.907    | 58.059       | 61.525       | 62.485       | 65.201    | 58.392    | 100.0%  | 99    |
|  -> neutron.create_floating_ip (x10) | 17.66     | 19.317       | 20.681       | 21.672       | 22.886    | 19.454    | 100.0%  | 99    |
|  -> nova.associate_floating_ip (x10) | 34.806    | 38.649       | 41.68        | 42.952       | 45.676    | 38.938    | 100.0%  | 99    |
|  --> neutron.list_ports (x10)        | 0.631     | 0.769        | 1.013        | 1.102        | 1.218     | 0.807     | 100.0%  | 99    |
| vm.wait_for_ping (x10)               | 32.165    | 44.567       | 58.444       | 64.483       | 72.591    | 46.503    | 100.0%  | 99    |
| nova.boot_server (x5)                | 50.279    | 50.279       | 50.279       | 50.279       | 50.279    | 50.279    | 100.0%  | 1     |
| vm.attach_floating_ip (x5)           | 89.939    | 89.939       | 89.939       | 89.939       | 89.939    | 89.939    | 0.0%    | 1     |
|  -> neutron.create_floating_ip (x5)  | 9.567     | 9.567        | 9.567        | 9.567        | 9.567     | 9.567     | 100.0%  | 1     |
|  -> nova.associate_floating_ip (x5)  | 80.372    | 80.372       | 80.372       | 80.372       | 80.372    | 80.372    | 0.0%    | 1     |
|  --> neutron.list_ports (x5)         | 0.592     | 0.592        | 0.592        | 0.592        | 0.592     | 0.592     | 100.0%  | 1     |
| vm.wait_for_ping (x4)                | 33.221    | 33.221       | 33.221       | 33.221       | 33.221    | 33.221    | 100.0%  | 1     |
| <no-name-action>                     | 0.0       | 0            | 0            | 0            | 0.0       | 0.0       | 0.0%    | 1     |
| total                                | 186.67    | 233.486      | 255.626      | 258.234      | 265.533   | 233.804   | 99.0%   | 100   |
|  -> duration                         | 181.67    | 223.486      | 245.626      | 248.234      | 255.533   | 223.854   | 99.0%   | 100   |
|  -> idle_duration                    | 5.0       | 10.0         | 10.0         | 10.0         | 10.0      | 9.95      | 99.0%   | 100   |
+--------------------------------------+-----------+--------------+--------------+--------------+-----------+-----------+---------+-------+

Reference link:
 https://github.com/cloud-bulldozer/browbeat/blob/master/browbeat-config.yaml#L296-L302
 https://github.com/cloud-bulldozer/browbeat/blob/master/rally/rally-plugins/netcreate-boot/netcreate_nova_boot_fip_ping.py
 
3. Associated network & routers created by rally was deleted cleanup step.

(overcloud) [stack@undercloud ~]$ openstack router list | grep s_rally_
(overcloud) [stack@undercloud ~]$ openstack network lsit | grep s_rally_

4. But while namespace status in three controllers, we noticed the neutron-keepalived-qrouter containers still exist.


$ ansible -i hosts controller --become -m shell -a 'podman ps|grep neutron-keepalived-qrouter|wc -l'

controller-1 | CHANGED | rc=0 >>
102

controller-2 | CHANGED | rc=0 >>
102

controller-0 | CHANGED | rc=0 >>
102


5. After rebooted the three controller nodes, all those unwanted "neutron-keepalived-qrouter" namespace containers are cleaned.

$ ansible -i hosts controller --become -m shell -a 'podman ps|grep neutron-keepalived-qrouter|wc -l'
controller-1 | CHANGED | rc=0 >>
2

controller-2 | CHANGED | rc=0 >>
2

controller-0 | CHANGED | rc=0 >>
2

Actual results:
neutron-keepalived-qrouter-xxxx namespace containers are not cleaned up after the deletion of the associated router.

Expected results:
neutron-keepalived-qrouter-xxxx namespace containers should be cleaned up after the deletion of the associated router.

Regards,
Pradipta

Comment 2 Bernard Cafarelli 2020-04-30 13:55:52 UTC

Pasting results from similar bug #1829757

Description of problem:
Stale neutron-keepalived-qrouter containers after the router delete in OVS deployed RHSOP-16

Version-Release number of selected component (if applicable):
Red Hat Openstack Platform - 16 (RHOSP-16)

How reproducible:
Always

Steps to Reproduce:
1. Create a router
2. Attach a gateway and detach
3. Now, delete the router

Actual results:
neutron-keepalived-qrouter-$ID present in UP state on the controllers nodes

Expected results:
neutron-keepalived-qrouter-$ID containers also deleted with router delete

Additional info:
(overcloud) [stack@machine-16 ~]$ openstack network list
+--------------------------------------+----------------------------------------------------+--------------------------------------+
| ID                                   | Name                                               | Subnets                              |
+--------------------------------------+----------------------------------------------------+--------------------------------------+
| 09881516-cd27-45bb-8eeb-25de315fe8ee | HA network tenant 671470eaddec445db420e9cf5bddd969 | cee2c86e-2aab-46e0-940a-6e4aa60df233 |
| 8a0de287-0c31-494d-b221-72752914b266 | testnetwork                                        | 8826a9f1-4beb-4bef-bac4-bae29037090f |
+--------------------------------------+----------------------------------------------------+--------------------------------------+
(overcloud) [stack@machine-16 ~]$ 
(overcloud) [stack@machine-16 ~]$ openstack subnet list
+--------------------------------------+---------------------------------------------------+--------------------------------------+------------------+
| ID                                   | Name                                              | Network                              | Subnet           |
+--------------------------------------+---------------------------------------------------+--------------------------------------+------------------+
| 8826a9f1-4beb-4bef-bac4-bae29037090f | testsubnet                                        | 8a0de287-0c31-494d-b221-72752914b266 | 192.168.122.0/24 |
| cee2c86e-2aab-46e0-940a-6e4aa60df233 | HA subnet tenant 671470eaddec445db420e9cf5bddd969 | 09881516-cd27-45bb-8eeb-25de315fe8ee | 169.254.192.0/18 |
+--------------------------------------+---------------------------------------------------+--------------------------------------+------------------+

(overcloud) [stack@machine-16 ~]$ openstack router list
+--------------------------------------+-------+--------+-------+----------------------------------+------+
| ID                                   | Name  | Status | State | Project                          | HA   |
+--------------------------------------+-------+--------+-------+----------------------------------+------+
| 253b1c97-4105-4bf6-9346-374b34fb92ad | route | ACTIVE | UP    | 671470eaddec445db420e9cf5bddd969 | True |
+--------------------------------------+-------+--------+-------+----------------------------------+------+
(overcloud) [stack@machine-16 ~]$


(undercloud) [stack@machine-16 ~]$ nova list
+--------------------------------------+-------------------------+--------+------------+-------------+------------------------+
| ID                                   | Name                    | Status | Task State | Power State | Networks               |
+--------------------------------------+-------------------------+--------+------------+-------------+------------------------+
| 6b6cce33-c2a8-43be-9cd5-fbce94c1add0 | overcloud-controller-0  | ACTIVE | -          | Running     | ctlplane=192.168.24.11 |
| 9220f843-dd8c-41d5-a1c9-ad88d148ead7 | overcloud-controller-1  | ACTIVE | -          | Running     | ctlplane=192.168.24.15 |
| 7fad735e-076a-4e29-a6e6-251b7fea79ba | overcloud-controller-2  | ACTIVE | -          | Running     | ctlplane=192.168.24.21 |
| 55c88777-19bb-4402-9a09-b6599b6d328c | overcloud-novacompute-0 | ACTIVE | -          | Running     | ctlplane=192.168.24.7  |
+--------------------------------------+-------------------------+--------+------------+-------------+------------------------+
(undercloud) [stack@machine-16 ~]$


(overcloud) [stack@machine-16 ~]$ ssh heat-admin.24.11 "sudo podman ps | grep -i neutron"
ee70a6a377a5  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-l3-agent:16.0-95           /usr/sbin/keepali...  7 minutes ago      Up 7 minutes ago             neutron-keepalived-qrouter-253b1c97-4105-4bf6-9346-374b34fb92ad
b6715a07ab99  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-dhcp-agent:16.0-94         /usr/sbin/dnsmasq...  About an hour ago  Up About an hour ago         neutron-dnsmasq-qdhcp-8a0de287-0c31-494d-b221-72752914b266
f33d4da542f3  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-openvswitch-agent:16.0-94  kolla_start           About an hour ago  Up About an hour ago         neutron_ovs_agent
9621b3fe1898  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-metadata-agent:16.0-95     kolla_start           About an hour ago  Up About an hour ago         neutron_metadata_agent
ea13d04586a9  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-l3-agent:16.0-95           kolla_start           About an hour ago  Up About an hour ago         neutron_l3_agent
f02c8d307e48  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-dhcp-agent:16.0-94         kolla_start           About an hour ago  Up About an hour ago         neutron_dhcp
11a933635eed  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-server:16.0-94             kolla_start           2 hours ago        Up 2 hours ago               neutron_api
(overcloud) [stack@machine-16 ~]$ 


(overcloud) [stack@machine-16 ~]$ ssh heat-admin.24.15 "sudo podman ps | grep -i neutron"
515d7b57d109  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-l3-agent:16.0-95           /bin/bash -c HAPR...  7 minutes ago      Up 7 minutes ago             neutron-haproxy-qrouter-253b1c97-4105-4bf6-9346-374b34fb92ad
edddc818df24  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-l3-agent:16.0-95           /usr/sbin/keepali...  7 minutes ago      Up 7 minutes ago             neutron-keepalived-qrouter-253b1c97-4105-4bf6-9346-374b34fb92ad
736f825342b3  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-dhcp-agent:16.0-94         /usr/sbin/dnsmasq...  About an hour ago  Up About an hour ago         neutron-dnsmasq-qdhcp-8a0de287-0c31-494d-b221-72752914b266
3a55cce49e0b  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-openvswitch-agent:16.0-94  kolla_start           About an hour ago  Up About an hour ago         neutron_ovs_agent
55258961cb36  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-metadata-agent:16.0-95     kolla_start           About an hour ago  Up About an hour ago         neutron_metadata_agent
0ec4ddf9aa39  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-l3-agent:16.0-95           kolla_start           About an hour ago  Up About an hour ago         neutron_l3_agent
00e3e24c5107  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-dhcp-agent:16.0-94         kolla_start           About an hour ago  Up About an hour ago         neutron_dhcp
c469edaae51a  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-server:16.0-94             kolla_start           2 hours ago        Up 2 hours ago               neutron_api
(overcloud) [stack@machine-16 ~]$ 
(overcloud) [stack@machine-16 ~]$ 


(overcloud) [stack@machine-16 ~]$ ssh heat-admin.24.21 "sudo podman ps | grep -i neutron"
922358f8cc13  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-l3-agent:16.0-95           /usr/sbin/keepali...  7 minutes ago      Up 7 minutes ago             neutron-keepalived-qrouter-253b1c97-4105-4bf6-9346-374b34fb92ad
eaedd62297a4  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-dhcp-agent:16.0-94         /usr/sbin/dnsmasq...  About an hour ago  Up About an hour ago         neutron-dnsmasq-qdhcp-8a0de287-0c31-494d-b221-72752914b266
f6c508d26299  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-openvswitch-agent:16.0-94  kolla_start           About an hour ago  Up About an hour ago         neutron_ovs_agent
1a3a1498ea85  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-metadata-agent:16.0-95     kolla_start           About an hour ago  Up About an hour ago         neutron_metadata_agent
955e6c47dd80  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-l3-agent:16.0-95           kolla_start           About an hour ago  Up About an hour ago         neutron_l3_agent
2699e7b00475  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-dhcp-agent:16.0-94         kolla_start           About an hour ago  Up About an hour ago         neutron_dhcp
4d892b53d526  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-server:16.0-94             kolla_start           2 hours ago        Up 2 hours ago               neutron_api



openstack router unset --external-gateway route

(overcloud) [stack@machine-16 ~]$ openstack router delete route
(overcloud) [stack@machine-16 ~]$ 
(overcloud) [stack@machine-16 ~]$ openstack router list

(overcloud) [stack@machine-16 ~]$ 
(overcloud) [stack@machine-16 ~]$ ssh heat-admin.24.21 "sudo podman ps | grep -i neutron"
922358f8cc13  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-l3-agent:16.0-95           /usr/sbin/keepali...  14 minutes ago     Up 14 minutes ago            neutron-keepalived-qrouter-253b1c97-4105-4bf6-9346-374b34fb92ad
eaedd62297a4  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-dhcp-agent:16.0-94         /usr/sbin/dnsmasq...  About an hour ago  Up About an hour ago         neutron-dnsmasq-qdhcp-8a0de287-0c31-494d-b221-72752914b266
f6c508d26299  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-openvswitch-agent:16.0-94  kolla_start           2 hours ago        Up 2 hours ago               neutron_ovs_agent
1a3a1498ea85  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-metadata-agent:16.0-95     kolla_start           2 hours ago        Up 2 hours ago               neutron_metadata_agent
955e6c47dd80  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-l3-agent:16.0-95           kolla_start           2 hours ago        Up 2 hours ago               neutron_l3_agent
2699e7b00475  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-dhcp-agent:16.0-94         kolla_start           2 hours ago        Up 2 hours ago               neutron_dhcp
4d892b53d526  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-server:16.0-94             kolla_start           2 hours ago        Up 2 hours ago               neutron_api
(overcloud) [stack@machine-16 ~]$ 
(overcloud) [stack@machine-16 ~]$ 
(overcloud) [stack@machine-16 ~]$ ssh heat-admin.24.21 "sudo podman ps | grep -i neutron"
922358f8cc13  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-l3-agent:16.0-95           /usr/sbin/keepali...  14 minutes ago     Up 14 minutes ago            neutron-keepalived-qrouter-253b1c97-4105-4bf6-9346-374b34fb92ad
eaedd62297a4  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-dhcp-agent:16.0-94         /usr/sbin/dnsmasq...  About an hour ago  Up About an hour ago         neutron-dnsmasq-qdhcp-8a0de287-0c31-494d-b221-72752914b266
f6c508d26299  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-openvswitch-agent:16.0-94  kolla_start           2 hours ago        Up 2 hours ago               neutron_ovs_agent
1a3a1498ea85  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-metadata-agent:16.0-95     kolla_start           2 hours ago        Up 2 hours ago               neutron_metadata_agent
955e6c47dd80  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-l3-agent:16.0-95           kolla_start           2 hours ago        Up 2 hours ago               neutron_l3_agent
2699e7b00475  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-dhcp-agent:16.0-94         kolla_start           2 hours ago        Up 2 hours ago               neutron_dhcp
4d892b53d526  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-server:16.0-94             kolla_start           2 hours ago        Up 2 hours ago               neutron_api
(overcloud) [stack@machine-16 ~]$ 
(overcloud) [stack@machine-16 ~]$ ssh heat-admin.24.14 "sudo podman ps | grep -i neutron"
^C
(overcloud) [stack@machine-16 ~]$ ssh heat-admin.24.15 "sudo podman ps | grep -i neutron"
edddc818df24  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-l3-agent:16.0-95           /usr/sbin/keepali...  14 minutes ago     Up 14 minutes ago            neutron-keepalived-qrouter-253b1c97-4105-4bf6-9346-374b34fb92ad
736f825342b3  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-dhcp-agent:16.0-94         /usr/sbin/dnsmasq...  About an hour ago  Up About an hour ago         neutron-dnsmasq-qdhcp-8a0de287-0c31-494d-b221-72752914b266
3a55cce49e0b  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-openvswitch-agent:16.0-94  kolla_start           2 hours ago        Up 2 hours ago               neutron_ovs_agent
55258961cb36  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-metadata-agent:16.0-95     kolla_start           2 hours ago        Up 2 hours ago               neutron_metadata_agent
0ec4ddf9aa39  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-l3-agent:16.0-95           kolla_start           2 hours ago        Up 2 hours ago               neutron_l3_agent
00e3e24c5107  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-dhcp-agent:16.0-94         kolla_start           2 hours ago        Up 2 hours ago               neutron_dhcp
c469edaae51a  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-server:16.0-94             kolla_start           2 hours ago        Up 2 hours ago               neutron_api
(overcloud) [stack@machine-16 ~]$ 
(overcloud) [stack@machine-16 ~]$ 
(overcloud) [stack@machine-16 ~]$ ssh heat-admin.24.11 "sudo podman ps | grep -i neutron"
ee70a6a377a5  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-l3-agent:16.0-95           /usr/sbin/keepali...  14 minutes ago     Up 14 minutes ago            neutron-keepalived-qrouter-253b1c97-4105-4bf6-9346-374b34fb92ad
b6715a07ab99  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-dhcp-agent:16.0-94         /usr/sbin/dnsmasq...  About an hour ago  Up About an hour ago         neutron-dnsmasq-qdhcp-8a0de287-0c31-494d-b221-72752914b266
f33d4da542f3  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-openvswitch-agent:16.0-94  kolla_start           2 hours ago        Up 2 hours ago               neutron_ovs_agent
9621b3fe1898  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-metadata-agent:16.0-95     kolla_start           2 hours ago        Up 2 hours ago               neutron_metadata_agent
ea13d04586a9  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-l3-agent:16.0-95           kolla_start           2 hours ago        Up 2 hours ago               neutron_l3_agent
f02c8d307e48  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-dhcp-agent:16.0-94         kolla_start           2 hours ago        Up 2 hours ago               neutron_dhcp
11a933635eed  machine-16.ctlplane.localdomain:8787/rhosp-rhel8/openstack-neutron-server:16.0-94             kolla_start           2 hours ago        Up 2 hours ago               neutron_api
(overcloud) [stack@machine-16 ~]$ 
(overcloud) [stack@machine-16 ~]$ 
(overcloud) [stack@machine-16 ~]$ date
Tue 28 Apr 06:02:36 EDT 2020
(overcloud) [stack@machine-16 ~]$
~~~

~~
(overcloud) [stack@machine-16 ~]$ cat /etc/rhosp-release 
Red Hat OpenStack Platform release 16.0.0 Beta (Train)
(overcloud) [stack@machine-16 ~]$ 
~~

Comment 3 Bernard Cafarelli 2020-04-30 13:56:17 UTC

*** Bug 1829757 has been marked as a duplicate of this bug. ***

Comment 7 Lon Hohberger 2020-07-23 10:57:37 UTC

According to our records, this should be resolved by openstack-tripleo-heat-templates-11.3.2-0.20200405044627.ec9970c.el8ost.  This build is available now.

Note You need to log in before you can comment on or make changes to this bug.