Bug 1175251

Summary:	neutron-netns-cleanup OCF doesn't clean up radvd or keepalived
Product:	Red Hat OpenStack	Reporter:	Miguel Angel Ajo <majopela>
Component:	openstack-neutron	Assignee:	Miguel Angel Ajo <majopela>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Asaf Hirshberg <ahirshbe>
Severity:	high	Docs Contact:
Priority:	high
Version:	6.0 (Juno)	CC:	bperkins, chrisw, lpeer, majopela, nyechiel, oblaut, yeylon, yfried
Target Milestone:	ga
Target Release:	6.0 (Juno)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-neutron-2014.2.1-5.el7ost	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2015-02-10 15:14:13 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1177026

Description Miguel Angel Ajo 2014-12-17 11:14:26 UTC

Description of problem:

Neutron netns-cleanup doesn't clean up the radvd or keepalived
services.

It does for metadata proxy D/S because we use a killall neutron-ns-metadata-proxy on the init.d/OCF resource scripts


How reproducible:
Always

Steps to Reproduce:
1. Create a router in neutron when l3_ha is enabled
2. A keepalived will show up when there are instances connected to the router
3. stop neutron-l3-agent
4. /usr/lib/ocf/lib/neutron/neutron-netns-cleanup start
5. /usr/lib/ocf/lib/neutron/neutron-netns-cleanup stop

Actual results:

keepalived is running.

Expected results:

keepalived is gone.

Additional info:

If we setup an internal network with IPv6 and SLAAC we may have radvd spawn , and killed by neutron-netns-cleanup the same way.

We have a D/S workaround to implement by calling killall in the netns-cleanup-script, but we should fix this upstream as soon as possible too in the netns-cleanup.

Comment 7 Miguel Angel Ajo 2015-01-13 18:55:12 UTC


Hi Ofer,

   Please test using those commands which are what we're using on HA (where netns-cleanup is relevant)


4. /usr/lib/ocf/lib/neutron/neutron-netns-cleanup start

5. /usr/lib/ocf/lib/neutron/neutron-netns-cleanup stop

We included a workaround to do this in our scripts, and we created
an upstream bug for a proper fix upstream.

Comment 8 Miguel Angel Ajo 2015-01-13 18:58:35 UTC

I have modified the topic to make clear that our main concern D/S are the OCF scripts for pacemaker.

And we are working on 
https://bugs.launchpad.net/bugs/1403455

via plain neutron-netns-cleanup to have the proper long term bug fix.

Comment 10 Ofer Blaut 2015-01-28 08:03:56 UTC

I have tested it on packstack (radvd & keepalived) , and also keepalive on staypuft 

[root@networker-ebefeb24-214f-4133-aee8-4b40ac37e2a4 ~]# ps aux | grep keep
root       950  0.0  0.0 110004   848 ttyS0    Ss+  Jan20   0:00 /sbin/agetty --keep-baud ttyS0 115200 38400 9600
root      5856  0.0  0.0 112084  1324 ?        Ss   Jan20   0:48 keepalived -P -f /var/lib/neutron/ha_confs/32bd618b-9d88-43b3-83b2-b8f7a8e6e0af/keepalived.conf -p /var/lib/neutron/ha_confs/32bd618b-9d88-43b3-83b2-b8f7a8e6e0af.pid -r /var/lib/neutron/ha_confs/32bd618b-9d88-43b3-83b2-b8f7a8e6e0af.pid-vrrp
root      5857  0.0  0.0 118440  2556 ?        S    Jan20   1:34 keepalived -P -f /var/lib/neutron/ha_confs/32bd618b-9d88-43b3-83b2-b8f7a8e6e0af/keepalived.conf -p /var/lib/neutron/ha_confs/32bd618b-9d88-43b3-83b2-b8f7a8e6e0af.pid -r /var/lib/neutron/ha_confs/32bd618b-9d88-43b3-83b2-b8f7a8e6e0af.pid-vrrp
root     24656  0.0  0.0 112644   956 pts/0    R+   02:52   0:00 grep --color=auto keep
[root@networker-ebefeb24-214f-4133-aee8-4b40ac37e2a4 ~]# systemctl stop neutron-l3-agent
[root@networker-ebefeb24-214f-4133-aee8-4b40ac37e2a4 ~]# ps aux | grep radvd
root      6007  0.0  0.0   8536   140 ?        S    Jan20   0:00 radvd -C /var/lib/neutron/ra/32bd618b-9d88-43b3-83b2-b8f7a8e6e0af.radvd.conf -p /var/lib/neutron/external/pids/32bd618b-9d88-43b3-83b2-b8f7a8e6e0af.pid.radvd
root      6009  0.0  0.0   8536   704 ?        S    Jan20   1:11 radvd -C /var/lib/neutron/ra/32bd618b-9d88-43b3-83b2-b8f7a8e6e0af.radvd.conf -p /var/lib/neutron/external/pids/32bd618b-9d88-43b3-83b2-b8f7a8e6e0af.pid.radvd
root     24705  0.0  0.0 112644   956 pts/0    R+   02:52   0:00 grep --color=auto radvd

[root@networker-ebefeb24-214f-4133-aee8-4b40ac37e2a4 ~]# /usr/lib/ocf/lib/neutron/neutron-netns-cleanup start
[root@networker-ebefeb24-214f-4133-aee8-4b40ac37e2a4 ~]# /usr/lib/ocf/lib/neutron/neutron-netns-cleanup stop
[root@networker-ebefeb24-214f-4133-aee8-4b40ac37e2a4 ~]# 
[root@networker-ebefeb24-214f-4133-aee8-4b40ac37e2a4 ~]# 
[root@networker-ebefeb24-214f-4133-aee8-4b40ac37e2a4 ~]# ps aux | grep radvd
root     25009  0.0  0.0 112640   956 pts/0    S+   02:53   0:00 grep --color=auto radvd
[root@networker-ebefeb24-214f-4133-aee8-4b40ac37e2a4 ~]# ps aux | grep keep
root       950  0.0  0.0 110004   848 ttyS0    Ss+  Jan20   0:00 /sbin/agetty --keep-baud ttyS0 115200 38400 9600
root     25020  0.0  0.0 112640   960 pts/0    R+   02:53   0:00 grep --color=auto keep
[root@networker-ebefeb24-214f-4133-aee8-4b40ac37e2a4 ~]# rpm -qa | grep neutron
openstack-neutron-2014.2.1-6.el7ost.noarch
python-neutron-2014.2.1-6.el7ost.noarch
openstack-neutron-openvswitch-2014.2.1-6.el7ost.noarch
python-neutronclient-2.3.9-1.el7ost.noarch
openstack-neutron-metering-agent-2014.2.1-6.el7ost.noarch

Comment 11 Scott Lewis 2015-02-10 15:14:13 UTC

This bug has been closed as a part of the RHEL-OSP 6 general availability release. For details, see https://rhn.redhat.com/errata/rhel7-rhos-6-errata.html