Bug 1051036
Summary: | neutron-l3-agent doesn't clean after itself when service is shut down | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Product: | Red Hat OpenStack | Reporter: | Miguel Angel Ajo <majopela> | ||||||||||
Component: | openstack-neutron | Assignee: | Miguel Angel Ajo <mangelajo> | ||||||||||
Status: | CLOSED ERRATA | QA Contact: | Ofer Blaut <oblaut> | ||||||||||
Severity: | high | Docs Contact: | |||||||||||
Priority: | high | ||||||||||||
Version: | 4.0 | CC: | amuller, breeler, chrisw, e.vnn1, fdinitto, lpeer, sputhenp, yeylon | ||||||||||
Target Milestone: | z4 | Keywords: | ZStream | ||||||||||
Target Release: | 4.0 | ||||||||||||
Hardware: | Unspecified | ||||||||||||
OS: | Unspecified | ||||||||||||
Whiteboard: | |||||||||||||
Fixed In Version: | openstack-neutron-2013.2.2-9.el6ost | Doc Type: | Bug Fix | ||||||||||
Doc Text: |
Cause: Neutron l3 agent is known to not to clean up resources (netns, iptables, processes, etc.) when the service is stopped. This is a feature, intended to allow upgrades to the agent without service disruption.
Consequence: When trying remove a node from the cluster, and stop the services, the l3 services/resources will remain active, but will get updated as soon as there are changes to the served tenant networks.
Fix: Added the neutron-netns-cleanup init script to allow cleanup of the l3 service resources as needed.
Result: The resources can be cleaned up now by running the script.
|
Story Points: | --- | ||||||||||
Clone Of: | 1051028 | Environment: | |||||||||||
Last Closed: | 2014-05-29 20:18:22 UTC | Type: | Bug | ||||||||||
Regression: | --- | Mount Type: | --- | ||||||||||
Documentation: | --- | CRM: | |||||||||||
Verified Versions: | Category: | --- | |||||||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||||||
Embargoed: | |||||||||||||
Bug Depends On: | 1051028, 1062685, 1173435 | ||||||||||||
Bug Blocks: | 1066642, 1080561 | ||||||||||||
Attachments: |
|
Description
Miguel Angel Ajo
2014-01-09 15:30:16 UTC
Launchpad bug#1273095 prevents from properly selecting which kind of namespace we want to cleanup (dhcp or l3-agent). Launchpad bug #1115999 prevents from properly cleaning the metadata-proxies in namespaces (qdhcp or qrouter), that needs to be fixed to have a workaround here. using /etc/init.d/neutron-netns-forced-cleanup start cleans up the network namespaces and all internal iptable rules + interfaces, the fix up is provided in this repo: http://file.rdu.redhat.com/~majopela/neutron-ha-fixes-bz-1051028-and-36-cleanup/ neutron needs to be patched (netns_cleanup script). Created attachment 865582 [details]
Patch for netns_cleanup
Created attachment 865583 [details]
neutron-netns-cleanup script
Created attachment 865584 [details]
neutron-netns-forced-cleanup script
Upstream review on the fixes: https://review.openstack.org/#/c/80261/ Created attachment 879527 [details]
netns cleanup script
Thought to be used in pacemaker.
It does a normal cleanup at start
It does a forced cleanup at stop
Please check https://bugzilla.redhat.com/show_bug.cgi?id=1051028#c11 for details on how the neutron-netns-cleanup script will behave. I have tested that when service neutron-netns-cleanup stop used netns are cleaned The stop conditions are HA related and not script one. openstack-neutron-2013.2.3-4.el6ost.noarch [root@puma05 ~]# ip netns qdhcp-a76e98a5-7ae3-4f91-b721-4f81cebcfa6f qdhcp-6dcaa203-e61a-4003-a1fe-95d60853516f qrouter-15ef1247-b52a-43fc-bfa2-27478dbfe1f3 [root@puma05 ~]# service neutron-netns-cleanup stop [root@puma05 ~]# ip netns [root@puma05 ~]# [root@puma05 ~]# [root@puma05 ~]# service neutron-netns-cleanup start [root@puma05 ~]# ip netns [root@puma05 ~]# openstack-status == neutron services == neutron-server: inactive (disabled on boot) neutron-dhcp-agent: active neutron-l3-agent: active neutron-metadata-agent: active neutron-lbaas-agent: inactive (disabled on boot) neutron-openvswitch-agent: active == Support services == openvswitch: active messagebus: active [root@puma05 ~]# service neutron-dhcp-agent restart Stopping neutron-dhcp-agent: [ OK ] Starting neutron-dhcp-agent: [ OK ] [root@puma05 ~]# service neutron-l3-agent restart Stopping neutron-l3-agent: [ OK ] Starting neutron-l3-agent: [ OK ] [root@puma05 ~]# ip netns qdhcp-a76e98a5-7ae3-4f91-b721-4f81cebcfa6f qdhcp-6dcaa203-e61a-4003-a1fe-95d60853516f [root@puma05 ~]# ip netns qdhcp-a76e98a5-7ae3-4f91-b721-4f81cebcfa6f qdhcp-6dcaa203-e61a-4003-a1fe-95d60853516f qrouter-15ef1247-b52a-43fc-bfa2-27478dbfe1f3 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHSA-2014-0516.html |