Red Hat Bugzilla – Bug 1133920
'openstack-service restart neutron' uses neutron cleanup scripts
Last modified: 2016-01-04 09:46:19 EST
Description of problem: LBaaS doesn't work after upgrade to Icehouse, because the old process of haproxy still alive after the upgrade instead of being stopped and restarted after the upgrade. Version-Release number of selected component (if applicable): From: http://download.eng.bos.redhat.com/rel-eng/OpenStack/4.0/2014-08-15.1/ To: http://download.eng.bos.redhat.com/rel-eng/OpenStack/5.0-RHEL-6/2014-08-22.1/ How reproducible: Tested once. Steps to Reproduce: 1.Install Havana with LB 2.Configure pool/VIP/members/healthmonitor 3.The load balancer works properly 4.Upgrade via http://docbuilder.usersys.redhat.com/22898/ 4.1 Method All at once 5.LB doesn't work. Workaround: Kill the old haproxy process and restart the lbaas_agent.
The issue here is that device with IP of created member in lbaas disappears from lbaas's namespace. Workaround is to kill old haproxy process and start lbaas-agent again. haproxy conf file: frontend 351fee2f-fb77-4333-9d9b-36d424680d38 option tcplog bind 10.0.0.4:80 mode http default_backend ddec1997-653b-4b96-891d-9c5010e51a18 option forwardfor [root@localhost ~]# ip netns exec qlbaas-ddec1997-653b-4b96-891d-9c5010e51a18 haproxy -f /var/lib/neutron/lbaas/ddec1997-653b-4b96-891d-9c5010e51a18/conf -p /var/lib/neutron/lbaas/ddec1997-653b-4b96-891d-9c5010e51a18/pid -sf 69264 [ALERT] 208/062930 (77444) : Starting frontend 351fee2f-fb77-4333-9d9b-36d424680d38: cannot bind sock [root@localhost ~]# ip netns exec qlbaas-ddec1997-653b-4b96-891d-9c5010e51a18 ip a 232: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever
The issue here is because cleanup scripts remove tap devices from network namespaces. Administrator should be aware of actions he takes. openstack-service uses services that are enabled on certain runlevels and cleanup is ran on boot which makes it to be run silently in openstack-service.
So, this is also affecting l3-ha (VRRP). Restarting neutron with this tool breaks all ha routers (all are set to fault) I think the main problem is that neutorn-ovs-cleaup is packed as a service while it is a script that shouldn't be executed against an active neutron env.
Can you please provide the steps needed to verify this bug?
(In reply to Itzik Brown from comment #15) > Can you please provide the steps needed to verify this bug? Just make sure that 'openstack-service restart' doesn't run neutron cleanup scripts and namespaces, tap devices and tunnels are persistent across the 'openstack-service restart'.
Checked with openstack-utils-2014.2-1.el7ost.noarch
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2015-0825.html