Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1133920

Summary: 'openstack-service restart neutron' uses neutron cleanup scripts
Product: Red Hat OpenStack Reporter: Toni Freger <tfreger>
Component: openstack-utilsAssignee: Pádraig Brady <pbrady>
Status: CLOSED ERRATA QA Contact: Itzik Brown <itbrown>
Severity: high Docs Contact:
Priority: high    
Version: 5.0 (RHEL 6)CC: adahms, bperkins, chrisw, ddomingo, jlibosva, nyechiel, oblaut, pbrady, scohen, yeylon, yfried
Target Milestone: z4Keywords: Rebase, ZStream
Target Release: 5.0 (RHEL 7)   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: openstack-utils-2014.2-1.el6ost, openstack-utils-2014.2-1.el7ost Doc Type: Known Issue
Doc Text:
If an existing haproxy process was already running before installing and running LBaaS (Load-Balancing-as-a-Service), attempting to start LBaaS will fail. This typically happens when upgrading to Red Hat Enterprise Linux OpenStack Platform 5 with an existing LBaaS service. To work around this, you will have to kill the running haproxy process and restart the LBaaS agent: # kill $(pgrep haproxy) # service neutron-lbaas-agent restart
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-04-16 14:37:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Toni Freger 2014-08-26 12:47:22 UTC
Description of problem:
LBaaS doesn't work after upgrade to Icehouse, because the old process of haproxy still alive after the upgrade instead of being stopped and restarted after the upgrade.

Version-Release number of selected component (if applicable):
From: http://download.eng.bos.redhat.com/rel-eng/OpenStack/4.0/2014-08-15.1/
To: http://download.eng.bos.redhat.com/rel-eng/OpenStack/5.0-RHEL-6/2014-08-22.1/

How reproducible:
Tested once.

Steps to Reproduce:
1.Install Havana with LB
2.Configure pool/VIP/members/healthmonitor 
3.The load balancer works properly
4.Upgrade via http://docbuilder.usersys.redhat.com/22898/
 4.1  Method All at once
5.LB doesn't work.



Workaround:

Kill the old haproxy process and restart the lbaas_agent.

Comment 2 Jakub Libosvar 2014-08-26 17:02:25 UTC
The issue here is that device with IP of created member in lbaas disappears from lbaas's namespace. Workaround is to kill old haproxy process and start lbaas-agent again.

haproxy conf file:
frontend 351fee2f-fb77-4333-9d9b-36d424680d38
        option tcplog
        bind 10.0.0.4:80
        mode http
        default_backend ddec1997-653b-4b96-891d-9c5010e51a18
        option forwardfor


[root@localhost ~]# ip netns exec qlbaas-ddec1997-653b-4b96-891d-9c5010e51a18 haproxy -f /var/lib/neutron/lbaas/ddec1997-653b-4b96-891d-9c5010e51a18/conf -p /var/lib/neutron/lbaas/ddec1997-653b-4b96-891d-9c5010e51a18/pid -sf 69264
[ALERT] 208/062930 (77444) : Starting frontend 351fee2f-fb77-4333-9d9b-36d424680d38: cannot bind sock

[root@localhost ~]# ip netns exec qlbaas-ddec1997-653b-4b96-891d-9c5010e51a18 ip a
232: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever

Comment 7 Jakub Libosvar 2014-08-28 16:20:00 UTC
The issue here is because cleanup scripts remove tap devices from network namespaces. Administrator should be aware of actions he takes. openstack-service uses services that are enabled on certain runlevels and cleanup is ran on boot which makes it to be run silently in openstack-service.

Comment 13 yfried 2014-12-08 15:27:41 UTC
So, this is also affecting l3-ha (VRRP).
Restarting neutron with this tool breaks all ha routers (all are set to fault)

I think the main problem is that neutorn-ovs-cleaup is packed as a service while it is a script that shouldn't be executed against an active neutron env.

Comment 15 Itzik Brown 2015-03-29 11:15:14 UTC
Can you please provide the steps needed to verify this bug?

Comment 16 Jakub Libosvar 2015-03-31 13:28:28 UTC
(In reply to Itzik Brown from comment #15)
> Can you please provide the steps needed to verify this bug?

Just make sure that 'openstack-service restart' doesn't run neutron cleanup scripts and namespaces, tap devices and tunnels are persistent across the 'openstack-service restart'.

Comment 17 Itzik Brown 2015-03-31 13:57:06 UTC
Checked with openstack-utils-2014.2-1.el7ost.noarch

Comment 19 errata-xmlrpc 2015-04-16 14:37:22 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0825.html