Bug 1292587 - neutron-l3-agent cannot correctly create routers on RHEL 7.2
neutron-l3-agent cannot correctly create routers on RHEL 7.2
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-neutron (Show other bugs)
7.0 (Kilo)
All Linux
urgent Severity urgent
: async
: 7.0 (Kilo)
Assigned To: Arie Bregman
Ofer Blaut
: Reopened, Triaged, ZStream
Depends On: 1292570 1292589 1292591
Blocks: 1191185 1235795
  Show dependency treegraph
 
Reported: 2015-12-17 15:50 EST by Assaf Muller
Modified: 2016-04-26 14:17 EDT (History)
14 users (show)

See Also:
Fixed In Version: openstack-neutron-2015.1.2-6.el7ost
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1292570
Environment:
Last Closed: 2016-01-21 08:52:54 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Comment 3 Chandra Ganguly 2016-01-07 12:45:06 EST
Cisco is hitting this issue with Kilo (OSP7) and RHEL7.2

Description of problem: neutron-l3-agent cannot correctly create routers on RHEL 7.2 due to updated 'ip netns' output. New 'ip netns list' output includes an (id: ) field that the l3-agent can't parse:

  [root@mcp-rhel-7 vagrant]# ip netns list
  qrouter-ec16ed02-229c-47dc-a927-63b2d91a8721 (id: 0)

Appears to be fixed in upstream, and I've confirmed the patch addresses the issue: https://review.openstack.org/#/c/258493/1/neutron/agent/linux/ip_lib.py

Version-Release number of selected component (if applicable):  OSP8 Beta (2015-12-03.2)

How reproducible: Consistently

Steps to Reproduce:
1. Install and configure neutron with ml2 and linuxbridge. 
2. Install and configure l3-agent.
3. Attempt to create a router.

Actual results:
Router creation reports success, but log shows errors.

Expected results:
Error free creation of routers, as with above linked patch.

Additional info:

Stack trace from neutron-l3-agent:
2015-12-17 19:52:18.821 20224 ERROR neutron.agent.l3.router_info
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent [-] Failed to process compatible router 'ec16ed02-229c-47dc-a927-63b2d91a8721'
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent Traceback (most recent call last):
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 498, in _process_router_update
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     self._process_router_if_compatible(router)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 441, in _process_router_if_compatible
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     self._process_added_router(router)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 449, in _process_added_router
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     ri.process(self)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/common/utils.py", line 359, in call
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     self.logger(e)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     six.reraise(self.type_, self.value, self.tb)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/common/utils.py", line 356, in call
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     return func(*args, **kwargs)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 694, in process
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     self.process_external(agent)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 660, in process_external
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     self._process_external_gateway(ex_gw_port, agent.pd)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 569, in _process_external_gateway
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     self.external_gateway_added(ex_gw_port, interface_name)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 534, in external_gateway_added
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     ex_gw_port, interface_name, self.ns_name, preserve_ips)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 496, in _external_gateway_added
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     self._plug_external_gateway(ex_gw_port, interface_name, ns_name)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 477, in _plug_external_gateway
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     prefix=EXTERNAL_DEV_PREFIX)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 252, in plug
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     bridge, namespace, prefix)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 483, in plug_new
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     namespace2=namespace)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 144, in add_veth
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     self.ensure_namespace(namespace2)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 163, in ensure_namespace
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     ip = self.netns.add(name)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 793, in add
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     self._as_root([], ('add', name), use_root_namespace=True)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 280, in _as_root
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     use_root_namespace=use_root_namespace)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 80, in _as_root
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     log_fail_as_error=self.log_fail_as_error)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 89, in _execute
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     log_fail_as_error=log_fail_as_error)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent     raise RuntimeError(m)
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent RuntimeError:
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent Command: ['ip', 'netns', 'add', u'qrouter-ec16ed02-229c-47dc-a927-63b2d91a8721']
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent Exit code: 1
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent Stdin:
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent Stdout:
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent Stderr: Cannot create namespace file "/var/run/netns/qrouter-ec16ed02-229c-47dc-a927-63b2d91a8721": File exists
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent
2015-12-17 19:52:18.824 20224 ERROR neutron.agent.l3.agent
Comment 7 Ofer Blaut 2016-01-20 04:59:28 EST
I have create router on L3_HA setup, 
listed the host holding the router and check the logs, no error is found

Tested with OVS

openstack-neutron-2015.1.2-6.el7ost.noarch
Comment 9 errata-xmlrpc 2016-01-21 08:52:54 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:0060

Note You need to log in before you can comment on or make changes to this bug.