Bug 1382413

Summary:	agent traces about bridge-nf-call sysctl values missing in RHEL 7.3
Product:	Red Hat OpenStack	Reporter:	Ihar Hrachyshka <ihrachys>
Component:	openstack-neutron	Assignee:	Ihar Hrachyshka <ihrachys>
Status:	CLOSED ERRATA	QA Contact:	GenadiC <gcheresh>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	8.0 (Liberty)	CC:	amuller, chrisw, ihrachys, nyechiel, rhos-flags, sclewis, srevivo, tfreger
Target Milestone:	async	Keywords:	Triaged, ZStream
Target Release:	8.0 (Liberty)
Hardware:	Unspecified
OS:	Linux
Whiteboard:
Fixed In Version:	openstack-neutron-7.1.1-8.el7ost	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:	1382412	Environment:
Last Closed:	2016-12-21 16:45:12 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Bug Depends On:	1382412, 1382414, 1382416, 1382417
Bug Blocks:

Description Ihar Hrachyshka 2016-10-06 15:35:37 UTC

+++ This bug was initially created as a clone of Bug #1382412 +++

2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent Traceback (most recent call last):
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 450, in daemon_loop
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent sync = self.process_network_devices(device_info)
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/local/lib/python2.7/dist-packages/osprofiler/profiler.py", line 154, in wrapper
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent return f(*args, **kwargs)
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 200, in process_network_devices
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent device_info.get('updated'))
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 265, in setup_port_filters
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent self.prepare_devices_filter(new_devices)
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 130, in decorated_function
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent *args, **kwargs)
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 138, in prepare_devices_filter
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent self._apply_port_filter(device_ids)
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 163, in _apply_port_filter
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent self.firewall.prepare_port_filter(device)
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/stack/new/neutron/neutron/agent/linux/iptables_firewall.py", line 170, in prepare_port_filter
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent self._enable_netfilter_for_bridges()
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/stack/new/neutron/neutron/agent/linux/iptables_firewall.py", line 114, in _enable_netfilter_for_bridges
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent run_as_root=True)
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/stack/new/neutron/neutron/agent/linux/utils.py", line 138, in execute
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent raise RuntimeError(msg)
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent RuntimeError: Exit code: 255; Stdin: ; Stdout: ; Stderr: sysctl: cannot stat /proc/sys/net/bridge/bridge-nf-call-arptables: No such file or directory
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent
2016-09-13 07:37:33.437 13401 ERROR neutron.plugins.ml2.drivers.agent._common_agent

This should be solved by loading br_netfilter kernel module in RHEL 7.3 before starting the agent.

--- Additional comment from Ihar Hrachyshka on 2016-10-06 11:32:50 EDT ---

Note: this is solved in OSP10 by the virtue of the patch: https://review.rdoproject.org/r/3114 that should arrive on next delorean sync into OSP.

Comment 4 GenadiC 2016-11-20 11:52:58 UTC

Please provide steps to verify this bug

Comment 5 Ihar Hrachyshka 2016-11-22 13:26:01 UTC

Something like: restart node; start the OVS/Linuxbridge agent with iptables firewall enabled; check that no traceback is in the logs; check that br_netfilter kernel module is loaded.

Comment 6 Ihar Hrachyshka 2016-11-22 15:12:34 UTC

OK, one more step to trigger the error is to actually land an instance on the compute node in question. Without it, the OVS agent will not attempt to configure any iptables rules for ports because there are no ports to configure in the first place.

Comment 7 GenadiC 2016-11-22 15:59:32 UTC

Verified in openstack-neutron-7.2.0-5.el7ost.noarch

Comment 9 errata-xmlrpc 2016-12-21 16:45:12 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2988.html