Bug 1125207 – neutron.plugins.openvswitch.agent.ovs_neutron_agent Stderr: "ip6tables-restore v1.4.7: ip6tables-restore: unable to initialize table 'filter'\n\nError occurred at line: 2\n

Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.

Bug 1125207 - neutron.plugins.openvswitch.agent.ovs_neutron_agent Stderr: "ip6tables-restore v1.4.7: ip6tables-restore: unable to initialize table 'filter'\n\nError occurred at line: 2\n

Summary:	neutron.plugins.openvswitch.agent.ovs_neutron_agent Stderr: "ip6tables-restor...

Status:	CLOSED ERRATA

Aliases:	None

Product:	Red Hat OpenStack
Classification:	Red Hat
Component:	openstack-neutron (Show other bugs)
Sub Component:
Version:	4.0
Hardware:	Unspecified Unspecified

Priority	medium Severity medium
Target Milestone:	z2
Target Release:	5.0 (RHEL 7)
Assigned To:	Jakub Libosvar
QA Contact:	Roey Dekel
Docs Contact:

URL:
Whiteboard:
Keywords:	ZStream

Duplicates:	1002569 (view as bug list)
Depends On:
Blocks:	1150898
	Show dependency tree / graph

Reported:	2014-07-31 06:26 EDT by Eduard Barrera
Modified:	2018-02-08 05:06 EST (History)
CC List:	8 users (show)

See Also:
Fixed In Version:	openstack-neutron-2014.1.2-5.el6ost openstack-neutron-2014.1.2-3.el7ost
Doc Type:	Bug Fix
Doc Text:
Story Points:	---
Clone Of:
Clones:	1150898 (view as bug list)
Environment:
Last Closed:	2014-11-03 03:38:28 EST
Type:	Bug
Regression:	---
Mount Type:	---
Documentation:	---
CRM:
Verified Versions:
Category:	---
oVirt Team:	---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---

Attachments	(Terms of Use)
Add an attachment (proposed patch, testcase, etc.)

External Trackers
Tracker	ID	Priority	Status	Summary	Last Updated
Launchpad	1352893	None	None	None	Never
OpenStack gerrit	112253	None	None	None	Never
Red Hat Product Errata	RHSA-2014:1786	normal	SHIPPED_LIVE	Moderate: openstack-neutron security, bug fix, and enhancement update	2014-11-03 08:36:33 EST

Groups: None (edit)

Description Eduard Barrera 2014-07-31 06:26:18 EDT

Description of problem:

I completly disabled IPv6 after installing packstack. And now I'm having this error flooding the audit.log and openvswitch-agent.log 

modules/ipv6.conf
------------------
options ipv6 disable=1
install ipv6 /bin/true
blacklist ipv6
alias ipv6 off


openvswitch-agent.log 
---------------------
2014-07-31 11:45:16.039 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent 
2014-07-31 11:45:16.040 1891 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Agent out of sync with plugin!
2014-07-31 11:45:16.041 1891 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Agent tunnel out of sync with plugin!
2014-07-31 11:45:17.933 1891 INFO neutron.agent.securitygroups_rpc [-] Preparing filters for devices set([u'085562f6-de05-4294-984c-bdd5b8448b88', u'aca468f0-7518-4ec0-bcba-dc819d961248', u'c8338884-adc6-477f-bbe6-1f459a401083'])
2014-07-31 11:45:18.445 1891 ERROR neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Error in agent event loop
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Traceback (most recent call last):
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 1081, in rpc_loop
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     sync = self.process_network_ports(port_info)
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 1011, in process_network_ports
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     resync_a = self.treat_devices_added(port_info['added'])
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 912, in treat_devices_added
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.sg_agent.prepare_devices_filter(devices)
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/securitygroups_rpc.py", line 123, in prepare_devices_filter
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.firewall.prepare_port_filter(device)
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.gen.next()
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/firewall.py", line 107, in defer_apply
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.filter_defer_apply_off()
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_firewall.py", line 353, in filter_defer_apply_off
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.iptables.defer_apply_off()
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_manager.py", line 346, in defer_apply_off
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self._apply()
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/openstack/common/lockutils.py", line 233, in inner
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     retval = f(*args, **kwargs)
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_manager.py", line 382, in _apply
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     root_helper=self.root_helper)
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/utils.py", line 75, in execute
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     raise RuntimeError(m)
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent RuntimeError: 
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip6tables-restore', '-c']
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Exit code: 2
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Stdout: ''
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Stderr: "ip6tables-restore v1.4.7: ip6tables-restore: unable to initialize table 'filter'\n\nError occurred at line: 2\nTry `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.\n"
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent 
2014-07-31 11:45:18.446 1891 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Agent out of sync with plugin!
2014-07-31 11:45:18.446 1891 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Agent tunnel out of sync with plugin!



This is how the file grows in 10 seconds

# > openvswitch-agent.log; ll openvswitch-agent.log ; sleep 10; ll openvswitch-agent.log
-rw-r--r--. 1 neutron neutron 0 Jul 31 11:42 openvswitch-agent.log
-rw-r--r--. 1 neutron neutron 15482 Jul 31 11:42 openvswitch-agent.log


Version-Release number of selected component (if applicable):

openstack-nova-conductor-2013.2.2-2.el6ost.noarch
openstack-nova-cert-2013.2.2-2.el6ost.noarch
openstack-neutron-openvswitch-2013.2.2-1.el6ost.noarch
redhat-access-plugin-openstack-4.0.0-0.el6ost.noarch
openstack-keystone-2013.2.2-1.el6ost.noarch
openstack-cinder-2013.2.2-1.el6ost.noarch
openstack-utils-2013.2-3.el6ost.noarch
openstack-nova-api-2013.2.2-2.el6ost.noarch
openstack-nova-compute-2013.2.2-2.el6ost.noarch
openstack-nova-scheduler-2013.2.2-2.el6ost.noarch
openstack-dashboard-2013.2.2-1.el6ost.noarch
openstack-ceilometer-central-2013.2.2-1.el6ost.noarch
openstack-ceilometer-collector-2013.2.2-1.el6ost.noarch
openstack-ceilometer-api-2013.2.2-1.el6ost.noarch
openstack-ceilometer-compute-2013.2.2-1.el6ost.noarch
openstack-nova-novncproxy-2013.2.2-2.el6ost.noarch
openstack-neutron-2013.2.2-1.el6ost.noarch
openstack-packstack-2013.2.1-0.25.dev987.el6ost.noarch
python-django-openstack-auth-1.1.2-2.el6ost.noarch
openstack-dashboard-theme-2013.2.2-1.el6ost.noarch
openstack-selinux-0.1.3-2.el6ost.noarch
openstack-ceilometer-alarm-2013.2.2-1.el6ost.noarch
sos-plugins-openstack-2013.2-5.el6ost.noarch
openstack-glance-2013.2.2-2.el6ost.noarch
openstack-nova-common-2013.2.2-2.el6ost.noarch
openstack-ceilometer-common-2013.2.2-1.el6ost.noarch
openstack-nova-console-2013.2.2-2.el6ost.noarch


commenting the entries on modules/ipv6.conf and executing modprobe ipv6 fixes the issue.

I didn't found any way to disable the ipv6 behavior in neutron by any conf option


This is the cpu (vm 1 core) usage during the issue:

Tasks: 142 total,   2 running, 140 sleeping,   0 stopped,   0 zombie
Cpu(s): 78.7%us, 20.7%sy,  0.0%ni,  0.3%id,  0.3%wa,  0.0%hi,  0.0%si,  0.0%st

This is cpu usage after modprobe ipv6

Tasks: 142 total,   1 running, 141 sleeping,   0 stopped,   0 zombie
Cpu(s):  4.1%us,  1.4%sy,  0.0%ni, 85.0%id,  9.5%wa,  0.0%hi,  0.0%si,  0.0%st

Comment 2 Jakub Libosvar 2014-08-18 11:55:45 EDT

*** Bug 1002569 has been marked as a duplicate of this bug. ***

Comment 6 Roey Dekel 2014-10-14 08:49:14 EDT

Verified on Icehouse with:

Version-Release number of selected component:
---------------------------------------------
Red Hat Enterprise Linux Server release 7.0 (Maipo)
puddle: 2014-10-10.4
openstack-neutron-ml2-2014.1.3-4.el7ost.noarch

Reproduce Steps:
----------------
1. Edit file /etc/sysctl.conf, enter the following command:
    1.1	[root@devops ~]# vim /etc/sysctl.conf

2. Adding the below lines into that file:
    2.1	net.ipv6.conf.all.disable_ipv6 = 1
    2.2	net.ipv6.conf.default.disable_ipv6 = 1
    2.3	net.ipv6.conf.lo.disable_ipv6 = 1

3. Save and close that file, then restart sysctl with the following command:
    3.1	[root@devops ~]# sysctl -p

4. check “disable_ipv6″ file to check if IPv6 is disabled,enter:
    4.1	[root@devops ~]# cat /proc/sys/net/ipv6/conf/all/disable_ipv6
    4.2	1

Results:
--------
No errors in audit.log nor openvswitch-agent.log

Comment 8 errata-xmlrpc 2014-11-03 03:38:28 EST

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2014-1786.html

Note You need to log in before you can comment on or make changes to this bug.