Bug 1125207 - neutron.plugins.openvswitch.agent.ovs_neutron_agent Stderr: "ip6tables-restore v1.4.7: ip6tables-restore: unable to initialize table 'filter'\n\nError occurred at line: 2\n
Summary: neutron.plugins.openvswitch.agent.ovs_neutron_agent Stderr: "ip6tables-restor...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-neutron
Version: 4.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: z2
: 5.0 (RHEL 7)
Assignee: Jakub Libosvar
QA Contact: Roey Dekel
URL:
Whiteboard:
: 1002569 (view as bug list)
Depends On:
Blocks: 1150898
TreeView+ depends on / blocked
 
Reported: 2014-07-31 10:26 UTC by Eduard Barrera
Modified: 2018-12-03 21:10 UTC (History)
8 users (show)

Fixed In Version: openstack-neutron-2014.1.2-5.el6ost openstack-neutron-2014.1.2-3.el7ost
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1150898 (view as bug list)
Environment:
Last Closed: 2014-11-03 08:38:28 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2014:1786 normal SHIPPED_LIVE Moderate: openstack-neutron security, bug fix, and enhancement update 2014-11-03 13:36:33 UTC
OpenStack gerrit 112253 None None None Never
Launchpad 1352893 None None None Never

Description Eduard Barrera 2014-07-31 10:26:18 UTC
Description of problem:

I completly disabled IPv6 after installing packstack. And now I'm having this error flooding the audit.log and openvswitch-agent.log 

modules/ipv6.conf
------------------
options ipv6 disable=1
install ipv6 /bin/true
blacklist ipv6
alias ipv6 off


openvswitch-agent.log 
---------------------
2014-07-31 11:45:16.039 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent 
2014-07-31 11:45:16.040 1891 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Agent out of sync with plugin!
2014-07-31 11:45:16.041 1891 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Agent tunnel out of sync with plugin!
2014-07-31 11:45:17.933 1891 INFO neutron.agent.securitygroups_rpc [-] Preparing filters for devices set([u'085562f6-de05-4294-984c-bdd5b8448b88', u'aca468f0-7518-4ec0-bcba-dc819d961248', u'c8338884-adc6-477f-bbe6-1f459a401083'])
2014-07-31 11:45:18.445 1891 ERROR neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Error in agent event loop
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Traceback (most recent call last):
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 1081, in rpc_loop
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     sync = self.process_network_ports(port_info)
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 1011, in process_network_ports
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     resync_a = self.treat_devices_added(port_info['added'])
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 912, in treat_devices_added
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.sg_agent.prepare_devices_filter(devices)
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/securitygroups_rpc.py", line 123, in prepare_devices_filter
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.firewall.prepare_port_filter(device)
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.gen.next()
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/firewall.py", line 107, in defer_apply
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.filter_defer_apply_off()
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_firewall.py", line 353, in filter_defer_apply_off
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.iptables.defer_apply_off()
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_manager.py", line 346, in defer_apply_off
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self._apply()
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/openstack/common/lockutils.py", line 233, in inner
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     retval = f(*args, **kwargs)
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_manager.py", line 382, in _apply
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     root_helper=self.root_helper)
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/utils.py", line 75, in execute
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     raise RuntimeError(m)
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent RuntimeError: 
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip6tables-restore', '-c']
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Exit code: 2
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Stdout: ''
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Stderr: "ip6tables-restore v1.4.7: ip6tables-restore: unable to initialize table 'filter'\n\nError occurred at line: 2\nTry `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.\n"
2014-07-31 11:45:18.445 1891 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent 
2014-07-31 11:45:18.446 1891 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Agent out of sync with plugin!
2014-07-31 11:45:18.446 1891 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Agent tunnel out of sync with plugin!



This is how the file grows in 10 seconds

# > openvswitch-agent.log; ll openvswitch-agent.log ; sleep 10; ll openvswitch-agent.log
-rw-r--r--. 1 neutron neutron 0 Jul 31 11:42 openvswitch-agent.log
-rw-r--r--. 1 neutron neutron 15482 Jul 31 11:42 openvswitch-agent.log


Version-Release number of selected component (if applicable):

openstack-nova-conductor-2013.2.2-2.el6ost.noarch
openstack-nova-cert-2013.2.2-2.el6ost.noarch
openstack-neutron-openvswitch-2013.2.2-1.el6ost.noarch
redhat-access-plugin-openstack-4.0.0-0.el6ost.noarch
openstack-keystone-2013.2.2-1.el6ost.noarch
openstack-cinder-2013.2.2-1.el6ost.noarch
openstack-utils-2013.2-3.el6ost.noarch
openstack-nova-api-2013.2.2-2.el6ost.noarch
openstack-nova-compute-2013.2.2-2.el6ost.noarch
openstack-nova-scheduler-2013.2.2-2.el6ost.noarch
openstack-dashboard-2013.2.2-1.el6ost.noarch
openstack-ceilometer-central-2013.2.2-1.el6ost.noarch
openstack-ceilometer-collector-2013.2.2-1.el6ost.noarch
openstack-ceilometer-api-2013.2.2-1.el6ost.noarch
openstack-ceilometer-compute-2013.2.2-1.el6ost.noarch
openstack-nova-novncproxy-2013.2.2-2.el6ost.noarch
openstack-neutron-2013.2.2-1.el6ost.noarch
openstack-packstack-2013.2.1-0.25.dev987.el6ost.noarch
python-django-openstack-auth-1.1.2-2.el6ost.noarch
openstack-dashboard-theme-2013.2.2-1.el6ost.noarch
openstack-selinux-0.1.3-2.el6ost.noarch
openstack-ceilometer-alarm-2013.2.2-1.el6ost.noarch
sos-plugins-openstack-2013.2-5.el6ost.noarch
openstack-glance-2013.2.2-2.el6ost.noarch
openstack-nova-common-2013.2.2-2.el6ost.noarch
openstack-ceilometer-common-2013.2.2-1.el6ost.noarch
openstack-nova-console-2013.2.2-2.el6ost.noarch


commenting the entries on modules/ipv6.conf and executing modprobe ipv6 fixes the issue.

I didn't found any way to disable the ipv6 behavior in neutron by any conf option


This is the cpu (vm 1 core) usage during the issue:

Tasks: 142 total,   2 running, 140 sleeping,   0 stopped,   0 zombie
Cpu(s): 78.7%us, 20.7%sy,  0.0%ni,  0.3%id,  0.3%wa,  0.0%hi,  0.0%si,  0.0%st

This is cpu usage after modprobe ipv6

Tasks: 142 total,   1 running, 141 sleeping,   0 stopped,   0 zombie
Cpu(s):  4.1%us,  1.4%sy,  0.0%ni, 85.0%id,  9.5%wa,  0.0%hi,  0.0%si,  0.0%st

Comment 2 Jakub Libosvar 2014-08-18 15:55:45 UTC
*** Bug 1002569 has been marked as a duplicate of this bug. ***

Comment 6 Roey Dekel 2014-10-14 12:49:14 UTC
Verified on Icehouse with:

Version-Release number of selected component:
---------------------------------------------
Red Hat Enterprise Linux Server release 7.0 (Maipo)
puddle: 2014-10-10.4
openstack-neutron-ml2-2014.1.3-4.el7ost.noarch

Reproduce Steps:
----------------
1. Edit file /etc/sysctl.conf, enter the following command:
    1.1	[root@devops ~]# vim /etc/sysctl.conf

2. Adding the below lines into that file:
    2.1	net.ipv6.conf.all.disable_ipv6 = 1
    2.2	net.ipv6.conf.default.disable_ipv6 = 1
    2.3	net.ipv6.conf.lo.disable_ipv6 = 1

3. Save and close that file, then restart sysctl with the following command:
    3.1	[root@devops ~]# sysctl -p

4. check “disable_ipv6″ file to check if IPv6 is disabled,enter:
    4.1	[root@devops ~]# cat /proc/sys/net/ipv6/conf/all/disable_ipv6
    4.2	1

Results:
--------
No errors in audit.log nor openvswitch-agent.log

Comment 8 errata-xmlrpc 2014-11-03 08:38:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2014-1786.html


Note You need to log in before you can comment on or make changes to this bug.