Bug 1685705

Summary:	openvswitch agent cannot create iptables rules correctly on RHEL8
Product:	Red Hat OpenStack	Reporter:	Alex Schultz <aschultz>
Component:	openstack-tripleo-heat-templates	Assignee:	Michele Baldessari <michele>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Gurenko Alex <agurenko>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	15.0 (Stein)	CC:	amuller, bfournie, chrisw, mburns, michele, njohnston, scohen
Target Milestone:	---	Keywords:	Triaged
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-03-08 09:55:26 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Alex Schultz 2019-03-05 20:50:33 UTC

Description of problem:

When trying to setup an OSP15 undercloud, the openvswitch agent container cannot create the iptables rules required to forward traffic.

2019-03-05 15:41:56.814 534226 DEBUG neutron.agent.securitygroups_rpc [req-a004e117-dba8-42bf-81fe-abf7a9f2faab - - - - -] Init firewall settings (driver=iptables_hybrid) init_firewall /usr/lib/python3.6/site-packages/neutron/agent/securitygroups_rpc.py:76
2019-03-05 15:41:56.839 534226 DEBUG oslo_concurrency.lockutils [req-a004e117-dba8-42bf-81fe-abf7a9f2faab - - - - -] Lock "conntrack" acquired by "neutron.agent.linux.ip_conntrack.get_conntrack" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:327
2019-03-05 15:41:56.840 534226 DEBUG neutron.agent.linux.utils [req-a004e117-dba8-42bf-81fe-abf7a9f2faab - - - - -] Running command (rootwrap daemon): ['iptables-save', '-t', 'raw'] execute_rootwrap_daemon /usr/lib/python3.6/site-packages/neutron/agent/linux/utils.py:103
2019-03-05 15:41:56.847 534226 DEBUG oslo_rootwrap.client [req-a004e117-dba8-42bf-81fe-abf7a9f2faab - - - - -] Popen for ['sudo', 'neutron-rootwrap-daemon', '/etc/neutron/rootwrap.conf'] command has been instantiated _initialize /usr/lib/python3.6/site-packages/oslo_rootwrap/client.py:73
2019-03-05 15:41:57.078 534226 INFO oslo_rootwrap.client [req-a004e117-dba8-42bf-81fe-abf7a9f2faab - - - - -] Spawned new rootwrap daemon process with pid=534237
2019-03-05 15:41:57.095 534226 ERROR neutron.agent.linux.utils [req-a004e117-dba8-42bf-81fe-abf7a9f2faab - - - - -] Exit code: 1; Stdin: ; Stdout: Table `raw' does not exist
; Stderr: 
2019-03-05 15:41:57.096 534226 DEBUG oslo_concurrency.lockutils [req-a004e117-dba8-42bf-81fe-abf7a9f2faab - - - - -] Lock "conntrack" released by "neutron.agent.linux.ip_conntrack.get_conntrack" :: held 0.257s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339
2019-03-05 15:41:57.096 534226 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-a004e117-dba8-42bf-81fe-abf7a9f2faab - - - - -] Exit code: 1; Stdin: ; Stdout: Table `raw' does not exist
; Stderr:  Agent terminated!: neutron_lib.exceptions.ProcessExecutionError: Exit code: 1; Stdin: ; Stdout: Table `raw' does not exist
2019-03-05 15:41:57.098 534226 INFO oslo_rootwrap.client [-] Stopping rootwrap daemon process with pid=534237


Version-Release number of selected component (if applicable):
openstack-neutron-openvswitch-14.0.0-0.20190226194817.d1c2f77.el8ost.noarch

How reproducible:
every time

Steps to Reproduce:
1. install undercloud
2. systemctl restart tripleo_neutron_ovs_agent
3. 

Actual results:
No iptables rules are created by the ovs agent

Expected results:
iptables rules are correctly created

Additional info:

Comment 2 Alex Schultz 2019-03-05 23:43:17 UTC

This is why the ovs agent keeps restarting.  I commented out https://git.openstack.org/cgit/openstack/neutron/tree/neutron/agent/linux/ip_conntrack.py#n75 in the container and it stopped restarting and created the iptables. It seems that iptables-save it raw doesn't work with the nftables backend even though there is a raw nftables.

Comment 4 Nate Johnston 2019-03-07 16:45:46 UTC

U/S change merged