1390938 – firewalld should be easily configurable for OVN hosts and OVN central server

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1390938 - firewalld should be easily configurable for OVN hosts and OVN central server

Summary: firewalld should be easily configurable for OVN hosts and OVN central server

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	openvswitch
Sub Component:
Version:	7.0
Hardware:	x86_64
OS:	Linux
Priority:	high
Severity:	medium
Target Milestone:	pre-dev-freeze
Target Release:	7.4
Assignee:	Lance Richardson
QA Contact:	qding
Docs Contact:	Ioanna Gkioka
URL:
Whiteboard:
Depends On:
Blocks:	1366899 1411730 1416748 1427110
TreeView+	depends on / blocked

Reported:	2016-11-02 09:51 UTC by Mor
Modified:	2017-11-10 09:00 UTC (History)
CC List:	15 users (show)
Fixed In Version:	openvswitch-2.6.1-7.git20161206.el7fdb
Doc Type:	Enhancement
Doc Text:	Installation of OVN now supports easily-configurable firewalld rules This feature adds firewalld configuration rules for Open Virtual Network (OVN) to the openvswitch packages. As a result, the user can install easier OVN with firewalld enabled, instead of needing to create firewalld configuration manually.
Clone Of:
Clones:	1416748 1427110 (view as bug list)
Environment:
Last Closed:	2017-07-12 15:36:34 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Description Mor 2016-11-02 09:51:47 UTC

Description of problem:
When installing OVN hosts and central servers on RHEL 7, as a apart of the installation, iptables/firewalld should be configured to allow OVN/OVS communication traffic. For both host->central communication and tunnel overlay between hosts. In addition, there's lack of setup documentation regarding firewall configuration requirements for OVN/OVS environments.

Version-Release number of selected component (if applicable):
OVS: openvswitch-2.6.90-1 
oVirt Engine Version: 4.1.0-0.0.master.20161101211323.git410903b.el7.centos

How reproducible:
100%

Steps to Reproduce:
1. Install oVirt-engine and OVN central server on RHEL 7.
2. Install 2 hosts and OVN controllers on RHEL 7.
3. Check that servers are running iptables/firewalld service by default.
4. Create OVN network.
5. Create 2 VM's and attach the OVN network to the VM's.
6. Run each VM on a different host.
7. Assign static IP's to the VM's.
8. Try to send ICMP ping between them.

Actual results:
OVN traffic is blocked.

Expected results:
OVN traffic should pass.

Additional info:
I tried to capture traffic on the tap interface (on host), but no traffic is passing through.

Comment 1 Red Hat Bugzilla Rules Engine 2016-11-02 09:51:58 UTC

Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 2 Dan Kenigsberg 2016-11-02 10:27:00 UTC

We'd be pleased to have a firewalld "service" exposing the needed ports on central and controllers.

Comment 4 Marcin Mirecki 2016-11-02 11:37:32 UTC

As an example, for the ovirt provider for ovn we give the user the following file:

/etc/firewalld/services/ovirt-provider-ovn.xml

<?xml version="1.0" encoding="utf-8"?>
<service>
  <short>ovirt-provider-ovn</short>
  <description>oVirt provider ovn</description>
  <port protocol="tcp" port="9696"/>
</service>

To add it to firewalld it's enought to run:
firewall-cmd --zone=<zone to add service to> --add-service=ovirt-provider-ovn --permanent

Comment 5 Numan Siddique 2016-12-15 14:54:14 UTC

In our test environment, we added a rule to allow UDP destination port 6081 (for geneve tunnel) in the INPUT table and it worked.

Comment 6 Mor 2016-12-18 11:56:53 UTC

Just to add, if we want to turn on firewalld on the controllers and central (engine in our case), we also need to configure oVirt services ports (e.g.: http://www.ovirt.org/documentation/how-to/faq/ -> "Which network ports should be enabled when setting up oVirt environment?"). For Vdsm, there's a RFE: https://bugzilla.redhat.com/show_bug.cgi?id=995362 without target milestone or target release.

Comment 7 Russell Bryant 2016-12-19 02:42:59 UTC

(In reply to Mor from comment #6)
> Just to add, if we want to turn on firewalld on the controllers and central
> (engine in our case), we also need to configure oVirt services ports (e.g.:
> http://www.ovirt.org/documentation/how-to/faq/ -> "Which network ports
> should be enabled when setting up oVirt environment?"). For Vdsm, there's a
> RFE: https://bugzilla.redhat.com/show_bug.cgi?id=995362 without target
> milestone or target release.

and I think it makes sense to ship some firewalld config files with OVN.  We just wanted to update results of our testing to confirm the firewall rule needed to allow geneve tunnels.

Comment 8 Mor 2017-01-12 12:45:04 UTC

Hello everyone,

I made some progress with this issue yesterday, and I managed to get it working with the following firewalld service configuration files:

On OVN central server:
----------------------

<?xml version="1.0" encoding="utf-8"?>
<service>
  <short>ovirt-provider-ovn</short>
  <description>oVirt provider ovn</description>
  <port protocol="tcp" port="9696"/>
  <port protocol="tcp" port="6641"/>
  <port protocol="tcp" port="6442"/>
</service>

On OVN host server:
-------------------

<?xml version="1.0" encoding="utf-8"?>
<service>
  <short>ovirt-provider-ovn-driver</short>
  <description>oVirt provider OVN driver</description>
  <port protocol="udp" port="6081"/>
</service>

Just to make sure, I still want to test it on a clean environment.

Comment 9 Mor 2017-01-19 11:04:50 UTC

(In reply to Mor from comment #8)
> Hello everyone,
> 
> I made some progress with this issue yesterday, and I managed to get it
> working with the following firewalld service configuration files:
> 
> On OVN central server:
> ----------------------
> 
> <?xml version="1.0" encoding="utf-8"?>
> <service>
>   <short>ovirt-provider-ovn</short>
>   <description>oVirt provider ovn</description>
>   <port protocol="tcp" port="9696"/>
>   <port protocol="tcp" port="6641"/>
>   <port protocol="tcp" port="6442"/>
> </service>
> 
> On OVN host server:
> -------------------
> 
> <?xml version="1.0" encoding="utf-8"?>
> <service>
>   <short>ovirt-provider-ovn-driver</short>
>   <description>oVirt provider OVN driver</description>
>   <port protocol="udp" port="6081"/>
> </service>
> 
> Just to make sure, I still want to test it on a clean environment.

Just adding, TCP port 9696 is used by our OVN provider service (which uses OVN).

Comment 10 Marcin Mirecki 2017-01-23 09:36:43 UTC

Note that 9696 is an ovirt related port.
This should be enough:

Central:
 <?xml version="1.0" encoding="utf-8"?>
 <service>
   <short>ovirt-provider-ovn</short>
   <description>oVirt provider ovn</description>
   <port protocol="tcp" port="6641"/>
   <port protocol="tcp" port="6442"/>
 </service>

Host:
 <?xml version="1.0" encoding="utf-8"?>
 <service>
   <short>ovirt-provider-ovn-driver</short>
   <description>oVirt provider OVN driver</description>
   <port protocol="udp" port="6081"/>
 </service>

Comment 11 Lance Richardson 2017-02-10 15:53:30 UTC

Brew build for fd beta package:

https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=12525976

Comment 15 qding 2017-02-28 03:16:01 UTC

Hi Mor,

I know nothing about oVirt. Can you please help verify this BZ?

Thanks
QJ

Comment 16 Mor 2017-03-01 07:55:58 UTC

(In reply to qding from comment #15)
> Hi Mor,
> 
> I know nothing about oVirt. Can you please help verify this BZ?
> 
> Thanks
> QJ

Hi QJ,

Sorry for the delay in response. Do you want to verify it with oVirt and OVN setup? or just with OVN?

Comment 17 Mor 2017-03-01 08:00:07 UTC

(In reply to qding from comment #15)
> Hi Mor,
> 
> I know nothing about oVirt. Can you please help verify this BZ?
> 
> Thanks
> QJ

I will verify it with my own setup of oVirt and OVN.

Comment 18 qding 2017-03-01 09:17:25 UTC

(In reply to Mor from comment #17)
> 
> I will verify it with my own setup of oVirt and OVN.
I hope OVN only if it can be used to verify this issue.

Thanks
QJ

Comment 19 qding 2017-03-01 09:23:04 UTC

Hi Mor,

Sorry for that I should have given more explanation. I hope cover this feature with OVN only if you can give me some suggestions. Thank you very much for help verify this issue, then should I assign QA contact to you?

Thanks
QJ

Comment 20 Mor 2017-03-01 11:48:41 UTC

(In reply to qding from comment #19)
> Hi Mor,
> 
> Sorry for that I should have given more explanation. I hope cover this
> feature with OVN only if you can give me some suggestions. Thank you very
> much for help verify this issue, then should I assign QA contact to you?
> 
> Thanks
> QJ

No problem.

Comment 21 Mor 2017-03-07 16:05:51 UTC

(In reply to qding from comment #19)
> Hi Mor,
> 
> Sorry for that I should have given more explanation. I hope cover this
> feature with OVN only if you can give me some suggestions. Thank you very
> much for help verify this issue, then should I assign QA contact to you?
> 
> Thanks
> QJ

Hi QJ,

Currently, I'm unable to verify it on RHV hypervisors when firewalld is on, because RHV hosts do not support firewalld yet. But in order to make some ticket progress, I want to try and setup OVN environment for it, and try it out with RHV engine and OVN-only hosts.

I will update with my findings.

Thanks.

Comment 22 Mor 2017-03-07 16:31:56 UTC

(In reply to Lance Richardson from comment #11)
> Brew build for fd beta package:
> 
> https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=12525976

Hi Lance, 

1. Can you please clarify which RPM in the candidate build includes the addition of the host firewalld service file? 

2. Is it internal build? Or it is also included in openvswitch-ovn-central-2.6.1-10.git20161206?

Thanks.

Comment 23 Lance Richardson 2017-03-07 17:08:21 UTC

(In reply to Mor from comment #22)
> (In reply to Lance Richardson from comment #11)
> > Brew build for fd beta package:
> > 
> > https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=12525976
> 
> Hi Lance, 
> 
> 1. Can you please clarify which RPM in the candidate build includes the
> addition of the host firewalld service file? 
> 
> 2. Is it internal build? Or it is also included in
> openvswitch-ovn-central-2.6.1-10.git20161206?
> 
> Thanks.

Hi Mor,

Yes, it is in 2.6.1-10.git20161206

That version can be found here:

http://download-node-02.eng.bos.redhat.com/brewroot/packages/openvswitch/2.6.1/10.git20161206.el7fdb/

Comment 24 qding 2017-03-08 03:22:21 UTC

(In reply to Mor from comment #21)
> 
> Currently, I'm unable to verify it on RHV hypervisors when firewalld is on,
> because RHV hosts do not support firewalld yet. But in order to make some
> ticket progress, I want to try and setup OVN environment for it, and try it
> out with RHV engine and OVN-only hosts.
> 
> I will update with my findings.
> 
Thanks

Comment 25 Mor 2017-03-08 14:04:47 UTC

Hi, 

I tried to look for the service file that contains 6081 UDP port (comment #10). Do you know which package provides it on 2.6.1-10? I can't find it.

Comment 26 Lance Richardson 2017-03-08 17:43:52 UTC

(In reply to Mor from comment #25)
> Hi, 
> 
> I tried to look for the service file that contains 6081 UDP port (comment
> #10). Do you know which package provides it on 2.6.1-10? I can't find it.

Hi Mor,

It seems the needed files are indeed not being installed, a fix will be
needed. Unfortunately, since this is on QE already it will likely have
to wait until the next fdb/fdp release (assuming it's not a blocker).

   Lance

Comment 27 Lance Richardson 2017-03-08 20:20:56 UTC

Fix pushed to fd beta and fd production packages. Builds can be found here:

http://download-node-02.eng.bos.redhat.com/brewroot/packages/openvswitch/2.6.1/11.git20161206.el7fdb/

and:

http://download-node-02.eng.bos.redhat.com/brewroot/packages/openvswitch/2.6.1/11.git20161206.el7fdp/

The host rules to allow UDP port 6081 (GENEVE) are included in the
openvswitch-ovn-host RPM:
/usr/lib/firewalld/services/ovn-host-firewall-service.xml

Comment 32 Lance Richardson 2017-03-21 11:53:26 UTC

The original upstream patch, which appears to have been based on
https://bugzilla.redhat.com/show_bug.cgi?id=1390938#c8 , has the
wrong port number (6442) for the southbound db connection.

This patch will need to be backported for the next fd drop:

https://mail.openvswitch.org/pipermail/ovs-dev/2017-March/329532.html

Comment 33 Mor 2017-03-21 14:35:06 UTC

(In reply to Lance Richardson from comment #32)
> The original upstream patch, which appears to have been based on
> https://bugzilla.redhat.com/show_bug.cgi?id=1390938#c8 , has the
> wrong port number (6442) for the southbound db connection.
> 
> This patch will need to be backported for the next fd drop:
> 
> https://mail.openvswitch.org/pipermail/ovs-dev/2017-March/329532.html

This could explain the reason for block. It was a typo. Thanks Lance.

Comment 35 Lance Richardson 2017-03-30 14:23:09 UTC

The patch referenced in https://bugzilla.redhat.com/show_bug.cgi?id=1390938#c32
has been back-ported to fast datapath beta and production packages. RPMs
with this patch are available here:

Fast datapath beta:
http://download-node-02.eng.bos.redhat.com/brewroot/packages/openvswitch/2.6.1/17.git20161206.el7fdb/

Fast datpath production:
http://download-node-02.eng.bos.redhat.com/brewroot/packages/openvswitch/2.6.1/14.git20161206.el7fdp/

Comment 37 qding 2017-06-07 02:30:56 UTC

Verified with openvswitch-2.7.0-7.git20170530.el7fdb.x86_64

[root@dell-per730-04 ovn]# uname -r
3.10.0-673.el7.x86_64
[root@dell-per730-04 ovn]# rpm -q openvswitch
openvswitch-2.7.0-7.git20170530.el7fdb.x86_64
[root@dell-per730-04 ovn]# 
[root@dell-per730-04 ovn]# cat /usr/lib/firewalld/services/ovn-host-firewall-service.xml 
<?xml version="1.0" encoding="utf-8"?>
<service>
  <short>ovn-host-firewall-service</short>
  <description>Firewall service for ovn host</description>
  <port protocol="udp" port="6081"/>
</service>
[root@dell-per730-04 ovn]# cat /usr/lib/firewalld/services/ovn-central-firewall-service.xml 
<?xml version="1.0" encoding="utf-8"?>
<service>
  <short>ovn-central-firewall-service</short>
  <description>Firewall service for ovn central</description>
  <port protocol="tcp" port="6641"/>
  <port protocol="tcp" port="6642"/>
</service>
[root@dell-per730-04 ovn]# 
[root@dell-per730-04 ovn]# firewall-cmd --zone=public --remove-service=ovn-host-firewall-service
success
[root@dell-per730-04 ovn]# firewall-cmd --zone=public --remove-service=ovn-central-firewall-service
success
[root@dell-per730-04 ovn]# ovn-sbctl show
[root@dell-per730-04 ovn]# firewall-cmd --zone=public --list-all
public (active)
  target: default
  icmp-block-inversion: no
  interfaces: em1
  sources: 
  services: ssh dhcpv6-client
  ports: 
  protocols: 
  masquerade: no
  forward-ports: 
  source-ports: 
  icmp-blocks: 
  rich rules: 
	
[root@dell-per730-04 ovn]# ovn-sbctl show
[root@dell-per730-04 ovn]# 
[root@dell-per730-04 ovn]# firewall-cmd --zone=public --add-service=ovn-host-firewall-service
success
[root@dell-per730-04 ovn]# firewall-cmd --zone=public --add-service=ovn-central-firewall-service
success
[root@dell-per730-04 ovn]# firewall-cmd --zone=public --list-all
public (active)
  target: default
  icmp-block-inversion: no
  interfaces: em1
  sources: 
  services: ssh dhcpv6-client ovn-host-firewall-service ovn-central-firewall-service
  ports: 
  protocols: 
  masquerade: no
  forward-ports: 
  source-ports: 
  icmp-blocks: 
  rich rules: 
	
[root@dell-per730-04 ovn]# ovn-sbctl show
Chassis "18b7c59a-8a0b-4338-9d28-358b45528a80"
    hostname: "dell-per730-05.rhts.eng.pek2.redhat.com"
    Encap geneve
        ip: "192.168.1.144"
        options: {csum="true"}

Comment 39 Numan Siddique 2017-11-10 08:59:30 UTC

Submitted the patch to fix this - https://review.openstack.org/#/c/518440/
and upstream bug link - https://bugs.launchpad.net/tripleo/+bug/1730711

Comment 40 Numan Siddique 2017-11-10 09:00:44 UTC

(In reply to Numan Siddique from comment #39)
> Submitted the patch to fix this - https://review.openstack.org/#/c/518440/
> and upstream bug link - https://bugs.launchpad.net/tripleo/+bug/1730711

Oops. Sorry. Please ignore - I mean to comment on this BZ - https://bugzilla.redhat.com/show_bug.cgi?id=1510879

Note You need to log in before you can comment on or make changes to this bug.