Bug 1761580

Summary:	[RHEL-8] OVN-DVR HA \| DNS Security group rule is applied but not working between VMs on different networks with FIPs
Product:	Red Hat Enterprise Linux Fast Datapath	Reporter:	Timothy Redaelli <tredaelli>
Component:	openvswitch2.11	Assignee:	Dumitru Ceara <dceara>
Status:	CLOSED ERRATA	QA Contact:	Jianlin Shi <jishi>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	FDP 19.F	CC:	apevec, astupnik, chrisw, ctrautma, dalvarez, dceara, ekuris, haili, jhsiao, jishi, jlibosva, kfida, lhh, majopela, qding, ralongi, rhos-maint, scohen, shdunne, slinaber, tredaelli, ushkalim
Target Milestone:	---	Keywords:	Triaged, ZStream
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openvswitch2.11-2.11.0-26.el8fdp	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:	1761461	Environment:
Last Closed:	2019-11-06 05:22:47 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Timothy Redaelli 2019-10-14 19:01:20 UTC

+++ This bug was initially created as a clone of Bug #1761461 +++

+++ This bug was initially created as a clone of Bug #1740770 +++

Description of problem:
* Setup is OVN-DVR HA. Installing OpenShift 3.11 on top as a tenant.
* openshift instances have fips attached to them.
* On the same tenant, created an instance to act as the DNS server for the openshift instances.
* The DNS-instance is on a separated subnet than the openshift nodes.
* Each network is connected to an openstack router and the external_gateway network is the same on both routers
* Security group rules allow DNS traffic on both instances.

Eventually, DNS queries are not reaching the DNS server.

The same configuration on OVS and on OVN-HA is working (from a Jenkins job) - we only see failure on OVN-DVR HA

Note: ICMP and SSH are allowed on the SG and working ok.

Version-Release number of selected component (if applicable):
python-networking-ovn-4.0.3-7.el7ost

How reproducible:
100%

Steps to Reproduce:
1. Deploy OVN-DVR HA
2. Create a tenant and 2 networks+subnets
3. Create FIPS
4. Boot 2 instances - one on each network and attach the fips to them
5. allow icmp ssh and dns on the security group assigned to the instances
6. set one of the vms as the DNS server of the other and try to resolve a hostname (www.google.com) 

Actual results:
DNS traffic is going out of one of the vm but not seen in the other.

Expected results:
All allowed traffic should be seen.

Additional info:
sosreports attached

Comment 2 Jianlin Shi 2019-10-23 01:55:54 UTC

reproduced on openvswitch-2.11.0-21 following step https://bugzilla.redhat.com/show_bug.cgi?id=1761461#c7:

[root@ibm-x3650m5-03 bz1761580]# ip netns exec vm2 tcpdump -i vm2 -nnle
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode                            
listening on vm2, link-type EN10MB (Ethernet), capture size 262144 bytes
21:49:23.137605 00:00:00:00:00:01 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 42.42.42.1 tell 42.42.42.2, length 28
21:49:23.138880 00:00:00:00:01:00 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 42.42.42.3 tell 42.42.42.1, length 28
21:49:23.138921 00:00:00:00:00:02 > 00:00:00:00:01:00, ethertype ARP (0x0806), length 42: Reply 42.42.42.3 is-at 00:00:00:00:00:02, length 28

<==== doesn't receive dns related packets

[root@ibm-x3650m5-03 ~]# rpm -qa | grep openvswitch
openvswitch2.11-2.11.0-21.el7fdp.x86_64
openvswitch-selinux-extra-policy-1.0-14.el7fdp.noarch

Verified on openvswitch-2.11.0-26:

[root@ibm-x3650m5-03 bz1761580]# ip netns exec vm2 tcpdump -i vm2 -nnle
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vm2, link-type EN10MB (Ethernet), capture size 262144 bytes                              
21:55:12.106546 00:00:00:00:01:00 > 00:00:00:00:00:02, ethertype IPv4 (0x0800), length 67: 66.66.66.63.56466 > 42.42.42.3.53: 36630+ A? foo.com. (25)
21:55:12.106574 00:00:00:00:00:02 > 00:00:00:00:01:00, ethertype IPv4 (0x0800), length 95: 42.42.42.3 > 66.66.66.63: ICMP 42.42.42.3 udp port 53 unreachable, length 61
^C                                                                                                    
2 packets captured                                                                                    
2 packets received by filter                                                                          
0 packets dropped by kernel               

<=== packet received
                                                            
[root@ibm-x3650m5-03 bz1761580]# rpm -qa | grep openvswitch                                           
openvswitch2.11-2.11.0-26.el7fdp.x86_64                                                               
openvswitch-selinux-extra-policy-1.0-14.el7fdp.noarch

Comment 3 Jianlin Shi 2019-10-23 02:18:52 UTC

the right result should be https://bugzilla.redhat.com/show_bug.cgi?id=1761461#c8

the result in comment2 is for rhel-7

Comment 5 errata-xmlrpc 2019-11-06 05:22:47 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3720