Bug 1744991

Summary: ovn-controller crashes when DNS is configured and a (specific/malformed) DNS packet is received
Product: Red Hat Enterprise Linux Fast Datapath Reporter: Dumitru Ceara <dceara>
Component: ovn2.11Assignee: Dumitru Ceara <dceara>
Status: CLOSED ERRATA QA Contact: haidong li <haili>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: FDP 19.FCC: ctrautma, fleitner, kfida, qding, yinxu
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1744994 1746198 (view as bug list) Environment:
Last Closed: 2019-10-01 07:21:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1744994, 1746198    

Description Dumitru Ceara 2019-08-23 12:47:11 UTC
Description of problem:
When DNS is configured in OVN for a Logical Switch ovn-controller crashes when certain DNS packets are received.

Version-Release number of selected component (if applicable):


How reproducible:

Steps to Reproduce:
1. Configure a Logical Switch and a VIF
2. Configure a DNS entry for the Logical Switch.
3. Inject a specific DNS packet from the VIF. For example, with Scapy:

    >>> p = IP(dst='10.0.0.2',src='10.0.0.3')/UDP(dport=53)/('a'*364)
    >>> send(p)

Actual results:
ovn-controller stays stuck in an infinite loop and keeps allocating memory until it aborts.

Expected results:
The packet should be processed without issues.


Additional info:

Comment 4 haidong li 2019-09-09 11:18:25 UTC
reproduced on the old version:
[root@dell-per730-19 ~]# uname -a
Linux dell-per730-19.rhts.eng.pek2.redhat.com 3.10.0-1062.el7.x86_64 #1 SMP Thu Jul 18 20:25:13 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
[root@dell-per730-19 ~]# rpm -qa | grep openvswitch
kernel-kernel-networking-openvswitch-ovn-1.0-138.noarch
openvswitch2.11-2.11.0-18.el7fdp.x86_64
openvswitch-selinux-extra-policy-1.0-13.el7fdp.noarch
[root@dell-per730-19 ~]# rpm -qa | grep ovn
ovn2.11-2.11.0-26.el7fdp.x86_64
ovn2.11-central-2.11.0-26.el7fdp.x86_64
ovn2.11-host-2.11.0-26.el7fdp.x86_64
[root@dell-per730-19 ~]# 


sendp(Ether(src="00:de:ad:01:00:01", dst="00:de:ad:ff:01:02")/IP(src="172.16.102.11", dst="172.16.102.254")/UDP(dport=53)/('a'*364) , iface="eth1")
.
Sent 1 packets.
>>> 
top - 06:53:16 up 4 days,  7:55,  4 users,  load average: 0.32, 0.28, 0.23
Tasks: 471 total,   1 running, 470 sleeping,   0 stopped,   0 zombie
%Cpu(s):  1.4 us,  1.7 sy,  0.0 ni, 96.8 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem : 65708808 total, 10249372 free, 49659584 used,  5799852 buff/cache
KiB Swap: 29241340 total, 29241340 free,        0 used. 15470060 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND     
10836 root      10 -10   49.3g  44.3g   1684 S 100.0 70.8   0:19.53 ovn-contro+ 
10761 openvsw+  10 -10 2331696 101856  17932 S   1.3  0.2   0:08.39 ovs-vswitc+ 
27544 root      20   0  162296   2624   1580 R   0.7  0.0   0:00.30 top         
    9 root      20   0       0      0      0 S   0.3  0.0   5:57.08 rcu_sched   
  802 root      20   0       0      0      0 S   0.3  0.0   1:24.67 xfsaild/dm+ 
 4348 root      20   0       0      0      0 S   0.3  0.0   0:00.71 kworker/u6+ 
10993 qemu      20   0 2917732 762072  10248 S   0.3  1.2   1:09.57 qemu-kvm    
11164 qemu      20   0 2929000 541788  10248 S   0.3  0.8   1:05.59 qemu-kvm    
    1 root      20   0  202004  15140   4216 S   0.0  0.0   0:22.96 systemd     
    2 root      20   0       0      0      0 S   0.0  0.0   0:00.18 kthreadd    
    4 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/0:+ 
    6 root      20   0       0      0      0 S   0.0  0.0   0:01.57 ksoftirqd/0 
    7 root      rt   0       0      0      0 S   0.0  0.0   0:01.30 migration/0 
    8 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcu_bh      
   10 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 lru-add-dr+ 
   11 root      rt   0       0      0      0 S   0.0  0.0   0:01.25 watchdog/0  

===========================================================================================
verified on the latest version:

[root@dell-per730-19 ~]# rpm -qa | grep openvswitch
kernel-kernel-networking-openvswitch-ovn-1.0-138.noarch
openvswitch-selinux-extra-policy-1.0-13.el7fdp.noarch
openvswitch2.11-2.11.0-21.el7fdp.x86_64
[root@dell-per730-19 ~]# rpm -qa | grep ovn
ovn2.11-central-2.11.0-36.el7fdp.x86_64
kernel-kernel-networking-openvswitch-ovn-1.0-138.noarch
ovn2.11-2.11.0-36.el7fdp.x86_64
ovn2.11-host-2.11.0-36.el7fdp.x86_64
[root@dell-per730-19 ~]# 
sendp(Ether(src="00:de:ad:01:00:01", dst="00:de:ad:ff:01:02")/IP(src="172.16.102.11", dst="172.16.102.254")/UDP(dport=53)/('a'*364) , iface="eth1")
.
Sent 1 packets.
>>>

top - 07:16:34 up 4 days,  8:18,  4 users,  load average: 0.06, 0.13, 0.21
Tasks: 470 total,   1 running, 469 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.0 us,  0.0 sy,  0.0 ni, 99.9 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem : 65708808 total, 56864120 free,  3050720 used,  5793968 buff/cache
KiB Swap: 29241340 total, 29241340 free,        0 used. 62081140 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND     
14437 root      20   0  162296   2624   1580 R   1.0  0.0   0:00.36 top         
29103 openvsw+  10 -10 2331700 102780  17936 S   1.0  0.2   0:12.62 ovs-vswitc+ 
29517 qemu      20   0 2921828 536440  10248 S   0.3  0.8   1:13.29 qemu-kvm    
    1 root      20   0  202004  15140   4216 S   0.0  0.0   0:24.36 systemd     
    2 root      20   0       0      0      0 S   0.0  0.0   0:00.19 kthreadd    
    4 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/0:+ 
    6 root      20   0       0      0      0 S   0.0  0.0   0:01.58 ksoftirqd/0 
    7 root      rt   0       0      0      0 S   0.0  0.0   0:01.31 migration/0 
    8 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcu_bh      
    9 root      20   0       0      0      0 S   0.0  0.0   5:57.98 rcu_sched   
   10 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 lru-add-dr+ 
   11 root      rt   0       0      0      0 S   0.0  0.0   0:01.25 watchdog/0  
   12 root      rt   0       0      0      0 S   0.0  0.0   0:01.14 watchdog/1  
   13 root      rt   0       0      0      0 S   0.0  0.0   0:02.97 migration/1 
   14 root      20   0       0      0      0 S   0.0  0.0   0:21.65 ksoftirqd/1 
   16 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/1:+ 
   18 root      rt   0       0      0      0 S   0.0  0.0   0:01.04 watchdog/2  

[root@dell-per730-19 ~]#

Comment 6 errata-xmlrpc 2019-10-01 07:21:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2943