Bug 1888445

Summary: error parsing actions for logical flow due to load balancer selection_fields
Product: Red Hat Enterprise Linux Fast Datapath Reporter: ffernand <ffernand>
Component: OVNAssignee: lorenzo bianconi <lorenzo.bianconi>
Status: CLOSED CURRENTRELEASE QA Contact: ying xu <yinxu>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: RHEL 8.0CC: ctrautma, dcbw, lorenzo.bianconi, mmichels, nusiddiq
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ovn2.13-20.09.0-13.el7fdn Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-08-17 13:56:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
script for reproducing issue none

Description ffernand 2020-10-14 21:19:13 UTC 
Description of problem:

When load balancer is configured with a selection_fields value, there seems
to exist a bug in the way logical flows with the drop action get generated.

When using lb with Load_Balancer_Health_Check, the action taken when all
members go offline causes the following error:

  2020-10-14T20:36:11.523Z|00019|lflow|WARN|error parsing actions "drop; hash_fields="ip_dst,ip_src,tcp_dst,tcp_src");": Syntax error at `hash_fields' expecting end of input.

Version-Release number of selected component (if applicable):

  $ ovn-nbctl --version
  ovn-nbctl 20.06.1
  Open vSwitch Library 2.13.2
  DB Schema 5.23.0

  This is likely happening on earlier version of OVN that have support
  for 'selection_fields' in the load_balancer table.


How reproducible:

  100% reproducible

Steps to Reproduce:
  [NOTE: see attached script for a more detailed list of steps]
  1. create lb with health checker
  2. stop all members of lb, making health checker members offline
  3. look for parsing error in ovn-controller log

Actual results:

2020-10-14T20:36:11.523Z|00019|lflow|WARN|error parsing actions "drop; hash_fields="ip_dst,ip_src,tcp_dst,tcp_src");": Syntax error at `hash_fields' expecting end of input.


Expected results:
""


Additional info:

See attached script for an easy way of reproducing this problem.
Comment 1 ffernand 2020-10-14 21:24:02 UTC 
Created attachment 1721604 [details]
script for reproducing issue
Comment 2 ffernand 2020-10-14 21:27:31 UTC 
Expected output of script attached looks like this:

vagrant@stack:~$ ./bug.sh
+./bug.sh:main:3                           set -e
+./bug.sh:main:5                           curl --silent https://gist.githubusercontent.com/flavio-fernandes/130fd5d602116d69e1ca1b7d405acc6d/raw/e273531581ac4a3e8016d5bcb0ce68c51d3b3ea5/create_fake_vm.sh
+./bug.sh:main:7                           chmod 755 create_fake_vm.sh
+./bug.sh:main:8                           echo ok
ok
+./bug.sh:main:10                          ovn-nbctl ls-add sw0
+./bug.sh:main:12                          ovn-nbctl lsp-add sw0 member1prt -- lsp-set-addresses member1prt 'fa:16:3e:11:11:11 10.0.0.11'
+./bug.sh:main:13                          ovn-nbctl lsp-add sw0 member2prt -- lsp-set-addresses member2prt 'fa:16:3e:22:22:22 10.0.0.22'
+./bug.sh:main:14                          sudo ./create_fake_vm.sh member1prt member1 fa:16:3e:11:11:11 10.0.0.11 24 10.0.0.1
+./bug.sh:main:15                          sudo ./create_fake_vm.sh member2prt member2 fa:16:3e:22:22:22 10.0.0.22 24 10.0.0.1
+./bug.sh:main:17                          ovn-nbctl lsp-add sw0 testprt -- lsp-set-addresses testprt 'fa:16:3e:12:13:12 10.0.0.123'
+./bug.sh:main:18                          sudo ./create_fake_vm.sh testprt test fa:16:3e:12:13:12 10.0.0.123 24 10.0.0.1
+./bug.sh:main:20                          ovn-nbctl lb-add lb1 10.0.0.100:80 10.0.0.11:8080,10.0.0.22:8080 tcp
+./bug.sh:main:21                          ovn-nbctl ls-lb-add sw0 lb1
+./bug.sh:main:23                          cat
+./bug.sh:main:32                          chmod 755 ./member.sh
+./bug.sh:main:37                          HEALTH_SRC=10.0.0.123
+./bug.sh:main:35                          sudo ip netns exec member2 ./member.sh server2
+./bug.sh:main:34                          sudo ip netns exec member1 ./member.sh server1
++./bug.sh:main:38                          ovn-nbctl --bare --column _uuid find load_balancer name=lb1 protocol=tcp
+./bug.sh:main:38                          OVN_LB_ID=7332f54e-a1a1-40d6-899a-70ae89b7a7f8
+./bug.sh:main:38                          echo 7332f54e-a1a1-40d6-899a-70ae89b7a7f8
7332f54e-a1a1-40d6-899a-70ae89b7a7f8
+./bug.sh:main:39                          ovn-nbctl set load_balancer 7332f54e-a1a1-40d6-899a-70ae89b7a7f8 ip_port_mappings:10.0.0.11=member1prt:10.0.0.123
+./bug.sh:main:40                          ovn-nbctl set load_balancer 7332f54e-a1a1-40d6-899a-70ae89b7a7f8 ip_port_mappings:10.0.0.22=member2prt:10.0.0.123
+./bug.sh:main:42                          ovn-nbctl --wait=sb -- --id=@hc create Load_Balancer_Health_Check 'vip=10.0.0.100\:80' -- add Load_Balancer 7332f54e-a1a1-40d6-899a-70ae89b7a7f8 health_check @hc
7e037390-195a-4afb-aa87-2e66feae9c57
+./bug.sh:main:45                          ovn-nbctl list load_balancer
_uuid               : 7332f54e-a1a1-40d6-899a-70ae89b7a7f8
external_ids        : {}
health_check        : [7e037390-195a-4afb-aa87-2e66feae9c57]
ip_port_mappings    : {"10.0.0.11"="member1prt:10.0.0.123", "10.0.0.22"="member2prt:10.0.0.123"}
name                : lb1
protocol            : tcp
selection_fields    : []
vips                : {"10.0.0.100:80"="10.0.0.11:8080,10.0.0.22:8080"}
+./bug.sh:main:46                          ovn-nbctl list Load_Balancer_Health_Check
_uuid               : 7e037390-195a-4afb-aa87-2e66feae9c57
external_ids        : {}
options             : {}
vip                 : "10.0.0.100:80"
+./bug.sh:main:47                          ovn-sbctl list service_monitor
_uuid               : 4b17a02c-55ad-4bb0-81db-1390ef0c0318
external_ids        : {}
ip                  : "10.0.0.22"
logical_port        : member2prt
options             : {}
port                : 8080
protocol            : tcp
src_ip              : "10.0.0.123"
src_mac             : "be:04:b3:84:6e:23"
status              : online

_uuid               : 1848b875-c2f6-4480-9ca1-58e24c5c2218
external_ids        : {}
ip                  : "10.0.0.11"
logical_port        : member1prt
options             : {}
port                : 8080
protocol            : tcp
src_ip              : "10.0.0.123"
src_mac             : "be:04:b3:84:6e:23"
status              : online
+./bug.sh:main:59                          wait_for_members 2
+./bug.sh:wait_for_members:50              EXP_MEMBERS=2
+./bug.sh:wait_for_members:51              echo -n 'Waiting until 2 members are online '
Waiting until 2 members are online +./bug.sh:wait_for_members:52              :
++./bug.sh:wait_for_members:53              ovn-sbctl list service_monitor
++./bug.sh:wait_for_members:53              grep status
++./bug.sh:wait_for_members:53              grep -c online
+./bug.sh:wait_for_members:53              '[' 2 -eq 2 ']'
+./bug.sh:wait_for_members:53              break
+./bug.sh:wait_for_members:56              echo ' done'
 done
+./bug.sh:main:63                          ovn-nbctl set load_balancer 7332f54e-a1a1-40d6-899a-70ae89b7a7f8 selection_fields=ip_dst,ip_src,tp_dst,tp_src
+./bug.sh:main:65                          sleep 1
+./bug.sh:main:68                          sudo killall -9 member.sh nc
./bug.sh: line 68: 29099 Killed                  sudo ip netns exec member1 ./member.sh server1
./bug.sh: line 68: 29100 Killed                  sudo ip netns exec member2 ./member.sh server2
+./bug.sh:main:70                          wait_for_members 0
+./bug.sh:wait_for_members:50              EXP_MEMBERS=0
+./bug.sh:wait_for_members:51              echo -n 'Waiting until 0 members are online '
Waiting until 0 members are online +./bug.sh:wait_for_members:52              :
++./bug.sh:wait_for_members:53              ovn-sbctl list service_monitor
++./bug.sh:wait_for_members:53              grep status
++./bug.sh:wait_for_members:53              grep -c online
+./bug.sh:wait_for_members:53              '[' 2 -eq 0 ']'
+./bug.sh:wait_for_members:54              sleep 1
+./bug.sh:wait_for_members:54              echo -n .
.+./bug.sh:wait_for_members:52              :
++./bug.sh:wait_for_members:53              grep -c online
++./bug.sh:wait_for_members:53              ovn-sbctl list service_monitor
++./bug.sh:wait_for_members:53              grep status
+./bug.sh:wait_for_members:53              '[' 2 -eq 0 ']'
+./bug.sh:wait_for_members:54              sleep 1
+./bug.sh:wait_for_members:54              echo -n .
.+./bug.sh:wait_for_members:52              :
++./bug.sh:wait_for_members:53              grep status
++./bug.sh:wait_for_members:53              grep -c online
++./bug.sh:wait_for_members:53              ovn-sbctl list service_monitor
+./bug.sh:wait_for_members:53              '[' 2 -eq 0 ']'
+./bug.sh:wait_for_members:54              sleep 1
+./bug.sh:wait_for_members:54              echo -n .
.+./bug.sh:wait_for_members:52              :
++./bug.sh:wait_for_members:53              grep -c online
++./bug.sh:wait_for_members:53              grep status
++./bug.sh:wait_for_members:53              ovn-sbctl list service_monitor
+./bug.sh:wait_for_members:53              '[' 2 -eq 0 ']'
+./bug.sh:wait_for_members:54              sleep 1
+./bug.sh:wait_for_members:54              echo -n .
.+./bug.sh:wait_for_members:52              :
++./bug.sh:wait_for_members:53              grep status
++./bug.sh:wait_for_members:53              grep -c online
++./bug.sh:wait_for_members:53              ovn-sbctl list service_monitor
+./bug.sh:wait_for_members:53              '[' 0 -eq 0 ']'
+./bug.sh:wait_for_members:53              break
+./bug.sh:wait_for_members:56              echo ' done'
 done
+./bug.sh:main:76                          LOGFILE=/opt/stack/logs/ovn-controller.log
+./bug.sh:main:77                          grep 'error parsing' /opt/stack/logs/ovn-controller.log
2020-10-14T21:26:52.893Z|00019|lflow|WARN|error parsing actions "drop; hash_fields="ip_dst,ip_src,tcp_dst,tcp_src");": Syntax error at `hash_fields' expecting end of input.
vagrant@stack:~$
vagrant@stack:~$
vagrant@stack:~$ tail -5 /opt/stack/logs/ovn-controller.log
2020-10-14T21:26:47.170Z|00015|binding|INFO|Claiming lport member2prt for this chassis.
2020-10-14T21:26:47.170Z|00016|binding|INFO|member2prt: Claiming fa:16:3e:22:22:22 10.0.0.22
2020-10-14T21:26:47.506Z|00017|binding|INFO|Claiming lport testprt for this chassis.
2020-10-14T21:26:47.506Z|00018|binding|INFO|testprt: Claiming fa:16:3e:12:13:12 10.0.0.123
2020-10-14T21:26:52.893Z|00019|lflow|WARN|error parsing actions "drop; hash_fields="ip_dst,ip_src,tcp_dst,tcp_src");": Syntax error at `hash_fields' expecting end of input.
vagrant@stack:~$
vagrant@stack:~$
vagrant@stack:~$
Comment 4 Dan Williams 2021-06-07 19:26:03 UTC 
Final fix accepted upstream (that landed in downstream too) was http://patchwork.ozlabs.org/project/ovn/patch/366090e4b70fd9d276804e08e3f66b113f8eaa88.1605098198.git.lorenzo.bianconi@redhat.com/
Comment 5 Mark Michelson 2023-08-17 13:56:15 UTC 
Closing this as CURRENTRELEASE since this has been fixed for some time.