Bug 1982601

Description of problem:
Two VMs in the same chassis cannot communicate through a port that has been forwarded, using the FIP.

  VM-A --> fixed-ip --> FIP (port 22 forwarded) --> fixed-ip --> VM-B



Version-Release number of selected component (if applicable):
* Reported in master:
** https://bugs.launchpad.net/neutron/+bug/1935959
** OVN 21.03

* Tested internally in OSP16.1 and OSP16.2
** OVN 21.03.1



Steps to Reproduce:
1. Create two VMs, located in the same compute node.
2. Inside the VMs, enable SSHD, enable root logging and create a root password (NOTE1)
3. Create a SG rule to accept all traffic to port TCP 22
  # openstack security group rule create --dst-port 22 --protocol tcp --ingress $sg
4. Create two FIPs
  # openstack floating ip create nova
5. Forward the internal port 22 to external port 22 in both VM ports
  # openstack floating ip port forwarding create --internal-ip-address $fixed_ip1 --port $port1 --internal-protocol-port 22 --external-protocol-port 22 --protocol tcp $fip1



Actual results:
a. If we try to SSH from VM-A to VM-B using the fixed IP (private IP), the SSH connection works fine.
b. If we try to SSH from VM-A to VM-B using the FIP (and the forwarded port 22->22), the SSH does NOT work.
c. If we try to SSH to VM-B using the FIP from the undercloud host, the SSH connection works fine.

What I saw in (b) is that the traffic from VM-A port goes to the geneve port and is sent outside the chassis. This traffic does not return. I followed the traffic through br-int tables 18->19->20->21->22->23->24->25->26->27->28->29->30->37. In table 37, this traffic goes to the tunnel interface:

 cookie=0x55e2bb72, duration=58068.952s, table=37, n_packets=3634, n_bytes=381198, idle_age=307, priority=100,reg15=0x2,metadata=0x3 actions=load:0x3->NXM_NX_TUN_ID[0..23],set_field:0x2->tun_metadata0,move:NXM_NX_REG14[0..14]->NXM_NX_TUN_METADATA0[16..30],output:2                                                    

[root@compute-0 ~]# ovs-ofctl show br-int
...
 2(ovn-de6c7e-0): addr:a2:5c:95:20:01:1d
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max

[root@compute-0 ~]# ovs-vsctl show
...
        Port ovn-de6c7e-0
            Interface ovn-de6c7e-0
                type: geneve
                options: {csum="true", key=flow, remote_ip="172.17.2.113"}



Additional info:
NOTE1: I'm very lazy to use SSH keys. I usually modify the cloud images to enable the root user with a trivial password.
1. Install virt-customize
  # sudo yum install libguestfs-tools-c
2. Modify the image:
  # virt-customize -a image_file.img --root-password password:pass