Bug 1975039

Summary: [OVS IPsec] no ESP in packets for RHEL9 in sefl-signed/CA-signed certificate mode
Product: Red Hat Enterprise Linux Fast Datapath Reporter: qding
Component: openvswitch2.15Assignee: Mohammad Heib <mheib>
Status: CLOSED MIGRATED QA Contact: qding
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: FDP 21.ECC: akaris, ctrautma, fleitner, jhsiao, mheib, qding, ralongi
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2024-03-11 18:16:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description qding 2021-06-23 02:15:50 UTC
Description of problem:

With RHEL-9, when SELinux in Enforcing mode, it will fail to start service openvswitch-ipsec. So I run the test when SELinux in Perssive mode, but it still has problem that there is no ESP in packets for both sefl-signed and CA-signed certificate mode

Version-Release number of selected component (if applicable):
RHEL-9.0.0-20210617.1 
openvswitch-selinux-extra-policy-1.0-29.el9fdp.noarch.rpm
openvswitch2.15-2.15.0-16.el9fdp.x86_64.rpm
selinux-policy-34.1.8-1.el9.noarch

How reproducible: always

Additional info:

beaker job: https://beaker.engineering.redhat.com/jobs/5494202

Comment 7 Mohammad Heib 2022-02-14 08:57:30 UTC
Hi Qding,

thank you for adding the commands outputs above.
i submitted a patch upstream which suppose to fix the issue and tried it on rhel9 and everything seems to be fine.
i just want to make sure that you are not facing a different issue so please add the output of those commands below:

    # certutil -L -d /var/lib/ipsec/nss/
    # certutil -L -d /etc/ipsec.d/
    
    and this command needs the IPsec interface name which can be found in /etc/ipsec.conf
      i assume based on your logs above that the name is 'tun123-in-1' please make sure that you have it in /etc/ipsec.conf
     
     # ipsec auto --start tun123-in-1

upstream fix:
    https://patchwork.ozlabs.org/project/openvswitch/patch/20220214083947.30774-1-mheib@redhat.com/

Thanks

Comment 8 qding 2022-02-17 10:03:16 UTC
[root@dell-per730-05 ~]# uname -r
5.14.0-58.el9.x86_64
[root@dell-per730-05 ~]# rpm -qa | grep openvswitch
kernel-kernel-networking-openvswitch-ipsec-1.0-41.noarch
openvswitch-selinux-extra-policy-1.0-31.el9fdp.noarch
openvswitch2.16-2.16.0-43.el9fdp.x86_64
python3-openvswitch2.16-2.16.0-43.el9fdp.x86_64
openvswitch2.16-ipsec-2.16.0-43.el9fdp.x86_64
[root@dell-per730-05 ~]# ovs-vsctl show
f611de56-024d-4a65-acda-6df129267057
    Bridge ovsbr0
        Port ovsbr0
            Interface ovsbr0
                type: internal
        Port tun123
            Interface tun123
                type: vxlan
                options: {local_ip="192.168.123.1", remote_cert="/tmp/keys/h2-cert.pem", remote_ip="192.168.123.2"}
    ovs_version: "2.16.3"
[root@dell-per730-05 ~]# certutil -L -d /var/lib/ipsec/nss/

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

[root@dell-per730-05 ~]# certutil -L -d /etc/ipsec.d/

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

ovs_certkey_h1                                               u,u,u
ovs_cert_h2                                                  P,P,P
[root@dell-per730-05 ~]# ipsec auto --start tun123-in-1
036 "tun123-in-1": failed to add connection: left certificate 'ovs_certkey_h1' not found in the NSS database
000 initiating all conns with alias='tun123-in-1'
021 no connection named "tun123-in-1"
[root@dell-per730-05 ~]# 
[root@dell-per730-05 ~]# cat /etc/ipsec.conf
# Generated by ovs-monitor-ipsec...do not modify by hand!


config setup
    uniqueids=yes

conn %default
    keyingtries=%forever
    type=transport
    auto=route
    ike=aes_gcm256-sha2_256
    esp=aes_gcm256
    ikev2=insist

conn tun123-in-1
    left=192.168.123.1
    right=192.168.123.2
    leftid=@h1
    rightid=@h2
    leftcert="ovs_certkey_h1"
    rightcert="ovs_cert_h2"
    leftrsasigkey=%cert
    leftprotoport=udp/4789
    rightprotoport=udp

conn tun123-out-1
    left=192.168.123.1
    right=192.168.123.2
    leftid=@h1
    rightid=@h2
    leftcert="ovs_certkey_h1"
    rightcert="ovs_cert_h2"
    leftrsasigkey=%cert
    leftprotoport=udp
    rightprotoport=udp/4789

[root@dell-per730-05 ~]#

Comment 11 Andreas Karis 2022-06-01 12:56:27 UTC
I also hit this here in Fedora 35 so I'd just like to say thank you as the suggested solution worked.

I ran into this issue on fedora 35 and this definitely unblocked me. Given that the patch hasn't merged upstream, yet, I applied the patch manually:
~~~
vi /usr/share/openvswitch/scripts/ovs-monitor-ipsec
~~~

And replace these lines:
~~~
-        ipsec_d = args.ipsec_d if args.ipsec_d else "/etc/ipsec.d"
+        ipsec_d = args.ipsec_d if args.ipsec_d else "/var/lib/ipsec/nss"
~~~

Thanks!