Bug 1782056 - [RFE] Integration of built-in ipsec feature in RHV/RHHI-V with OVN
Summary: [RFE] Integration of built-in ipsec feature in RHV/RHHI-V with OVN
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 4.3.6
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ovirt-4.5.0
: 4.5.0
Assignee: Ales Musil
QA Contact: msheena
URL:
Whiteboard:
Depends On: 1782141 1885144 1898615 1940824 2002278
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-12-11 04:56 UTC by Abhishekh Patil
Modified: 2022-05-26 16:22 UTC (History)
15 users (show)

Fixed In Version: ovirt-engine-4.5.0 ovirt-provider-ovn-1.2.35
Doc Type: Release Note
Doc Text:
With this release, IPSec for the OVN feature is available on hosts with configured ovirt-provider-ovn, OVN version 2021 or later and OvS version 2.15 or later.
Clone Of:
Environment:
Last Closed: 2022-05-26 16:22:26 UTC
oVirt Team: Network
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 5682661 0 None None None 2021-01-05 17:18:31 UTC
Red Hat Product Errata RHSA-2022:4711 0 None None None 2022-05-26 16:22:59 UTC
oVirt gerrit 116551 0 master MERGED ansible: Generate separate OVN certificate 2021-09-13 07:45:43 UTC
oVirt gerrit 116553 0 None None None 2021-09-08 11:27:00 UTC
oVirt gerrit 116559 0 master MERGED ansible: Pass Host hostname to ovn-config 2021-09-13 07:45:45 UTC
oVirt gerrit 116560 0 master MERGED driver: Configure IPSec so user can just enable it on engine 2021-09-13 07:44:58 UTC
oVirt gerrit 117161 0 master MERGED ansible: Add IPSec port to firewalld services 2021-10-18 11:04:21 UTC

Description Abhishekh Patil 2019-12-11 04:56:15 UTC
Description of problem:

Provide OVN Ipsec support in RHVH / RHHI-V hosts

Comment 2 Pavel 2019-12-12 08:53:55 UTC
In case oVirt engine manages VMs on cloud-based baremetal servers, data protection is a requirement.

Disk encryption can be handled by OS inside VMs themselves.

However, management network (the flow over 'ovirtmgmt' interface) is currently not fully protected:

- GlusterFS storage by default does not use SSL encryption. This can be changed manually. However, this can be considered as another feature request.
- Communications between VMs which use "ovirt-provider-ovn" are not encrypted at all. This exposes all internal traffic to a malicious actor on internal management network. For instance, cloud provider could misconfigure its network devices causing traffic leakage (this is a real world scenario in our experience).

These days Open vSwitch supports IPSec for communications between its instances, see http://docs.openvswitch.org/en/latest/howto/ipsec/ and http://docs.openvswitch.org/en/latest/tutorials/ipsec/.

oVirt uses geneve type of interfaces in the following way:

`ovs-vsctl add-port br-int ovn-XXX-0 -- set interface ovn-XXX-0 type=geneve options:csum=true key=flow options:remote_ip=1.1.1.2`

What would be great is to create IPSec tunnel instead, like in the following example:

`ovs-vsctl set Open_vSwitch . other_config:certificate=/path/to/local_cert.pem other_config:private_key=/path/to/priv_key.pem`
`ovs-vsctl add-port br-int ovn-XXX-0 -- set interface ovn-XXX-0 type=gre options:remote_ip=1.1.1.2 options:remote_cert=/path/to/remote_cert.pem`

oVirt already has PKI implemented, so certificates can be taken from the existing configuration.

Comment 3 Pavel 2019-12-12 15:51:24 UTC
I have made some progress on this topic.

1. I was wrongly looking into Open vSwitch whereas I had to start with OVN (http://docs.openvswitch.org/en/stable/tutorials/ovn-ipsec/), thanks @Dominik.

2. Following the above article can lead to something, I will describe below.

CentOS 7 + oVirt has openvswitch-ipsec package which does the job... almost correctly, there are few bugs in it.

1. It is compiled with "/usr/local" path hardcoded for some functions, see the following error log:

```
Dec 12 13:29:22 ovirt-h2 ovs-monitor-ips[28207]: ovs|  1  | ovs-monitor-ipsec | ERR | traceback
                                                           Traceback (most recent call last):
                                                             File "/usr/share/openvswitch/scripts/ovs-monitor-ipsec", line 1229, in <module>...
Dec 12 13:29:22 ovirt-h2 ovs-ctl[28193]: 2019-12-12T13:29:22Z |  1  | ovs-monitor-ipsec | ERR | traceback
Dec 12 13:29:22 ovirt-h2 ovs-ctl[28193]: Traceback (most recent call last):
Dec 12 13:29:22 ovirt-h2 ovs-ctl[28193]: File "/usr/share/openvswitch/scripts/ovs-monitor-ipsec", line 1229, in <module>
Dec 12 13:29:22 ovirt-h2 ovs-ctl[28193]: main()
Dec 12 13:29:22 ovirt-h2 ovs-ctl[28193]: File "/usr/share/openvswitch/scripts/ovs-monitor-ipsec", line 1179, in main
Dec 12 13:29:22 ovirt-h2 ovs-ctl[28193]: schema_helper = ovs.db.idl.SchemaHelper()
Dec 12 13:29:22 ovirt-h2 ovs-ctl[28193]: File "/usr/lib64/python2.7/site-packages/ovs/db/idl.py", line 1766, in __init__
Dec 12 13:29:22 ovirt-h2 ovs-ctl[28193]: schema_json = ovs.json.from_file(location)
Dec 12 13:29:22 ovirt-h2 ovs-ctl[28193]: File "/usr/lib64/python2.7/site-packages/ovs/json.py", line 69, in from_file
Dec 12 13:29:22 ovirt-h2 ovs-ctl[28193]: stream = open(name, "r")
Dec 12 13:29:22 ovirt-h2 ovs-ctl[28193]: IOError: [Errno 2] No such file or directory: '/usr/local/share/openvswitch/vswitch.ovsschema'
```

2. `ovs-monitor-ipsec` creates its Unix control file in "/var/run" directory instead of "/var/run/openvswitch". Thus, monitoring with `ovs-appctl -t ovs-monitor-ipsec tunnels/show` is not possible by default.

Apart from that it is possible to initiate IPSec setup by doing the following:

On each host:
```
yum install openvswitch-ipsec
systemctl enable openvswitch-ipsec.service
ln -s /usr/share/openvswitch /usr/local/share/openvswitch
systemctl start openvswitch-ipsec.service
```

Next we need certificates.

On each host:
```
ovs-pki req -u `ovs-vsctl get Open_vSwitch . external_ids:system-id | tr -d \"`
scp *-req.pem root.com:/etc/pki/ovirt-engine/certs/
```

On engine (for each host system id, take into account the number of created certificate):
```
cd /etc/pki/ovirt-engine
openssl ca -config /etc/pki/ovirt-engine/openssl.conf -batch -in certs/304dae71-0b58-42a0-aeae-38a293184481-req.pem
scp certs/101D.pem root.com:304dae71-0b58-42a0-aeae-38a293184481-cert.pem
scp /etc/pki/ovirt-engine/ca.pem root.com:cacert.pem
```

On each host:
```
cp *.pem /etc/ssl
chassis=`ovs-vsctl get Open_vSwitch . external_ids:system-id | tr -d \"`
ovs-vsctl set Open_vSwitch . other_config:certificate=/etc/ssl/$chassis-cert.pem other_config:private_key=/etc/ssl/$chassis-privkey.pem other_config:ca_cert=/etc/ssl/cacert.pem
firewall-cmd --add-service="ipsec" --zone=public
firewall-cmd --add-service="ipsec" --zone=public --permanent
firewall-cmd --add-port=500/udp --zone=public
firewall-cmd --add-port=500/udp --zone=public --permanent
firewall-cmd --add-rich-rule='rule protocol value="esp" accept' --zone=public
firewall-cmd --add-rich-rule='rule protocol value="esp" accept' --zone=public --permanent
firewall-cmd --add-rich-rule='rule protocol value="ah" accept' --zone=public
firewall-cmd --add-rich-rule='rule protocol value="ah" accept' --zone=public --permanent
```

Here I am not sure about minimal set of firewall rules which would be enough.

On engine:
```
ovn-nbctl set nb_global . ipsec=true
```

* The issue I have now:

Each host in my setup has its public IP address along with internal ovirtmgmt network. This public IP address is used as default route. For some reason ISAKMP packets generated by ipsec daemon have source IP address set to this public IP.

Comment 4 Pavel 2019-12-13 12:05:03 UTC
I have made some more progress, getting and resolving new issues. :)

Below is the updated scenario based on the one above.

CentOS 7 libreswan package is old and can not properly verify peer's
certificates. So, install the package from libreswan team.
```
rpm -ihv https://download.libreswan.org/binaries/rhel/7/libreswan-release-7-1.noarch.rpm
yum install libreswan
```


On each host:
```
yum install openvswitch-ipsec
systemctl enable openvswitch-ipsec.service
ln -s /usr/share/openvswitch /usr/local/share/openvswitch
systemctl start openvswitch-ipsec.service
```

Symbolic link is required due to the incorrectly built "openvswitch-ipsec" package.

Next we need certificates.

On each host:
```
ovs-pki req -u `ovs-vsctl get Open_vSwitch . external_ids:system-id | tr -d \"`
scp *-req.pem root.com:/etc/pki/ovirt-engine/requests/
#+END_SRC

On engine (for each host system id, take into account the number of created certificate):
```
cd /etc/pki/ovirt-engine
xsid=304dae71-0b58-42a0-aeae-38a293184481
openssl ca -config <(cat /etc/pki/ovirt-engine/openssl.conf <(printf "[SAN]\nsubjectAltName=$xsid")) -extensions SAN -batch -in requests/$xsid-req.pem
scp certs/101D.pem root.com:$xsid-cert.pem
scp /etc/pki/ovirt-engine/ca.pem root.com:cacert.pem
```

On each host:
```
cp *.pem /etc/ssl
chassis=`ovs-vsctl get Open_vSwitch . external_ids:system-id | tr -d \"`
ovs-vsctl set Open_vSwitch . other_config:certificate=/etc/ssl/$chassis-cert.pem other_config:private_key=/etc/ssl/$chassis-privkey.pem other_config:ca_cert=/etc/ssl/cacert.pem
firewall-cmd --add-service="ipsec" --zone=public
firewall-cmd --add-service="ipsec" --zone=public --permanent
firewall-cmd --add-port=500/udp --zone=public
firewall-cmd --add-port=500/udp --zone=public --permanent
firewall-cmd --add-rich-rule='rule protocol value="esp" accept' --zone=public
firewall-cmd --add-rich-rule='rule protocol value="esp" accept' --zone=public --permanent
firewall-cmd --add-rich-rule='rule protocol value="ah" accept' --zone=public
firewall-cmd --add-rich-rule='rule protocol value="ah" accept' --zone=public --permanent
```

Update `/usr/share/openvswitch/scripts/ovs-monitor-ipsec` script on each host
with hardcoded right leg IP addresses. The default value of "%defaultroute" is
not correct in all cases. Other available macroses are also not suitable. Diff
example:
```
@@ -383,36 +383,36 @@ conn %%default
 
     SHUNT_POLICY = """conn prevent_unencrypted_gre
     type=drop
-    left=%defaultroute
+    left=172.18.53.201
     leftprotoport=gre
     mark={0}
 
 conn prevent_unencrypted_geneve
     type=drop
-    left=%defaultroute
+    left=172.18.53.201
     leftprotoport=udp/6081
     mark={0}

 conn prevent_unencrypted_stt
     type=drop
-    left=%defaultroute
+    left=172.18.53.201
     leftprotoport=tcp/7471
     mark={0}

 conn prevent_unencrypted_vxlan
     type=drop
-    left=%defaultroute
+    left=172.18.53.201
     leftprotoport=udp/4789
     mark={0}

 """

     auth_tmpl = {"psk": Template("""\
-    left=%defaultroute
+    left=172.18.53.201
     right=$remote_ip
     authby=secret"""),
                  "pki_remote": Template("""\
-    left=%defaultroute
+    left=172.18.53.201
     right=$remote_ip
     leftid=@$local_name
     rightid=@$remote_name
@@ -420,7 +420,7 @@ conn prevent_unencrypted_vxlan
     rightcert="$remote_name"
     leftrsasigkey=%cert"""),
                  "pki_ca": Template("""\
-    left=%defaultroute
+    left=172.18.53.201
     right=$remote_ip
     leftid=@$local_name
     rightid=@$remote_name
```

On engine:
```
ovn-nbctl set nb_global . ipsec=true
```

To disable IPSec:
```
ovn-nbctl set nb_global . ipsec=true
```

Monitoring (be ready to create a symbolic link for ctl file from /var/run to
/var/run/openvswitch):
```
ovs-appctl -t ovs-monitor-ipsec tunnels/show
```


* Pending issue *

IPSec channel is successfully established with above configuration:
```
# ovs-appctl -t ovs-monitor-ipsec tunnels/show
Interface name: ovn-304dae-0 v1 (CONFIGURED)
  Tunnel Type:    geneve
  Remote IP:      172.18.53.254
  SKB mark:       None
  Local cert:     /etc/ssl/6bb62cd8-6f97-47dd-8f0f-6bd1dbb73fd0-cert.pem
  Local name:     6bb62cd8-6f97-47dd-8f0f-6bd1dbb73fd0
  Local key:      /etc/ssl/6bb62cd8-6f97-47dd-8f0f-6bd1dbb73fd0-privkey.pem
  Remote cert:    None
  Remote name:    304dae71-0b58-42a0-aeae-38a293184481
  CA cert:        /etc/ssl/cacert.pem
  PSK:            None
  Ofport:         2
  CFM state:      Disabled
Kernel policies installed:
  src 172.18.53.202/32 dst 172.18.53.254/32 proto udp dport 6081
  src 172.18.53.202/32 dst 172.18.53.254/32 proto udp dport 6081
  src 172.18.53.202/32 dst 172.18.53.254/32 proto udp sport 6081
  src 172.18.53.202/32 dst 172.18.53.254/32 proto udp sport 6081
Kernel security associations installed:
  sel src 172.18.53.254/32 dst 172.18.53.202/32 proto udp sport 6081
  sel src 172.18.53.202/32 dst 172.18.53.254/32 proto udp dport 6081
  sel src 172.18.53.254/32 dst 172.18.53.202/32 proto udp sport 6081
  sel src 172.18.53.202/32 dst 172.18.53.254/32 proto udp dport 6081
  sel src 172.18.53.254/32 dst 172.18.53.202/32 proto udp dport 6081
  sel src 172.18.53.202/32 dst 172.18.53.254/32 proto udp sport 6081
IPsec connections that are active:

Interface name: ovn-ef08f0-0 v1 (CONFIGURED)
  Tunnel Type:    geneve
  Remote IP:      172.18.53.201
  SKB mark:       None
  Local cert:     /etc/ssl/6bb62cd8-6f97-47dd-8f0f-6bd1dbb73fd0-cert.pem
  Local name:     6bb62cd8-6f97-47dd-8f0f-6bd1dbb73fd0
  Local key:      /etc/ssl/6bb62cd8-6f97-47dd-8f0f-6bd1dbb73fd0-privkey.pem
  Remote cert:    None
  Remote name:    ef08f0d7-eaea-4878-bec4-f8c011620d2e
  CA cert:        /etc/ssl/cacert.pem
  PSK:            None
  Ofport:         1
  CFM state:      Disabled
Kernel policies installed:
  src 172.18.53.202/32 dst 172.18.53.201/32 proto udp dport 6081
  src 172.18.53.202/32 dst 172.18.53.201/32 proto udp dport 6081
  src 172.18.53.202/32 dst 172.18.53.201/32 proto udp sport 6081
  src 172.18.53.202/32 dst 172.18.53.201/32 proto udp sport 6081
Kernel security associations installed:
  sel src 172.18.53.201/32 dst 172.18.53.202/32 proto udp sport 6081
  sel src 172.18.53.202/32 dst 172.18.53.201/32 proto udp dport 6081
  sel src 172.18.53.201/32 dst 172.18.53.202/32 proto udp sport 6081
  sel src 172.18.53.202/32 dst 172.18.53.201/32 proto udp dport 6081
  sel src 172.18.53.201/32 dst 172.18.53.202/32 proto udp dport 6081
  sel src 172.18.53.202/32 dst 172.18.53.201/32 proto udp sport 6081
IPsec connections that are active:

```

However, all communications from VMs on OVN network are still attempted to be made over clear-text GENEVE tunnel. Probably, re-creation of virtual network is required or even host re-adding.

Comment 6 Raphaël HOAREAU 2020-01-20 14:31:58 UTC
Hi Pavel,

Did you managed to get encrypted communications over the GENEVE tunnel ?

I have a working RHHI-V cluster here and I did the same steps as you (with red hat compiled openvswitch-ipsec packages). Got the same results.

RHHI Cluster : 
 - rhhi-host1 : 10.10.10.01
 - rhhi-host2 : 10.10.10.02
 - rhhi-host3 : 10.10.10.03

VM running on rhhi-host2 : 172.16.0.10 / ovirtmgmt network
VM running on rhhi-host3 : 172.16.0.11 / ovirtmgmt network

##################
[root@rhhi-host2 log]# rpm -qa | grep openvswitch
openvswitch-selinux-extra-policy-1.0-14.el7fdp.noarch
openvswitch2.11-2.11.0-35.el7.x86_64
rhv-openvswitch-2.11-5.el7ev.noarch
rhv-openvswitch-ovn-host-2.11-5.el7ev.noarch
python-openvswitch2.11-2.11.0-35.el7.x86_64
openvswitch2.11-ipsec-2.11.0-35.el7.x86_64
rhv-openvswitch-ovn-common-2.11-5.el7ev.noarch
rhv-python-openvswitch-2.11-5.el7ev.noarch
openvswitch2.11-debuginfo-2.11.0-35.el7.x86_64
openvswitch2.11-test-2.11.0-35.el7.noarch
##################

##################
[root@rhhi-host2 log]# ovs-appctl -t ovs-monitor-ipsec tunnels/show
Interface name: ovn-40d3e4-0 v1 (CONFIGURED)
  Tunnel Type:    geneve
  Remote IP:      10.10.10.03
  SKB mark:       0/1
  Local cert:     /etc/ssl/57ecad94-604f-4bf0-b757-90e5ea5610bd-cert.pem
  Local name:     57ecad94-604f-4bf0-b757-90e5ea5610bd
  Local key:      /etc/ssl/57ecad94-604f-4bf0-b757-90e5ea5610bd-privkey.pem
  Remote cert:    None
  Remote name:    40d3e4b8-bec7-4c9f-bbf9-064b97a0bf9d
  CA cert:        /etc/ssl/cacert.pem
  PSK:            None
  Ofport:         3
  CFM state:      Disabled
Kernel policies installed:
  src 10.10.10.02/32 dst 10.10.10.03/32 proto udp dport 6081
  src 10.10.10.02/32 dst 10.10.10.03/32 proto udp dport 6081
Kernel security associations installed:
  sel src 10.10.10.03/32 dst 10.10.10.02/32 proto udp sport 6081
  sel src 10.10.10.02/32 dst 10.10.10.03/32 proto udp dport 6081
  sel src 10.10.10.03/32 dst 10.10.10.02/32 proto udp sport 6081
  sel src 10.10.10.02/32 dst 10.10.10.03/32 proto udp dport 6081
  sel src 10.10.10.03/32 dst 10.10.10.02/32 proto udp sport 6081
  sel src 10.10.10.02/32 dst 10.10.10.03/32 proto udp dport 6081
  sel src 10.10.10.03/32 dst 10.10.10.02/32 proto udp sport 6081
  sel src 10.10.10.02/32 dst 10.10.10.03/32 proto udp dport 6081
  sel src 10.10.10.03/32 dst 10.10.10.02/32 proto udp sport 6081
  sel src 10.10.10.02/32 dst 10.10.10.03/32 proto udp dport 6081
  sel src 10.10.10.03/32 dst 10.10.10.02/32 proto udp sport 6081
  sel src 10.10.10.02/32 dst 10.10.10.03/32 proto udp dport 6081
  sel src 10.10.10.03/32 dst 10.10.10.02/32 proto udp sport 6081
  sel src 10.10.10.02/32 dst 10.10.10.03/32 proto udp dport 6081
  sel src 10.10.10.03/32 dst 10.10.10.02/32 proto udp sport 6081
  sel src 10.10.10.02/32 dst 10.10.10.03/32 proto udp dport 6081
  sel src 10.10.10.03/32 dst 10.10.10.02/32 proto udp sport 6081
  sel src 10.10.10.02/32 dst 10.10.10.03/32 proto udp dport 6081
  sel src 10.10.10.03/32 dst 10.10.10.02/32 proto udp sport 6081
  sel src 10.10.10.02/32 dst 10.10.10.03/32 proto udp dport 6081
IPsec connections that are active:

Interface name: ovn-81497e-0 v1 (CONFIGURED)
  Tunnel Type:    geneve
  Remote IP:      10.10.10.01
  SKB mark:       0/1
  Local cert:     /etc/ssl/57ecad94-604f-4bf0-b757-90e5ea5610bd-cert.pem
  Local name:     57ecad94-604f-4bf0-b757-90e5ea5610bd
  Local key:      /etc/ssl/57ecad94-604f-4bf0-b757-90e5ea5610bd-privkey.pem
  Remote cert:    None
  Remote name:    81497e31-1b4c-4667-bd67-de4f0d7cebf6
  CA cert:        /etc/ssl/cacert.pem
  PSK:            None
  Ofport:         1
  CFM state:      Disabled
Kernel policies installed:
  src 10.10.10.02/32 dst 10.10.10.01/32 proto udp dport 6081
  src 10.10.10.02/32 dst 10.10.10.01/32 proto udp dport 6081
Kernel security associations installed:
  sel src 10.10.10.01/32 dst 10.10.10.02/32 proto udp sport 6081
  sel src 10.10.10.02/32 dst 10.10.10.01/32 proto udp dport 6081
  sel src 10.10.10.01/32 dst 10.10.10.02/32 proto udp sport 6081
  sel src 10.10.10.02/32 dst 10.10.10.01/32 proto udp dport 6081
  sel src 10.10.10.01/32 dst 10.10.10.02/32 proto udp sport 6081
  sel src 10.10.10.02/32 dst 10.10.10.01/32 proto udp dport 6081
  sel src 10.10.10.01/32 dst 10.10.10.02/32 proto udp sport 6081
  sel src 10.10.10.02/32 dst 10.10.10.01/32 proto udp dport 6081
  sel src 10.10.10.01/32 dst 10.10.10.02/32 proto udp sport 6081
  sel src 10.10.10.02/32 dst 10.10.10.01/32 proto udp dport 6081
  sel src 10.10.10.01/32 dst 10.10.10.02/32 proto udp sport 6081
  sel src 10.10.10.02/32 dst 10.10.10.01/32 proto udp dport 6081
  sel src 10.10.10.01/32 dst 10.10.10.02/32 proto udp sport 6081
  sel src 10.10.10.02/32 dst 10.10.10.01/32 proto udp dport 6081
  sel src 10.10.10.01/32 dst 10.10.10.02/32 proto udp sport 6081
  sel src 10.10.10.02/32 dst 10.10.10.01/32 proto udp dport 6081
  sel src 10.10.10.01/32 dst 10.10.10.02/32 proto udp sport 6081
  sel src 10.10.10.02/32 dst 10.10.10.01/32 proto udp dport 6081
  sel src 10.10.10.01/32 dst 10.10.10.02/32 proto udp sport 6081
  sel src 10.10.10.02/32 dst 10.10.10.01/32 proto udp dport 6081
IPsec connections that are active:
##################

Capturing ping (from 172.16.0.10 to 172.16.0.11)
##################
[root@rhhi-host2 log]# tcpdump -ni any host 172.16.0.10
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
15:19:51.207513 IP 172.16.0.10 > 172.16.0.11: ICMP echo request, id 1362, seq 2161, length 64
15:19:51.207529 IP 172.16.0.10 > 172.16.0.11: ICMP echo request, id 1362, seq 2161, length 64
15:19:51.207713 IP 172.16.0.11 > 172.16.0.10: ICMP echo reply, id 1362, seq 2161, length 64
15:19:51.207721 IP 172.16.0.11 > 172.16.0.10: ICMP echo reply, id 1362, seq 2161, length 64
##################

Capturing HTTP (curl from 172.16.0.10 to http://172.16.0.11/ ; note : nothing runs on port 80) : 
##################
15:22:42.383830 IP 172.16.0.10.51886 > 172.16.0.11.http: Flags [S], seq 764996093, win 29200, options [mss 1460,sackOK,TS val 3015446 ecr 0,nop,wscale 7], length 0
15:22:42.383852 IP 172.16.0.10.51886 > 172.16.0.11.http: Flags [S], seq 764996093, win 29200, options [mss 1460,sackOK,TS val 3015446 ecr 0,nop,wscale 7], length 0
##################

Comment 7 Pavel 2020-01-20 14:38:54 UTC
Hi Raphaël,

(In reply to Raphaël HOAREAU from comment #6)
> Hi Pavel,
> 
> Did you managed to get encrypted communications over the GENEVE tunnel ?
> 

Unfortunately no, I do not have any progress on this. :(
To be honest, I have not been trying a lot, but I do not have ideas where to look at anyway.

Comment 8 Raphaël HOAREAU 2020-01-20 16:15:28 UTC
Note that I had to create the certificate with : 
openssl ca -config <(cat /etc/pki/ovirt-engine/openssl.conf <(printf "[SAN]\nsubjectAltName=URI:$xsid")) -extensions SAN -batch -in requests/$xsid-req.pem

Instead of : 
openssl ca -config <(cat /etc/pki/ovirt-engine/openssl.conf <(printf "[SAN]\nsubjectAltName=$xsid")) -extensions SAN -batch -in requests/$xsid-req.pem

Openssl won't let me create cert with subjectAltName of "empty type" : 
##################
[root@rhhi-engine ovirt-engine]# openssl ca -config <(cat /etc/pki/ovirt-engine/openssl.conf <(printf "[SAN]\nsubjectAltName=$xsid")) -extensions SAN -batch -in requests/$xsid-req.pem
Using configuration from /dev/fd/63
Error Loading extension section SAN
140590925326224:error:2207507C:X509 V3 routines:v2i_GENERAL_NAME_ex:missing value:v3_alt.c:531:
140590925326224:error:22098080:X509 V3 routines:X509V3_EXT_nconf:error in extension:v3_conf.c:95:name=subjectAltName, value=81497e31-1b4c-4667-bd67-de4f0d7cebf6
##################

Tried a few (CN, otherName, etc.). Generation worked with URI:

Still, i have a few of these in /var/log/secure :
################## 
Jan 20 17:09:09 rhhi-host3 pluto[263518]: "ovn-57ecad-0-in-1" #212: STATE_PARENT_I2: retransmission; will wait 32 seconds for response
Jan 20 17:09:09 rhhi-host3 pluto[263518]: "ovn-57ecad-0-in-1" #212: certificate verified OK: CN=57ecad94-604f-4bf0-b757-90e5ea5610bd,OU=Open vSwitch certifier,O=Open vSwitch,ST=CA,C=US
Jan 20 17:09:09 rhhi-host3 pluto[263518]: "ovn-57ecad-0-in-1" #212: No matching subjectAltName found
Jan 20 17:09:09 rhhi-host3 pluto[263518]: "ovn-57ecad-0-in-1" #212: certificate does not contain subjectAltName=57ecad94-604f-4bf0-b757-90e5ea5610bd
Jan 20 17:09:09 rhhi-host3 pluto[263518]: "ovn-57ecad-0-in-1" #212: Peer public key SubjectAltName does not match peer ID for this connection
Jan 20 17:09:09 rhhi-host3 pluto[263518]: "ovn-57ecad-0-in-1" #212: X509: CERT payload does not match connection ID
##################

Comment 9 Dominik Holler 2020-01-22 13:37:41 UTC
Hello Timothy, can you please have a look at the previous comments?

Comment 10 Raphaël HOAREAU 2020-01-22 14:05:42 UTC
Additionnal note : certificates have been recreated with : 
openssl ca -config <(cat /etc/pki/ovirt-engine/openssl.conf <(printf "[SAN]\nsubjectAltName=DNS:$xsid")) -extensions SAN -batch -in requests/$xsid-req.pem

Using DNS instead of URI.
It gets rid of all the ipsec errors regarding "peer ID".

Found out that libreswan-3.25 is affected by this : https://lists.libreswan.org/pipermail/swan/2018/002697.html
Fixed in libreswan-3.26, not available on rhel7.
I backported the commit (https://github.com/libreswan/libreswan/commit/9d52ef1a3559d55cd7077edfabd01b14dd7e74f7) on top of libreswan-3.25-8.1.el7_7 with this patch : 
######################################
diff -u -r libreswan-3.25-pristine/programs/pluto/ikev2_child.c libreswan-3.25/programs/pluto/ikev2_child.c                                                                                                        
--- libreswan-3.25-pristine/programs/pluto/ikev2_child.c        2018-06-27 17:42:26.000000000 +0200
+++ libreswan-3.25/programs/pluto/ikev2_child.c 2020-01-22 12:16:11.300000000 +0100
@@ -867,7 +867,7 @@
                                                    d->name));
                                        int bfit_p =
                                                ikev2_evaluate_connection_port_fit(
-                                                       d, sra, role,
+                                                       d, sr, role,
                                                        tsi, tsr,
                                                        tsi_n, tsr_n,
                                                        &best_tsi_i,
@@ -881,7 +881,7 @@
                                                            best_tsr_i));
                                                int bfit_pr =
                                                        ikev2_evaluate_connection_protocol_fit(
-                                                               d, sra, role,
+                                                               d, sr, role,
                                                                tsi, tsr,
                                                                tsi_n, tsr_n,
                                                                &best_tsi_i,
######################################
Tunnels are now up in and out.

Still, ping goes in clear.

Comment 15 Michal Skrivanek 2020-06-23 12:35:22 UTC
This request is not currently committed to 4.4.z, moving it to 4.5

Comment 26 Michael Burman 2021-09-30 09:25:51 UTC
Note for QE:

The IPSec feature is available on hosts with configured ovirt-provider-ovn, OVN version 2021 or higher and OvS version 2.15 or higher. 
To enable IPSec for all hosts that comply with the requirements above, run following command on the engine: ovn-nbctl set nb_global . ipsec=true

Comment 32 msheena 2022-05-11 12:37:08 UTC
Verified
========
ovirt-engine-4.5.0.6-0.7.el8ev.noarch

ovirt-openvswitch-ovn-central-2.15-3.el8ev.noarch
ovn-2021-central-21.12.0-46.el8fdp.x86_64
ovirt-openvswitch-ovn-2.15-3.el8ev.noarch
ovirt-openvswitch-ovn-common-2.15-3.el8ev.noarch
ovirt-provider-ovn-1.2.36-1.el8ev.noarch
ovn-2021-21.12.0-46.el8fdp.x86_64

openvswitch2.15-ipsec-2.15.0-100.el8fdp.x86_64
ovirt-openvswitch-ipsec-2.15-3.el8ev.noarch

openvswitch-selinux-extra-policy-1.0-29.el8fdp.noarch
python3-openvswitch2.15-2.15.0-100.el8fdp.x86_64
ovirt-python-openvswitch-2.15-3.el8ev.noarch
ovirt-openvswitch-ovn-2.15-3.el8ev.noarch
ovirt-openvswitch-ovn-common-2.15-3.el8ev.noarch
openvswitch2.15-ipsec-2.15.0-100.el8fdp.x86_64
ovirt-openvswitch-2.15-3.el8ev.noarch
ovirt-openvswitch-ipsec-2.15-3.el8ev.noarch
openvswitch2.15-2.15.0-100.el8fdp.x86_64
ovirt-openvswitch-ovn-host-2.15-3.el8ev.noarch

Comment 38 errata-xmlrpc 2022-05-26 16:22:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: RHV Manager (ovirt-engine) [ovirt-4.5.0] security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:4711


Note You need to log in before you can comment on or make changes to this bug.