Bug 2162837 - [libvirt] virtual machine is leaking IP address when IKEv2/IPsec VPN connection is active on host
Summary: [libvirt] virtual machine is leaking IP address when IKEv2/IPsec VPN connecti...
Keywords:
Status: NEW
Alias: None
Product: Fedora EPEL
Classification: Fedora
Component: NetworkManager-strongswan
Version: epel8
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Petr Menšík
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-01-21 08:05 UTC by Joachim Frieben
Modified: 2023-03-19 19:49 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker FC-788 0 None None None 2023-03-19 08:09:03 UTC

Description Joachim Frieben 2023-01-21 08:05:14 UTC
Description of problem:
Virtual machines are leaking the system's IP address even when an IKEv2/IPsec VPN connection is active on the host.

Version-Release number of selected component (if applicable):
8.0.0-10.1.module_el8.7.0+3387+571b583b

How reproducible:
Always

Steps to Reproduce:
1. Activate an IKEv2/IPsec VPN connection on the host configured by means of NetworkManager-strongswan-gnome.
2. Launch a virtual machine in GNOME Boxes, e.g. a Fedora 37 live system.
3. Run curl https://api.ipify.org both in the virtual machine and on the host.

Actual results:
The output in the virtual machine erroneously shows the IP address assigned by the ISP whereas the output on the host correctly shows the IP address assigned by the VPN service provider.

Expected results:
In both cases the IP address assigned by the VPN service provider is displayed.

Additional info:
- In contrast with the IP leak occurring in the virtual machine, the host is affected by a DNS leak using either the ISP's DNS server or the VPN service provider's DNS server whereas in the virtual machine only the VPN service provider's DNS server is correctly being used.
- When activating an OpenVPN connection on the host configured by means of NetworkManager-openvpn-gnome instead of an IKEv2/IPsec connection, then no IP or DNS leaks occur neither on the host neither in the virtual machine. The content of /etc/resolv.conf is identical in both cases listing both name servers, one entry for the system resolver and one entry for the VPN service provider's DNS server.
- When the host is running Fedora 37, then -no- leaks are observed for an IKEv2/IPsec connection either.

Comment 3 Laine Stump 2023-03-14 18:36:15 UTC
(I haven't used any sort of IPSec implementation since 2003, so I needed to lookup some things; when looking for the versions of OpenSWAN used in RHEL8 and Fedora 37. I found that RHEL switched to LibreSWAN as of RHEL 6.8, and I'm sure Fedora switched even before that. So I'm assuming that is the package you mean, and I'll refer to it that way from now on)

My understanding of your report is that when an LibreSWAN VPN connection is active, then traffic originating from the host is going through the VPN connection, but traffic originating from the VM is not going through the VPN, and is instead going directly to its destination.

On the other hand, if the VPN is setup using OpenVPN, then all traffic goes through the VPN. Also, if the host is running the versions of either LibreSWAN or OpenVPN that are supplied with Fedora 37 rather than the versions that are in RHEL8, then all the traffic goes through the VPN.

Some more background - I'm assuming that gnome-boxes is attaching a tap device to virbr0, which is a bridge device created by libvirt (this can be confirmed by looking at the output of "virsh dumpxml $guestname" and looking for the <interface> secrtion). libvirt also runs a dhcp server, dns server, and adds iptables rules that will NAT outgoing traffic to have the IP address of the host. This NAT happens during the post-routing filter hook as the host is forwarding the packet out from the host:

(from "iptables -t nat -S")
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE

Since the traffic from the guests has the IP address of the host when it gets to the external IP address checking system, that means that it is not being sent through the tunnel, and has instead been NATed and then routed out normally. This would indicate that the LibreSWAN software is likely examining the traffic before the post-routing filter hook, and only directing traffic that already has the host's IP as source through the VPN. 

Since the same setup works properly (sending guest traffic through the VPN) on Fedora 37, I would guess that LibreSWAN has made a change since the version that's in RHEL8 that changes where/how they make the decision of which traffic to send through the VPN connection. That's a good thing, because there really is nothing libvirt can do to change this behavior.

I'm reassigning this BZ to the libreswan component (although my superficial guess is that the code that has changed this behavior is in the kernel, the maintainers of libreswan will have a better idea) so that the people responsible for that package can give their opinions.

Comment 4 Joachim Frieben 2023-03-15 17:44:09 UTC
The issue concerns package strongswan as implied by mentioning that the network connection was set up by means of package NetworkManager-strongswan-gnome.

Comment 5 Joachim Frieben 2023-03-16 06:51:58 UTC
Output of 'ifconfig' for IKEv2/IPsec based VPN:

enp0s25: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.216  netmask 255.255.255.0  broadcast 192.168.1.255
        ether 00:22:68:1b:74:75  txqueuelen 1000  (Ethernet)
        RX packets 94666  bytes 125978055 (120.1 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 51610  bytes 8287635 (7.9 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 20  memory 0xfc000000-fc020000  

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 84  bytes 7140 (6.9 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 84  bytes 7140 (6.9 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

nm-xfrm-1368477: flags=193<UP,RUNNING,NOARP>  mtu 1400
        inet 10.1.15.204  netmask 255.255.255.255
        inet6 fe80::1ece:3075:35a0:88fe  prefixlen 64  scopeid 0x20<link>
        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 1000  (UNSPEC)
        RX packets 633  bytes 86912 (84.8 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1059  bytes 399683 (390.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

tap0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::fca4:60ff:fe59:be52  prefixlen 64  scopeid 0x20<link>
        ether fe:a4:60:59:be:52  txqueuelen 1000  (Ethernet)
        RX packets 633  bytes 86912 (84.8 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1059  bytes 399683 (390.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        ether 52:54:00:38:0f:d3  txqueuelen 1000  (Ethernet)
        RX packets 4928  bytes 1420831 (1.3 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8560  bytes 8439830 (8.0 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Comment 6 Joachim Frieben 2023-03-16 06:54:28 UTC
Output of 'ifconfig' for OpenVPN based VPN:

enp0s25: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.216  netmask 255.255.255.0  broadcast 192.168.1.255
        ether 00:22:68:1b:74:75  txqueuelen 1000  (Ethernet)
        RX packets 93357  bytes 125452111 (119.6 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 50353  bytes 8083546 (7.7 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 20  memory 0xfc000000-fc020000  

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 48  bytes 4080 (3.9 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 48  bytes 4080 (3.9 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

tap0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::fcbb:2eff:fe48:d14a  prefixlen 64  scopeid 0x20<link>
        ether fe:bb:2e:48:d1:4a  txqueuelen 1000  (Ethernet)
        RX packets 4253  bytes 1398111 (1.3 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 9255  bytes 8120334 (7.7 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

tun0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>  mtu 1500
        inet 10.83.0.4  netmask 255.255.0.0  destination 10.83.0.4
        inet6 fe80::da9f:334e:b000:72b5  prefixlen 64  scopeid 0x20<link>
        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 100  (UNSPEC)
        RX packets 86593  bytes 116519400 (111.1 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 39356  bytes 3248341 (3.0 MiB)
        TX errors 0  dropped 2414 overruns 0  carrier 0  collisions 0

virbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        ether 52:54:00:38:0f:d3  txqueuelen 1000  (Ethernet)
        RX packets 4253  bytes 1338569 (1.2 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 7771  bytes 8040250 (7.6 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Comment 7 Joachim Frieben 2023-03-16 06:56:09 UTC
Output of 'iptables -t nat -S':

-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P POSTROUTING ACCEPT
-P OUTPUT ACCEPT
-N LIBVIRT_PRT
-A POSTROUTING -j LIBVIRT_PRT
-A LIBVIRT_PRT -s 192.168.122.0/24 -d 224.0.0.0/24 -j RETURN
-A LIBVIRT_PRT -s 192.168.122.0/24 -d 255.255.255.255/32 -j RETURN
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE

Comment 8 Joachim Frieben 2023-03-16 07:04:14 UTC
- The EPEL repository provides package strongswan-5.9.10-1.el8 for Red Hat EL8 which corresponds to latest upstream version which is also used by Fedora 37 and by Fedora 38.
- The EPEL repository provides package NetworkManager-strongswan-1.5.2-1.el8 for Red Hat EL8 which lags behind the latest upstream version 1.6.0 which is used by Fedora 37 and by Fedora 38.

Comment 9 Laine Stump 2023-03-17 14:54:04 UTC
(In reply to Joachim Frieben from comment #4)
> The issue concerns package strongswan as implied by mentioning that the
> network connection was set up by means of package
> NetworkManager-strongswan-gnome.

Sorry, I missed that when I read through the problem description.

The output of ifconfig unfortunately doesn't tell us much about how traffic will be routed or how the device plumbing is setup. We can probably assume that the "tap0" device is the tap device created for the guest, and that it is attached to virbr0 (since it's started with gnome-boxes, it will be unprivileged libvirtd, which uses qemu-bridge-helper to create the bridge device and connect it to the bridge).

Can you collect the output of "ip addr show; ip -4 route show; ip -6 route show; nslookup api.ipify.org" on 1) the RHEL8 host and 2) its F37 guest, as well as on 3) the F37 host and 4) its F37 guest. It's possible that the guest & host are resolving that domain to different IPs, which could cause the outbound routing to be different (although I can't imagine why/how).

I'll wait to re-change the component until we see this information.

Comment 10 Joachim Frieben 2023-03-18 12:20:37 UTC
Output of 'ip addr show; ip -4 route show; ip -6 route show; nslookup api.ipify.org' on EL8 host:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 10.1.10.186/32 scope global lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp0s25: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:22:68:1b:74:75 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.216/24 brd 192.168.1.255 scope global dynamic noprefixroute enp0s25
       valid_lft 43178sec preferred_lft 43178sec
    inet6 fd66:404d:bed6:0:222:68ff:fe1b:7475/64 scope global dynamic mngtmpaddr 
       valid_lft forever preferred_lft forever
    inet6 fe80::222:68ff:fe1b:7475/64 scope link 
       valid_lft forever preferred_lft forever
3: wlp3s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether e6:e3:16:46:9f:18 brd ff:ff:ff:ff:ff:ff permaddr 00:21:6a:95:82:d0
4: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 52:54:00:97:a6:a4 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
5: tap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master virbr0 state UNKNOWN group default qlen 1000
    link/ether fe:e6:43:e8:57:c9 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fce6:43ff:fee8:57c9/64 scope link 
       valid_lft forever preferred_lft forever
9: nm-xfrm-3442752@NONE: <NOARP,UP,LOWER_UP> mtu 1400 qdisc noqueue state UNKNOWN group default qlen 1000
    link/none 
    inet 10.1.10.186/32 scope global noprefixroute nm-xfrm-3442752
       valid_lft forever preferred_lft forever
    inet6 fe80::da9d:875f:8d6c:c560/64 scope link stable-privacy 
       valid_lft forever preferred_lft forever
default via 192.168.1.1 dev enp0s25 proto dhcp src 192.168.1.216 metric 100 
192.168.1.0/24 dev enp0s25 proto kernel scope link src 192.168.1.216 metric 100 
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 
::1 dev lo proto kernel metric 256 pref medium
fd66:404d:bed6::/64 dev enp0s25 proto kernel metric 256 pref medium
fe80::/64 dev tap0 proto kernel metric 256 pref medium
fe80::/64 dev enp0s25 proto kernel metric 256 pref medium
fe80::/64 dev nm-xfrm-3442752 proto kernel metric 256 pref medium
Server:		10.1.0.1
Address:	10.1.0.1#53

Non-authoritative answer:
api.ipify.org	canonical name = api4.ipify.org.
Name:	api4.ipify.org
Address: 173.231.16.76
Name:	api4.ipify.org
Address: 64.185.227.155
Name:	api4.ipify.org
Address: 104.237.62.211

Comment 11 Joachim Frieben 2023-03-18 12:22:11 UTC
Output of 'ip addr show; ip -4 route show; ip -6 route show; nslookup api.ipify.org' in F37 VM on EL8 host:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:aa:5c:bb brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    inet 192.168.122.232/24 brd 192.168.122.255 scope global dynamic noprefixroute ens3
       valid_lft 3597sec preferred_lft 3597sec
    inet6 fe80::2b42:dd11:6744:30d2/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
default via 192.168.122.1 dev ens3 proto dhcp src 192.168.122.232 metric 100 
192.168.122.0/24 dev ens3 proto kernel scope link src 192.168.122.232 metric 100 
::1 dev lo proto kernel metric 256 pref medium
fe80::/64 dev ens3 proto kernel metric 1024 pref medium
Server:		127.0.0.53
Address:	127.0.0.53#53

Non-authoritative answer:
api.ipify.org	canonical name = api4.ipify.org.
Name:	api4.ipify.org
Address: 64.185.227.155
Name:	api4.ipify.org
Address: 104.237.62.211
Name:	api4.ipify.org
Address: 173.231.16.76

Comment 12 Joachim Frieben 2023-03-18 13:55:13 UTC
Output of 'ip addr show; ip -4 route show; ip -6 route show; nslookup api.ipify.org' on F37 host:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 10.1.10.186/32 scope global lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp0s25: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:22:68:1b:74:75 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.216/24 brd 192.168.1.255 scope global dynamic noprefixroute enp0s25
       valid_lft 42984sec preferred_lft 42984sec
3: wlp3s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether da:56:1c:70:cf:c0 brd ff:ff:ff:ff:ff:ff permaddr 00:21:6a:95:82:d0
7: nm-xfrm-1849012@NONE: <NOARP,UP,LOWER_UP> mtu 1400 qdisc noqueue state UNKNOWN group default qlen 1000
    link/none 
    inet 10.1.10.186/32 scope global noprefixroute nm-xfrm-1849012
       valid_lft forever preferred_lft forever
    inet6 fe80::8066:bc97:c9c0:1a1b/64 scope link stable-privacy 
       valid_lft forever preferred_lft forever
default via 192.168.1.1 dev enp0s25 proto dhcp src 192.168.1.216 metric 100 
192.168.1.0/24 dev enp0s25 proto kernel scope link src 192.168.1.216 metric 100 
::1 dev lo proto kernel metric 256 pref medium
fe80::/64 dev nm-xfrm-1849012 proto kernel metric 256 pref medium
Server:		127.0.0.53
Address:	127.0.0.53#53

Non-authoritative answer:
api.ipify.org	canonical name = api4.ipify.org.
Name:	api4.ipify.org
Address: 64.185.227.155
Name:	api4.ipify.org
Address: 104.237.62.211
Name:	api4.ipify.org
Address: 173.231.16.76

Comment 13 Joachim Frieben 2023-03-18 13:56:57 UTC
Output of 'ip addr show; ip -4 route show; ip -6 route show; nslookup api.ipify.org' in F37 VM on F37 host:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:d8:d1:f2 brd ff:ff:ff:ff:ff:ff
    inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic noprefixroute enp1s0
       valid_lft 86392sec preferred_lft 86392sec
default via 10.0.2.2 dev enp1s0 proto dhcp src 10.0.2.15 metric 100 
10.0.2.0/24 dev enp1s0 proto kernel scope link src 10.0.2.15 metric 100 
::1 dev lo proto kernel metric 256 pref medium
Server:		127.0.0.53
Address:	127.0.0.53#53

Non-authoritative answer:
api.ipify.org	canonical name = api4.ipify.org.
Name:	api4.ipify.org
Address: 104.237.62.211
Name:	api4.ipify.org
Address: 64.185.227.155
Name:	api4.ipify.org
Address: 173.231.16.76

Comment 14 Joachim Frieben 2023-03-18 14:04:04 UTC
When comparing the interface setup of EL8 as shown in comment 5 and that of F37 then it becomes obvious that libvirt networking has changed significantly. There are no more virbr0 and tap0 devices as is shown below.

Output of 'ifconfig' for IKEv2/IPsec based VPN on F37 host:

enp0s25: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.216  netmask 255.255.255.0  broadcast 192.168.1.255
        ether 00:22:68:1b:74:75  txqueuelen 1000  (Ethernet)
        RX packets 274246  bytes 349336874 (333.1 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 173754  bytes 22155373 (21.1 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 20  memory 0xfc000000-fc020000  

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 333  bytes 37598 (36.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 333  bytes 37598 (36.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

nm-xfrm-1992895: flags=193<UP,RUNNING,NOARP>  mtu 1400
        inet 10.1.10.186  netmask 255.255.255.255
        inet6 fe80::7083:71f1:6bc:f910  prefixlen 64  scopeid 0x20<link>
        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 1000  (UNSPEC)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 9  dropped 9 overruns 0  carrier 9  collisions 0

Comment 15 Laine Stump 2023-03-19 08:07:50 UTC
libvirt networking hasn't changed; your configuration has changed.

Your F37 host doesn't have the libvirt default network running (probably because it doesn't have the package "libvirt-daemon-config-network" installed, but possibly it was just disabled). Because the default network isn't installed/active, gnome-boxes will create the guest with "usermode networking" ("<interface type='user'>, which uses usermode proxies in QEMU's slirp driver rather than a tap device). Because all of the traffic is seen to originate with a socket from an application on the host (rather than coming as packets from a tap device), it will be treated identically to the traffic that really does originate on the host.

If you want the F37 to behave identically to the RHEL8 host, just install the default network config:

    sudo dnf install libvirt-daemon-config-network

or if that package is already installed, you just need to start the network:

    sudo virsh network-start default

(and if you want it to always be started, also run:

    sudo virsh network-autostart default

)

Any new guest configuration you create with gnome-boxes while the libvirt default virtual network is active should be configured to use a tap device connected to the default network's virbr0 rather than the usermode slirp proxies, and my guess is that the F37 host will then behave identically to the RHEL8 host.

(Alternatively, if you want to the RHEL8 host to switch and behave like the F37 host, then "virsh net-destroy default; virsh net-autostart default --disable" and create a new gnome-boxes vm config)

=====

Back to the original issue - if all of the route table outputs were gathered while the VPN was active, then the decision of which traffic goes over the VPN is not being determined by routing decisions (and from the nslookup output on the machines, it appears that all of them are getting the same answers for the destination). I would then guess that (and this is pure speculation, but I can't think of anything else that fits with the data that's been gathered):

1) traffic to be sent over the VPN must get intercepted by a hook somewhere in the networking stack
2) this hook occurs prior to the iptables post-routing filter hook (so the NAT hasn't yet happened)
3) the hook only intercepts traffic that has a source IP matching the host.

A quick search for strongswan config shows that its config can be setup to bypass IPSec processing for certain traffic:

https://docs.strongswan.org/docs/5.9/config/quickstart.html#_passthroughbypass_policies

so you just need to learn how to configure it to do the opposite of that :-) (I do see sections in the configuration for host-to-host vs subnet-to-subnet setups; possibly there is a subnet-to-host config that does what you need?)

Anyway, it appears that the strongswan config is very different from libreswan config, so I am going to switch this to Fedora EPEL / NetworkManager-strongswan, but you may have better luck figuring it out via the upstream strongswan community, wherever that lives. The information I've outlined above about how guest traffic is routed/NATed may be of help to strongswan people in recommending how you should change your configuration.

Comment 16 Joachim Frieben 2023-03-19 10:08:23 UTC
(In reply to Laine Stump from comment #15)
Coming back to the orignal issues all given information does not explain why virtual machines run  by GNOME Boxes under EL8 do expose the host's IP address when an IKEv2/IPsec VPN connection is active whereas no IP leak was observed when an OpenVPN VPN connection was active. The comparison with a F37 host was only meant to help clarify the issue.

Comment 17 Joachim Frieben 2023-03-19 11:48:52 UTC
(In reply to Laine Stump from comment #15)
It turns out that virtnetworkd.service is disabled for a default installations of Fedora 37 (probably for Fedora 38 (development), too, because it behaved the same way).
After activating service virtnetworkd, interface configuration is indeed just like under EL8. Accordingly, the system IP address is exposed by virtual machines when an active IKEv2/IPsec VPN connection is active on the Fedora 37 host whereas no IP address leak is observed when an OpenVPN VPN is active.

Comment 18 Laine Stump 2023-03-19 17:03:57 UTC
(In reply to Joachim Frieben from comment #16)
> Coming back to the orignal issues all given information does not explain why
> virtual machines run  by GNOME Boxes under EL8 do expose the host's IP
> address when an IKEv2/IPsec VPN connection is active whereas no IP leak was
> observed when an OpenVPN VPN connection was active.

Sure it does. OpenVPN just sets up a tun device, then adds a default route such that *all* egressing traffic goes via that tunnel; the tunnel device encrypts all traffic and sends it to the remote end, where it is decrypted and source IP's NATed to the VPN remote endpoint's IP.

Based on the output of 'ip route' you provided, the IPSec implementation *does not* use the host's IP routing table to determine which traffic goes over the VPN (well, either that or the output of 'ip route' that you provided was produced when the IPSec VPN wasn't active...). Instead, it apparently intercepts the traffic "somewhere else" in the kernel, and makes the decision about which packets to encrypt and which to not encrypt based on other more sophisticated criteria (perhaps also looking at the *source* IP of the packets); if that "somewhere else" happens prior to the iptable post-routing hook that NATs the VM's internal IP to the host IP (according to iptables rules added by libvirt), and if it's using source IP to decide which traffic goes over the VPN, then traffic from the guests won't go over the VPN, but will instead follow the "normal" egress path, be NATed by the post-routing iptables hook, and go out onto the internet directly from the host. 

So the difference is in how and where the two very different VPN implementations grab the packets and stuff them into the VPN tunnel, and this is a part of those two VPN implementations, not of libvirt or QEMU.

There is nothing[*] that libvirt (or QEMU) can do to affect this decision (other than you changing the config to use a user-mode proxy instead of a tap device, so that the VM traffic really *is* originating from the host - this is what happened with the previous configuration on the F37 host). The solution is to find someone who knows StrongSWAN configuration (I don't), and find out how to configure StrongSWAN so that it will intercept *all* traffic from the host and send it through the IPSec tunnel.

https://docs.strongswan.org has a lot of information, and that's probably a good starting point. Alternately, you could try using libreswan, which is a part of RHEL8 (not just an EPEL package) and might have more support (I'm not sure, as I haven't investigated the space).

> The comparison with a
> F37 host was only meant to help clarify the issue.

And I was hopeful that the difference could have provided some clue to the cause, but it turned out to be unrelated. It was still worth investigating though, as sometimes problems end up being caused by the seemingly most inconsequential bits. Life is a series of red herrings, with an occasional lobster dinner sprinkled here and there...

(In reply to Joachim Frieben from comment #17)
> It turns out that virtnetworkd.service is disabled for a default
> installations of Fedora 37 (probably for Fedora 38 (development), too,
> because it behaved the same way).

Yeah, it's unrelated to your issue (although it clouded up the waters a bit), but it makes sense that it would be disabled in a fresh F37 install: there is a long-standing problem that, due to an interaction between the way that Fedora Live images are created, the way installations are done after booting from a Live image, and the way that libvirt decides on what subnet to use for the default virtual network, the virtual network of libvirt running inside a nested guest can interfere with the virtual network of the host, and end up disabling all networking in the guest. For several releases of Fedora now, the solution has been to disable libvirt's default network "by default". I hadn't paid attention to the details, and I'm kind of surprised that it's been done by disabling the entire virtnetworkd service, since that would also prevent use of other, admin-created, virtual networks until virtnetworkd was re-enabled (which would also re-enable the default network, again leading to the subnet conflict with the default network). But I'm sure the Fedora maintainers thought through all the alternatives and this on ended up being the least disruptive :-)

====
[*] nothing short of a complete reimplementation of the NAT used by libvirt networks such that it did all the NATing as the traffic came into the host network stack from the guest's tap device rather than as it egressed from the host to the outside; I've asked iptables/nftables people about that in the past, though, and it would be *much* more complicated, and require significant development effort for a limited gain; it would also mean that the host wouldn't be able to differentiate between host-originated and VM-originated traffic, which you don't want, but many other users might require.

Comment 19 Joachim Frieben 2023-03-19 19:49:24 UTC
(In reply to Laine Stump from comment #18)
Using libreswan is not an option because it does not support EAP.


Note You need to log in before you can comment on or make changes to this bug.