RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1932202 - ESP over TCP issues
Summary: ESP over TCP issues
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: kernel
Version: 8.4
Hardware: All
OS: Linux
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Sabrina Dubroca
QA Contact: xmu
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-02-24 08:48 UTC by Ondrej Moriš
Modified: 2021-03-09 19:18 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-03-09 19:18:49 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description Ondrej Moriš 2021-02-24 08:48:19 UTC 
Description of problem:

Since RHEL-8.4 libreswan (4.3) supports ESP over TCP. Kernel supports this since RHEL-8.3 (BZ#1446392). However, then tunnel is created everything seems to be correct except that no traffic goes through:

# ipsec auto --up test
181 "test" #1: initiating IKEv2 connection
181 "test" #1: sent IKE_SA_INIT request
182 "test" #1: sent IKE_AUTH request {auth=IKEv2 cipher=AES_GCM_16_256 integ=n/a prf=HMAC_SHA2_512 group=MODP2048}
002 "test" #2: IKEv2 mode peer ID is ID_IPV4_ADDR: '10.0.139.178'
003 "test" #1: authenticated using authby=secret
002 "test" #2: negotiated connection [10.0.138.36-10.0.138.36:0-65535 0] -> [10.0.139.178-10.0.139.178:0-65535 0]
004 "test" #2: IPsec SA established tunnel mode {ESPinTCP=>0x43837daa <0xad9cde5a xfrm=AES_GCM_16_256-NONE NATOA=none NATD=none DPD=passive}

# ip xfrm state
src 10.0.139.178 dst 10.0.138.36
        proto esp spi 0xad9cde5a reqid 16389 mode tunnel
        replay-window 32 flag af-unspec
        aead rfc4106(gcm(aes)) 0x162f588e5f70ad0de21948a96aef89d62032fd8f6191d6795c219a69d509b32e2968e8e4 128
        encap type espintcp sport 4500 dport 57284 addr 0.0.0.0
        anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
src 10.0.138.36 dst 10.0.139.178
        proto esp spi 0x43837daa reqid 16389 mode tunnel
        replay-window 32 flag af-unspec
        aead rfc4106(gcm(aes)) 0x6588e53fda1b7d9a4f523a45d4efe7d4d74f12d7a8c7aa0f889798139471ddba1dc9df49 128
        encap type espintcp sport 57284 dport 4500 addr 0.0.0.0
        anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000

# ip xfrm policy
src 10.0.138.36/32 dst 10.0.139.178/32
        dir out priority 2080702 ptype main
        tmpl src 10.0.138.36 dst 10.0.139.178
                proto esp reqid 16389 mode tunnel
src 10.0.139.178/32 dst 10.0.138.36/32
        dir fwd priority 2080702 ptype main
        tmpl src 10.0.139.178 dst 10.0.138.36
                proto esp reqid 16389 mode tunnel
src 10.0.139.178/32 dst 10.0.138.36/32
        dir in priority 2080702 ptype main
        tmpl src 10.0.139.178 dst 10.0.138.36
                proto esp reqid 16389 mode tunnel

# ping -n -c 4 -I eth0 10.0.139.178
PING 10.0.139.178 (10.0.139.178) from 10.0.138.36 eth0: 56(84) bytes of data.
 
--- 10.0.139.178 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3095ms

Also, /proc/net/xfrm/state reports non-zero values of  XfrmOutNoStates and XfrmInTmplMismatch.

Version-Release number of selected component (if applicable):

kernel-4.18.0-287.el8
libreswan-4.3-1.el8

How reproducible:

100%

Steps to Reproduce:

1. Turn off selinux (BZ#1931848) on both client and server.

2. Configure libreswan to use IKEv2/ESP over TCP:

   a. # cat /etc/ipsec.conf
      version 2.0

      config setup
        plutodebug="all"
        logappend=no
        plutostderrlog="/tmp/pluto.log"
        listen-tcp=yes
 
      conn test
        authby=secret
        left=<SERVER-IPv4>
        right=<CLIENT-IPv4>
        ikev2=insist
        enable-tcp=yes
        tcp-remoteport=4500

   b. # cat /etc/ipsec.secrets
      : PSK "fo0m4nchU"

3. Start ipsec service and add "test" connection on both sides:
   
   # service ipsec start
   # ipsec auto --add test

4. Initiate connection from the client:

   # ipsec auto --up test

5. Ping server from the client.

   # ping -c 3 <CLIENT-IPv4>

Actual results:

Connection is established successfully (see the description above). But no ICMP traffic goes through.

Expected results:

ICMP connection works.

Additional info:

The same behaviour can be observed with 5.8.18-200.fc32 and 5.10.13-200.fc33.
Comment 1 Ondrej Moriš 2021-02-24 08:56:41 UTC 
Full XFRM policy actually is as follows:

src 10.0.137.79/32 dst 10.0.136.25/32 
	dir out priority 2080702 ptype main 
	tmpl src 10.0.137.79 dst 10.0.136.25
		proto esp reqid 16389 mode tunnel
src 10.0.136.25/32 dst 10.0.137.79/32 
	dir fwd priority 2080702 ptype main 
	tmpl src 10.0.136.25 dst 10.0.137.79
		proto esp reqid 16389 mode tunnel
src 10.0.136.25/32 dst 10.0.137.79/32 
	dir in priority 2080702 ptype main 
	tmpl src 10.0.136.25 dst 10.0.137.79
		proto esp reqid 16389 mode tunnel
src ::/0 dst ::/0 
	socket out priority 0 ptype main 
src ::/0 dst ::/0 
	socket in priority 0 ptype main 
src ::/0 dst ::/0 
	socket out priority 0 ptype main 
src ::/0 dst ::/0 
	socket in priority 0 ptype main 
src 0.0.0.0/0 dst 0.0.0.0/0 
	socket out priority 0 ptype main 
src 0.0.0.0/0 dst 0.0.0.0/0 
	socket in priority 0 ptype main 
src 0.0.0.0/0 dst 0.0.0.0/0 
	socket out priority 0 ptype main 
src 0.0.0.0/0 dst 0.0.0.0/0 
	socket in priority 0 ptype main 
src 0.0.0.0/0 dst 0.0.0.0/0 
	socket out priority 0 ptype main 
src 0.0.0.0/0 dst 0.0.0.0/0 
	socket in priority 0 ptype main 
src 0.0.0.0/0 dst 0.0.0.0/0 
	socket out priority 0 ptype main 
src 0.0.0.0/0 dst 0.0.0.0/0 
	socket in priority 0 ptype main 
src ::/0 dst ::/0 proto ipv6-icmp type 135 
	dir out priority 1 ptype main 
src ::/0 dst ::/0 proto ipv6-icmp type 135 
	dir fwd priority 1 ptype main 
src ::/0 dst ::/0 proto ipv6-icmp type 135 
	dir in priority 1 ptype main 
src ::/0 dst ::/0 proto ipv6-icmp type 136 
	dir out priority 1 ptype main 
src ::/0 dst ::/0 proto ipv6-icmp type 136 
	dir fwd priority 1 ptype main 
src ::/0 dst ::/0 proto ipv6-icmp type 136 
	dir in priority 1 ptype main
Comment 3 Sabrina Dubroca 2021-02-24 12:19:40 UTC 
Looking for IP_XFRM_POLICY/IP_IPSEC_POLICY (optname=16/optname=17) and TCP_ULP=espintcp (which ends up in espintcp_init_sk):

           pluto  9478 [020]   432.271046: probe:do_ip_setsockopt: (ffffffff8642ed70) sk=0xffff9a17a4929400 optname=17
           pluto  9478 [020]   432.271048: probe:do_ip_setsockopt: (ffffffff8642ed70) sk=0xffff9a17a4929400 optname=17
           pluto  9478 [020]   432.271077: probe:do_ip_setsockopt: (ffffffff8642ed70) sk=0xffff9a17a492a300 optname=17
           pluto  9478 [020]   432.271078: probe:do_ip_setsockopt: (ffffffff8642ed70) sk=0xffff9a17a492a300 optname=17
           pluto  9478 [020]   432.271229: probe:do_ip_setsockopt: (ffffffff8642ed70) sk=0xffff9a178cb23a80 optname=17
           pluto  9478 [020]   432.271230: probe:do_ip_setsockopt: (ffffffff8642ed70) sk=0xffff9a178cb23a80 optname=17
           pluto  9478 [020]   432.271288: probe:do_ip_setsockopt: (ffffffff8642ed70) sk=0xffff9a17a492c100 optname=17
           pluto  9478 [020]   432.271290: probe:do_ip_setsockopt: (ffffffff8642ed70) sk=0xffff9a17a492c100 optname=17
           pluto  9478 [020]   432.271312: probe:do_ip_setsockopt: (ffffffff8642ed70) sk=0xffff9a17a4928f00 optname=17
           pluto  9478 [020]   432.271314: probe:do_ip_setsockopt: (ffffffff8642ed70) sk=0xffff9a17a4928f00 optname=17
           pluto  9478 [020]   432.271338: probe:do_ip_setsockopt: (ffffffff8642ed70) sk=0xffff9a178cb26b40 optname=17
           pluto  9478 [020]   432.271339: probe:do_ip_setsockopt: (ffffffff8642ed70) sk=0xffff9a178cb26b40 optname=17
           pluto  9478 [020]   432.271361: probe:do_ip_setsockopt: (ffffffff8642ed70) sk=0xffff9a17a4928000 optname=17
           pluto  9478 [020]   432.271363: probe:do_ip_setsockopt: (ffffffff8642ed70) sk=0xffff9a17a4928000 optname=17
           pluto  9478 [020]   432.271384: probe:do_ip_setsockopt: (ffffffff8642ed70) sk=0xffff9a17a4928500 optname=17
           pluto  9478 [020]   432.271385: probe:do_ip_setsockopt: (ffffffff8642ed70) sk=0xffff9a17a4928500 optname=17
           pluto  9478 [020]   432.271408: probe:do_ip_setsockopt: (ffffffff8642ed70) sk=0xffff9a178cb230c0 optname=17
           pluto  9478 [020]   432.271409: probe:do_ip_setsockopt: (ffffffff8642ed70) sk=0xffff9a178cb230c0 optname=17
           pluto  9478 [020]   454.970934: probe:espintcp_init_sk: (ffffffff864aaab0) sk=0xffff9a178cb22700            <--- connected encap socket with TCP_ULP

The actual encap socket needs the IP_XFRM_POLICY setsockopt as well, otherwise its packets will be dropped.

Did this ever work properly?
Comment 4 Paul Wouters 2021-02-24 18:06:29 UTC 
sdubroca: yes the current code is working fine upstream:

https://testing.libreswan.org/v4.3-17-g542a96f597-main/ikev2-tcp-00-yes/west.console.txt
Comment 5 Paul Wouters 2021-02-24 19:56:19 UTC 
I added some more kernel state display in the test case, see: https://github.com/libreswan/libreswan/blob/main/testing/pluto/ikev2-tcp-00-yes/west.console.txt
Comment 7 Ondrej Moriš 2021-02-25 10:42:52 UTC 
I confirm that. Connection work on RHEL-8.4 with kernel-4.18.0-291.el8.x86_64 and libreswan-4.3-1.el8.x86_64 with the following configuration:

version 2.0

config setup
 plutodebug="all"
 logappend=no
 plutostderrlog="/tmp/pluto.log"
 listen-tcp=yes
 
conn test
 authby=secret
 leftsubnet=10.1.136.0/24
 rightsubnet=10.1.139.0/24
 left=10.0.136.175
 right=10.0.139.226
 rightnexthop=10.0.136.175
 leftnexthop=10.0.139.226
 leftsourceip=10.1.136.254
 rightsourceip=10.1.139.254
 ikev2=insist
 enable-tcp=yes

CLIENT> ipsec auto --up test
181 "test" #1: initiating IKEv2 connection
181 "test" #1: sent IKE_SA_INIT request
182 "test" #1: sent IKE_AUTH request {auth=IKEv2 cipher=AES_GCM_16_256 integ=n/a prf=HMAC_SHA2_512 group=MODP2048}
002 "test" #1: IKEv2 mode peer ID is ID_IPV4_ADDR: '10.0.136.175'
003 "test" #1: authenticated using authby=secret
002 "test" #2: negotiated connection [10.1.139.0-10.1.139.255:0-65535 0] -> [10.1.136.0-10.1.136.255:0-65535 0]
004 "test" #2: IPsec SA established tunnel mode {ESPinTCP=>0x59985543 <0x16e561d1 xfrm=AES_GCM_16_256-NONE NATOA=none NATD=none DPD=passive}

CLIENT> ip xfrm state
src 10.0.136.175 dst 10.0.139.226
	proto esp spi 0x16e561d1 reqid 16389 mode tunnel
	replay-window 32 flag af-unspec
	aead rfc4106(gcm(aes)) 0xdc3e4f9b2bd766ba35bb7802f54e0d8ab418dd90d2b6b7e99dccccc67458645863f4831b 128
	encap type espintcp sport 4500 dport 36096 addr 0.0.0.0
	anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
src 10.0.139.226 dst 10.0.136.175
	proto esp spi 0x59985543 reqid 16389 mode tunnel
	replay-window 32 flag af-unspec
	aead rfc4106(gcm(aes)) 0x77cd938bc5deaf9c9478f04a578b51f8a12ee5f616c00552bc39877ab17c6eae519c5314 128
	encap type espintcp sport 36096 dport 4500 addr 0.0.0.0
	anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000

CLIENT> ping -n -c4 10.1.136.254
PING 10.1.136.254 (10.1.136.254) 56(84) bytes of data.
64 bytes from 10.1.136.254: icmp_seq=1 ttl=64 time=1.20 ms
64 bytes from 10.1.136.254: icmp_seq=2 ttl=64 time=0.712 ms
64 bytes from 10.1.136.254: icmp_seq=3 ttl=64 time=0.531 ms
64 bytes from 10.1.136.254: icmp_seq=4 ttl=64 time=0.902 ms

--- 10.1.136.254 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3088ms
rtt min/avg/max/mdev = 0.531/0.836/1.201/0.249 ms
CONF> ipsec trafficstatus
006 #2: "test", type=ESP, add_time=1614249494, inBytes=336, outBytes=336, id='10.0.136.175'

But as Sabrina pointed out - once I switch to different configuration - the one from the bug description, connection is established but all packets are lost. 

So it really boils down to libreswan missing some TCP socket policies.
Comment 8 Paul Wouters 2021-02-26 17:54:07 UTC 
note that conn is broken:

 leftsubnet=10.1.136.0/24
 rightsubnet=10.1.139.0/24
 left=10.0.136.175
 right=10.0.139.226

Basically your remote peer is within the remote subnet, so what do you expect to happen for a packet with destination 10.0.136.175 ? It would get encrypted and then send to 10.0.136.175, which would need to get encrypted.......

You could excempt the remote peer using:

conn passthrough
      left=10.0.136.175
      right=10.0.139.226
      authby=never
      type=passthrough

which would override the /32 to /32 from the /24 to /24


Note< i tested UDP encap transport mode, and it seems to work fine: https://github.com/libreswan/libreswan/tree/main/testing/pluto/ikev2-udp-05-transport-mode

 ipsec whack --trafficstatus
006 #2: "ikev2-west-east", type=ESP, add_time=1234567890, inBytes=256, outBytes=256, id='@east'
west #
 # should show tcp being used
west #
 ../../pluto/bin/ipsec-look.sh | grep encap
	encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
	encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
west #
 ../../pluto/bin/ipsec-look.sh
west NOW
XFRM state:
src 192.1.2.23 dst 192.1.2.45
	proto esp spi 0xSPISPI reqid REQID mode transport
	replay-window 32 
	aead rfc4106(gcm(aes)) 0xENCAUTHKEY 128
	encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
	sel src 192.1.2.23/32 dst 192.1.2.45/32 
src 192.1.2.45 dst 192.1.2.23
	proto esp spi 0xSPISPI reqid REQID mode transport
	replay-window 32 
	aead rfc4106(gcm(aes)) 0xENCAUTHKEY 128
	encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
	sel src 192.1.2.45/32 dst 192.1.2.23/32 
XFRM policy:
src 192.1.2.23/32 dst 192.1.2.45/32
	dir in priority 2080702 ptype main
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid REQID mode transport
src 192.1.2.45/32 dst 192.1.2.23/32
	dir out priority 2080702 ptype main
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid REQID mode transport
XFRM done

The exact same configuration, but with TCP, fails
Comment 12 Ondrej Moriš 2021-03-09 19:18:49 UTC 
Since this is not a bug in kernel but libreswan bug (BZ#1372050) I am going to close this bug. Thanks Sabrina and Paul for investigation and completing fixes for libreswan.
Note You need to log in before you can comment on or make changes to this bug.