Bug 1200635

Summary: ppp0: hw csum failure
Product: [Fedora] Fedora Reporter: Kristian McColm <kristianmccolm>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 21CC: dkocher, gansalmon, itamar, jonathan, kernel-maint, kristianmccolm, madhu.chinakonda, mchehab, ondrej.dolak
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-05-29 11:23:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Kristian McColm 2015-03-11 03:15:59 UTC
Description of problem:
When establishing an inbound IPSec/L2TP VPN tunnel with Strongswan/ppp/xl2tpd, each incoming packet causes the message "ppp0: hw csum failure" in /var/log/messages:

Mar 10 16:35:31 gw kernel: [133272.735620] ppp0: hw csum failure
Mar 10 16:35:31 gw kernel: [133272.735719] CPU: 1 PID: 26594 Comm: firewall-cmd Not tainted 3.18.7-200.fc21.x86_64 #1
Mar 10 16:35:31 gw kernel: [133272.735720] Hardware name: ASUS All Series/Z87-K, BIOS 1401 07/29/2014
Mar 10 16:35:31 gw kernel: [133272.735721]  0000000000000000 0000000017b5c96d ffff88011fb03a98 ffffffff8175e686
Mar 10 16:35:31 gw kernel: [133272.735723]  0000000000000000 ffff8800bebcd000 ffff88011fb03ab8 ffffffff81647122
Mar 10 16:35:31 gw kernel: [133272.735725]  ffffffff816358f0 ffff8800b6b64500 ffff88011fb03af8 ffffffff8163e305
Mar 10 16:35:31 gw kernel: [133272.735726] Call Trace:
Mar 10 16:35:31 gw kernel: [133272.735727]  <IRQ>  [<ffffffff8175e686>] dump_stack+0x46/0x58
Mar 10 16:35:31 gw kernel: [133272.735735]  [<ffffffff81647122>] netdev_rx_csum_fault+0x42/0x50
Mar 10 16:35:31 gw kernel: [133272.735737]  [<ffffffff816358f0>] ? reqsk_fastopen_remove+0x150/0x150
Mar 10 16:35:31 gw kernel: [133272.735740]  [<ffffffff8163e305>] __skb_checksum_complete+0xc5/0xd0
Mar 10 16:35:31 gw kernel: [133272.735743]  [<ffffffff816dea6a>] nf_ip_checksum+0xea/0x120
Mar 10 16:35:31 gw kernel: [133272.735751]  [<ffffffffa0474973>] tcp_error+0xe3/0x230 [nf_conntrack]
Mar 10 16:35:31 gw kernel: [133272.735755]  [<ffffffffa046f94b>] nf_conntrack_in+0xeb/0xa60 [nf_conntrack]
Mar 10 16:35:31 gw kernel: [133272.735757]  [<ffffffff81649bd2>] ? __netif_receive_skb_core+0x582/0x820
Mar 10 16:35:31 gw kernel: [133272.735761]  [<ffffffff81689760>] ? inet_del_offload+0x40/0x40
Mar 10 16:35:31 gw kernel: [133272.735770]  [<ffffffffa049a3a2>] ipv4_conntrack_in+0x22/0x30 [nf_conntrack_ipv4]
Mar 10 16:35:31 gw kernel: [133272.735772]  [<ffffffff816802ea>] nf_iterate+0xaa/0xc0
Mar 10 16:35:31 gw kernel: [133272.735775]  [<ffffffff81689760>] ? inet_del_offload+0x40/0x40
Mar 10 16:35:31 gw kernel: [133272.735776]  [<ffffffff81680384>] nf_hook_slow+0x84/0x130
Mar 10 16:35:31 gw kernel: [133272.735778]  [<ffffffff81689760>] ? inet_del_offload+0x40/0x40
Mar 10 16:35:31 gw kernel: [133272.735779]  [<ffffffff8168a1ce>] ip_rcv+0x2fe/0x3a0
Mar 10 16:35:31 gw kernel: [133272.735781]  [<ffffffff81649bd2>] __netif_receive_skb_core+0x582/0x820
Mar 10 16:35:31 gw kernel: [133272.735784]  [<ffffffff81649e8a>] __netif_receive_skb+0x1a/0x80
Mar 10 16:35:31 gw kernel: [133272.735785]  [<ffffffff8164ab72>] process_backlog+0xa2/0x150
Mar 10 16:35:31 gw kernel: [133272.735787]  [<ffffffff8164a33a>] net_rx_action+0x13a/0x240
Mar 10 16:35:31 gw kernel: [133272.735790]  [<ffffffff8109d7a4>] __do_softirq+0x124/0x2d0
Mar 10 16:35:31 gw kernel: [133272.735792]  [<ffffffff8109db75>] irq_exit+0x125/0x130
Mar 10 16:35:31 gw kernel: [133272.735795]  [<ffffffff817680fa>] do_IRQ+0x5a/0xf0
Mar 10 16:35:31 gw kernel: [133272.735797]  [<ffffffff81765f2d>] common_interrupt+0x6d/0x6d
Mar 10 16:35:31 gw kernel: [133272.735798]  <EOI>


Version-Release number of selected component (if applicable):
[root@gw ~]# uname -r
3.18.7-200.fc21.x86_64
[root@gw ~]# rpm -qva strongswan xl2tpd ppp
xl2tpd-1.3.6-6.fc21.x86_64
strongswan-5.2.0-4.fc21.x86_64
ppp-2.4.7-6.fc21.x86_64


How reproducible:
Always

Steps to Reproduce:
1. Configure Strongswan/xl2tpd/ppp as an IPSec/L2TP VPN server
2. Establish a VPN connection from a remote host
3. Generate packets over the VPN tunnel

Actual results:
Log flooded with kernel messages

Expected results:
No log flooding

Additional info:
N/A

Comment 1 Kristian McColm 2015-03-11 03:26:14 UTC
Possible duplicate bug which was opened against FC20: 

https://bugzilla.redhat.com/show_bug.cgi?id=1148612

Comment 2 Ondrej Dolak 2015-03-20 19:30:36 UTC
Same issue on my PC with IPSec/L2TP VPN connection

/var/log/messages
Mar 20 20:19:17 localhost kernel: ppp0: hw csum failure
Mar 20 20:19:17 localhost kernel: CPU: 1 PID: 0 Comm: swapper/1 Tainted: P           OE  3.18.9-200.fc21.x86_64 #1
Mar 20 20:19:17 localhost kernel: Hardware name: System manufacturer P5Q-E/P5Q-E, BIOS 2101    04/06/2009
Mar 20 20:19:17 localhost kernel: 0000000000000000 2aa69f79e173cdac ffff88012fc83bc8 ffffffff8175b71c
Mar 20 20:19:17 localhost kernel: 0000000000000000 ffff88009571c000 ffff88012fc83be8 ffffffff81644e02
Mar 20 20:19:17 localhost kernel: ffffffff81633520 ffff88011b7a8700 ffff88012fc83c28 ffffffff8163bf35
Mar 20 20:19:17 localhost kernel: Call Trace:
Mar 20 20:19:17 localhost kernel: <IRQ>  [<ffffffff8175b71c>] dump_stack+0x46/0x58
Mar 20 20:19:17 localhost kernel: [<ffffffff81644e02>] netdev_rx_csum_fault+0x42/0x50
Mar 20 20:19:17 localhost kernel: [<ffffffff81633520>] ? reqsk_fastopen_remove+0x150/0x150
Mar 20 20:19:17 localhost kernel: [<ffffffff8163bf35>] __skb_checksum_complete+0xc5/0xd0
Mar 20 20:19:17 localhost kernel: [<ffffffff816ae598>] tcp_v4_rcv+0x338/0x9a0
Mar 20 20:19:17 localhost kernel: [<ffffffff816875e0>] ? ip_rcv_finish+0x370/0x370
Mar 20 20:19:17 localhost kernel: [<ffffffff8167dea4>] ? nf_hook_slow+0x84/0x130
Mar 20 20:19:17 localhost kernel: [<ffffffff816875e0>] ? ip_rcv_finish+0x370/0x370
Mar 20 20:19:17 localhost kernel: [<ffffffff8168768e>] ip_local_deliver_finish+0xae/0x230
Mar 20 20:19:17 localhost kernel: [<ffffffff816879a8>] ip_local_deliver+0x48/0x80
Mar 20 20:19:17 localhost kernel: [<ffffffff816872f8>] ip_rcv_finish+0x88/0x370
Mar 20 20:19:17 localhost kernel: [<ffffffff81687c72>] ip_rcv+0x292/0x3a0
Mar 20 20:19:17 localhost kernel: [<ffffffff81647792>] __netif_receive_skb_core+0x582/0x820
Mar 20 20:19:17 localhost kernel: [<ffffffff81647a48>] __netif_receive_skb+0x18/0x60
Mar 20 20:19:17 localhost kernel: [<ffffffff81648712>] process_backlog+0xa2/0x150
Mar 20 20:19:17 localhost kernel: [<ffffffff81647eda>] net_rx_action+0x13a/0x240
Mar 20 20:19:17 localhost kernel: [<ffffffff8109d3ac>] __do_softirq+0x10c/0x2b0
Mar 20 20:19:17 localhost kernel: [<ffffffff8109d775>] irq_exit+0x125/0x130
Mar 20 20:19:17 localhost kernel: [<ffffffff81764dfa>] do_IRQ+0x5a/0xf0
Mar 20 20:19:17 localhost kernel: [<ffffffff81762c2d>] common_interrupt+0x6d/0x6d
Mar 20 20:19:17 localhost kernel: <EOI>  [<ffffffff810ff7a8>] ? hrtimer_start+0x18/0x20
Mar 20 20:19:17 localhost kernel: [<ffffffff8105ca56>] ? native_safe_halt+0x6/0x10
Mar 20 20:19:17 localhost kernel: [<ffffffff810f7443>] ? rcu_eqs_enter+0xa3/0xb0
Mar 20 20:19:17 localhost kernel: [<ffffffff8101e98f>] default_idle+0x1f/0xc0
Mar 20 20:19:17 localhost kernel: [<ffffffff8101f35f>] arch_cpu_idle+0xf/0x20
Mar 20 20:19:17 localhost kernel: [<ffffffff810da09c>] cpu_startup_entry+0x37c/0x3c0
Mar 20 20:19:17 localhost kernel: [<ffffffff81049dcf>] start_secondary+0x1af/0x1f0


Components:
[root@dsk log]# uname -r
3.18.9-200.fc21.x86_64
[root@dsk log]# rpm -qva libreswan xl2tpd ppp
xl2tpd-1.3.6-6.fc21.x86_64
ppp-2.4.7-6.fc21.x86_64
libreswan-3.12-1.fc21.x86_64

Comment 3 Darrien Lambert 2015-03-20 22:37:38 UTC
I have filed the bug report for upstream kernel: https://bugzilla.kernel.org/show_bug.cgi?id=95171

Comment 4 Fedora Kernel Team 2015-04-28 18:32:31 UTC
*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There is a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 21 kernel bugs.

Fedora 21 has now been rebased to 3.19.5-200.fc21.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you have moved on to Fedora 22, and are still experiencing this issue, please change the version to Fedora 22.

If you experience different issues, please open a new bug report for those.

Comment 5 Ondrej Dolak 2015-04-28 19:25:05 UTC
The bug is still present in 3.19.5-200.fc21.x86_64

Comment 6 Ondrej Dolak 2015-05-29 06:41:08 UTC
Kernel version 4.0.4-201.fc21.x86_64 fixed this bug (also see upstream info)