Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1461282

Summary: kernel: ICMP rate limiting is too aggressive on loopback
Product: Red Hat Enterprise Linux 7 Reporter: Florian Weimer <fweimer>
Component: kernelAssignee: Jesper Brouer <jbrouer>
kernel sub component: Networking QA Contact: Jianlin Shi <jishi>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: medium CC: atragler, fweimer, jbrouer, jiji, jishi, kzhang, rkhan, sdubroca, sukulkar
Version: 7.4   
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: kernel-3.10.0-703.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1458542 Environment:
Last Closed: 2018-04-10 20:47:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1458542    
Bug Blocks: 1469549, 1469551    
Attachments:
Description Flags
localhost-icmp.c none

Description Florian Weimer 2017-06-14 06:32:19 UTC 
Created attachment 1287532 [details]
localhost-icmp.c

+++ This bug was initially created as a clone of Bug #1458542 +++

Description of problem:

Aggressive ICMP rate limiting on the loopback interface causes test suite failures.

Version-Release number of selected component (if applicable):

kernel-3.10.0-677.el7.x86_64

How reproducible:

Always.

Steps to Reproduce:
1. Compile localhost-icmp.c.
2. Run ./localhost-icmp

Actual results:

Test runs into a timeout:

localhost-icmp: iteration 50: no ICMP message (poll timeout)

Expected results:

Test completes within a few milliseconds.

Additional info:

https://sourceware.org/ml/libc-alpha/2017-06/msg00167.html
http://marc.info/?l=linux-netdev&m=149656032817085
Comment 1 Jianlin Shi 2017-06-14 08:36:57 UTC 
the rate is limited by net.ipv4.icmp_msgs_burst. the default value for it is 50, so you can change its value to 1000 or change ITERATIONS to 50 in your code.

sysctl  -w net.ipv4.icmp_msgs_burst=1000 && ./localhost-icmp
Comment 2 Florian Weimer 2017-06-14 09:33:37 UTC 
(In reply to Jianlin Shi from comment #1)
> the rate is limited by net.ipv4.icmp_msgs_burst. the default value for it is
> 50, so you can change its value to 1000 or change ITERATIONS to 50 in your
> code.
> 
> sysctl  -w net.ipv4.icmp_msgs_burst=1000 && ./localhost-icmp

Sure, but the default is way too low, and it is far too easy to hit the rate limit.  In fact, I expect that this will introduce boot delays in some setups (such as those using NIS or other UDP-based RPC services on lcoalhost).
Comment 3 Jesper Brouer 2017-06-14 09:40:36 UTC 
Notice I did send a kernel RFC patch, that fix the issue:
 http://lkml.kernel.org/r/20170604163812.602cc089@redhat.com

I'm just not 100% convinced why localhost/loopback traffic require ICMP messages to be excluded from rate limiting?

Sure the reproducer and libc test-case demonstrate that the ratelimiting "works". 
But what use-case need this?

Notice that RFC1812 specify that we _should_ implement rate-limiting, but I cannot see that it recommend to excluded loopback traffic:

 https://tools.ietf.org/html/rfc1812#section-4.3.2.8
Comment 4 Florian Weimer 2017-06-14 09:55:44 UTC 
(In reply to Jesper Brouer from comment #3)
> Notice I did send a kernel RFC patch, that fix the issue:
>  http://lkml.kernel.org/r/20170604163812.602cc089@redhat.com
> 
> I'm just not 100% convinced why localhost/loopback traffic require ICMP
> messages to be excluded from rate limiting?
> 
> Sure the reproducer and libc test-case demonstrate that the ratelimiting
> "works". 
> But what use-case need this?

System boot without delays?  The ability to test ICMP-based failover?  (Non-loopback ICMP has been broken for a much longer time; I guess subtleties like these are one reason why people use load balancers with custom IP stacks.)

And isn't this question backwards?  What benefit does rate limiting on localhost bring?  What is the advantage when applications wait for a reply that never comes, instead of learning immediately via ICMP that something is wrong?

> Notice that RFC1812 specify that we _should_ implement rate-limiting, but I
> cannot see that it recommend to excluded loopback traffic:
> 
>  https://tools.ietf.org/html/rfc1812#section-4.3.2.8

That RFC talks about routers.  And in practice, the ICMP handling requirements for routers turned out to be quite different, see RFC 2644.
Comment 5 Jesper Brouer 2017-06-14 11:39:59 UTC 
Okay Florian, you convinced me, I posted an upstream patch:

 http://lkml.kernel.org/r/149743965710.19877.13901728209731997446.stgit@firesoul
 http://patchwork.ozlabs.org/patch/775744/

Lets see if upstream people will object to this...
Comment 6 sushil kulkarni 2017-06-16 13:40:49 UTC 
After discussions, even though it is a regression, it appears to be a fringe case with no known reported issues. Hence marking it as blocker-. However, recommend fixing this is the next z stream.

-Sushil
Comment 7 sushil kulkarni 2017-06-16 13:42:45 UTC 
Just to add..there is also a possible workaround in comment#1.

-Sushil
Comment 9 Rafael Aquini 2017-08-18 22:53:20 UTC 
Patch(es) committed on kernel repository and an interim kernel build is undergoing testing
Comment 11 Rafael Aquini 2017-08-21 11:41:56 UTC 
Patch(es) available on kernel-3.10.0-703.el7
Comment 13 Jianlin Shi 2017-10-26 00:59:22 UTC 
verified on 3.10.0-753:

[root@bkr-hv03-guest30 bz1461282]# uname -a
Linux bkr-hv03-guest30.dsal.lab.eng.bos.redhat.com 3.10.0-753.el7.x86_64 #1 SMP Tue Oct 24 20:20:11 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux
[root@bkr-hv03-guest30 bz1461282]# ./localhost-icmp
Comment 17 Jianlin Shi 2017-10-26 10:01:19 UTC 
reproducer failed on RHEL-7.4(3.10.0-693):
https://beaker.engineering.redhat.com/jobs/2113308

passed on RHEL-7.5-20171023.n.0(3.10.0-743):
https://beaker.engineering.redhat.com/jobs/2113306
Comment 18 errata-xmlrpc 2018-04-10 20:47:23 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:1062