Bug 1461282 - kernel: ICMP rate limiting is too aggressive on loopback
kernel: ICMP rate limiting is too aggressive on loopback
Status: POST
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: kernel (Show other bugs)
7.4
Unspecified Unspecified
medium Severity medium
: rc
: ---
Assigned To: Jesper Brouer
Jianlin Shi
:
Depends On: 1458542
Blocks: 1469549 1469551
  Show dependency treegraph
 
Reported: 2017-06-14 02:32 EDT by Florian Weimer
Modified: 2017-08-04 21:37 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1458542
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
localhost-icmp.c (1.63 KB, text/plain)
2017-06-14 02:32 EDT, Florian Weimer
no flags Details

  None (edit)
Description Florian Weimer 2017-06-14 02:32:19 EDT
Created attachment 1287532 [details]
localhost-icmp.c

+++ This bug was initially created as a clone of Bug #1458542 +++

Description of problem:

Aggressive ICMP rate limiting on the loopback interface causes test suite failures.

Version-Release number of selected component (if applicable):

kernel-3.10.0-677.el7.x86_64

How reproducible:

Always.

Steps to Reproduce:
1. Compile localhost-icmp.c.
2. Run ./localhost-icmp

Actual results:

Test runs into a timeout:

localhost-icmp: iteration 50: no ICMP message (poll timeout)

Expected results:

Test completes within a few milliseconds.

Additional info:

https://sourceware.org/ml/libc-alpha/2017-06/msg00167.html
http://marc.info/?l=linux-netdev&m=149656032817085
Comment 1 Jianlin Shi 2017-06-14 04:36:57 EDT
the rate is limited by net.ipv4.icmp_msgs_burst. the default value for it is 50, so you can change its value to 1000 or change ITERATIONS to 50 in your code.

sysctl  -w net.ipv4.icmp_msgs_burst=1000 && ./localhost-icmp
Comment 2 Florian Weimer 2017-06-14 05:33:37 EDT
(In reply to Jianlin Shi from comment #1)
> the rate is limited by net.ipv4.icmp_msgs_burst. the default value for it is
> 50, so you can change its value to 1000 or change ITERATIONS to 50 in your
> code.
> 
> sysctl  -w net.ipv4.icmp_msgs_burst=1000 && ./localhost-icmp

Sure, but the default is way too low, and it is far too easy to hit the rate limit.  In fact, I expect that this will introduce boot delays in some setups (such as those using NIS or other UDP-based RPC services on lcoalhost).
Comment 3 Jesper Brouer 2017-06-14 05:40:36 EDT
Notice I did send a kernel RFC patch, that fix the issue:
 http://lkml.kernel.org/r/20170604163812.602cc089@redhat.com

I'm just not 100% convinced why localhost/loopback traffic require ICMP messages to be excluded from rate limiting?

Sure the reproducer and libc test-case demonstrate that the ratelimiting "works". 
But what use-case need this?

Notice that RFC1812 specify that we _should_ implement rate-limiting, but I cannot see that it recommend to excluded loopback traffic:

 https://tools.ietf.org/html/rfc1812#section-4.3.2.8
Comment 4 Florian Weimer 2017-06-14 05:55:44 EDT
(In reply to Jesper Brouer from comment #3)
> Notice I did send a kernel RFC patch, that fix the issue:
>  http://lkml.kernel.org/r/20170604163812.602cc089@redhat.com
> 
> I'm just not 100% convinced why localhost/loopback traffic require ICMP
> messages to be excluded from rate limiting?
> 
> Sure the reproducer and libc test-case demonstrate that the ratelimiting
> "works". 
> But what use-case need this?

System boot without delays?  The ability to test ICMP-based failover?  (Non-loopback ICMP has been broken for a much longer time; I guess subtleties like these are one reason why people use load balancers with custom IP stacks.)

And isn't this question backwards?  What benefit does rate limiting on localhost bring?  What is the advantage when applications wait for a reply that never comes, instead of learning immediately via ICMP that something is wrong?

> Notice that RFC1812 specify that we _should_ implement rate-limiting, but I
> cannot see that it recommend to excluded loopback traffic:
> 
>  https://tools.ietf.org/html/rfc1812#section-4.3.2.8

That RFC talks about routers.  And in practice, the ICMP handling requirements for routers turned out to be quite different, see RFC 2644.
Comment 5 Jesper Brouer 2017-06-14 07:39:59 EDT
Okay Florian, you convinced me, I posted an upstream patch:

 http://lkml.kernel.org/r/149743965710.19877.13901728209731997446.stgit@firesoul
 http://patchwork.ozlabs.org/patch/775744/

Lets see if upstream people will object to this...
Comment 6 sushil kulkarni 2017-06-16 09:40:49 EDT
After discussions, even though it is a regression, it appears to be a fringe case with no known reported issues. Hence marking it as blocker-. However, recommend fixing this is the next z stream.

-Sushil
Comment 7 sushil kulkarni 2017-06-16 09:42:45 EDT
Just to add..there is also a possible workaround in comment#1.

-Sushil

Note You need to log in before you can comment on or make changes to this bug.