Bug 972690 - Virtual network doesn't start when there is a big routing table
Virtual network doesn't start when there is a big routing table
Status: CLOSED WONTFIX
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: libvirt (Show other bugs)
5.5
Unspecified Unspecified
unspecified Severity unspecified
: rc
: ---
Assigned To: Laine Stump
Virtualization Bugs
:
Depends On:
Blocks: 1049888
  Show dependency treegraph
 
Reported: 2013-06-10 08:13 EDT by Martin Edlman
Modified: 2014-02-04 10:08 EST (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-12-04 10:39:53 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Martin Edlman 2013-06-10 08:13:18 EDT
Description of problem:
Virtual network doesn't start when there is a big routing table. We have a server running Zebra routing daemon and there are 670 routes in the routing table. Libvirt complains

error: Failed to start network Servers
error: Failed to read file '/proc/net/route': Value too large for defined data type

Another virtual network LTSP is running, it was started before the routing table grew up.

Version-Release number of selected component (if applicable):
libvirt-0.8.2

How reproducible:
Start Zebra (and its companions) and then start libvirt. This is the normal start up order in the /etc/init.d

Steps to Reproduce:
1.
2.
3.

Actual results:
Virtual network Servers is not up and not running

Expected results:
Virtual network Servers is up and running

Additional info:
When the Zebra is stopped and RT has only few items, virtual nets start up as expected.
Comment 1 Dave Allan 2013-06-18 10:50:20 EDT
Martin, are you able to determine if this behavior exists with more recent libvirt, either that from RHEL6 or the upstream packages?
Comment 2 Laine Stump 2013-06-18 13:00:35 EDT
This is the problem (in src/network/bridge_driver.c/networkCheckRouteCollision()):

   enum {MAX_ROUTE_SIZE = 1024*64};

This size is used for the buffer allocated to hold the entire contents of /proc/net/route. The above is a bit misleading, though, because it turns out that each line in /proc/net/route is 128 characters long, not 64. So the maximum number of routes supported is 512.

I just posted this simple patch upstream:

https://www.redhat.com/archives/libvir-list/2013-June/msg00719.html
Comment 3 Martin Edlman 2013-06-19 01:02:15 EDT
(In reply to Dave Allan from comment #1)
> Martin, are you able to determine if this behavior exists with more recent
> libvirt, either that from RHEL6 or the upstream packages?

The system is up to date, there is no newer libvirt available. I cannot test it on RHEL6 as we have none with libvirt installed. And it's not possible to install it as these servers are production.
Comment 4 Martin Edlman 2013-06-19 01:04:28 EDT
(In reply to Laine Stump from comment #2)
> This is the problem (in
> src/network/bridge_driver.c/networkCheckRouteCollision()):
> 
>    enum {MAX_ROUTE_SIZE = 1024*64};
> 
> This size is used for the buffer allocated to hold the entire contents of
> /proc/net/route. The above is a bit misleading, though, because it turns out
> that each line in /proc/net/route is 128 characters long, not 64. So the
> maximum number of routes supported is 512.
> 
> I just posted this simple patch upstream:
> 
> https://www.redhat.com/archives/libvir-list/2013-June/msg00719.html

Please let me know when it's accepted and committed to the repository.
Comment 5 Dave Allan 2013-06-19 09:50:03 EDT
(In reply to Martin Edlman from comment #3)
> The system is up to date, there is no newer libvirt available. I cannot test
> it on RHEL6 as we have none with libvirt installed. And it's not possible to
> install it as these servers are production.

Thanks, I wanted to know if it was already fixed upstream, which Laine answered.
Comment 6 Laine Stump 2013-06-20 14:30:09 EDT
This is fixed upstream in the following commit:

commit 2bdf548f5f0f709c9dec8ba1872e4fcaf8c25fbf
Author: Laine Stump <laine@laine.org>
Date:   Tue Jun 18 12:42:41 2013 -0400

    network: increase max number of routes
    
    This fixes the problem reported in:
    
       https://bugzilla.redhat.com/show_bug.cgi?id=972690
    
    When checking for a collision of a new libvirt network's subnet with
    any existing routes, we read all of /proc/net/route into memory, then
    parse all the entries. The function that we use to read this file
    requires a "maximum length" parameter, which had previously been set
    to 64*1024. As each line in /proc/net/route is 128 bytes, this would
    allow for a maximum of 512 entries in the routing table.
    
    This patch increases that number to 128 * 100000, which allows for
    100,000 routing table entries. This means that it's possible that 12MB
    would be allocated, but that would only happen if there really were
    100,000 route table entries on the system, it's only held for a very
    short time.
    
    Since there is no method of specifying and unlimited max (and that
    would create a potential denial of service anyway) hopefully this
    limit is large enough to accomodate everyone.
Comment 7 Jiri Denemark 2013-12-04 10:39:53 EST
This bugzilla has been reviewed by Red Hat and is not planned
on being addressed in Red Hat Enterprise Linux 5 because it
does not meet the criteria for Production 3 phase
(https://access.redhat.com/site/support/policy/updates/errata/).
Therefore this bugzilla will be closed. If this bug is critical
to production systems, please contact your Red Hat support
representative and provide sufficient business justification.

Note You need to log in before you can comment on or make changes to this bug.