Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
For bugs related to Red Hat Enterprise Linux 5 product line. The current stable release is 5.10. For Red Hat Enterprise Linux 6 and above, please visit Red Hat JIRA https://issues.redhat.com/secure/CreateIssue!default.jspa?pid=12332745 to report new issues.

Bug 972690

Summary: Virtual network doesn't start when there is a big routing table
Product: Red Hat Enterprise Linux 5 Reporter: Martin Edlman <martin.edlman>
Component: libvirtAssignee: Laine Stump <laine>
Status: CLOSED WONTFIX QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 5.5CC: bsarathy, dallan, dyuan, gsun, honzhang, martin.edlman
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-12-04 15:39:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1049888    

Description Martin Edlman 2013-06-10 12:13:18 UTC
Description of problem:
Virtual network doesn't start when there is a big routing table. We have a server running Zebra routing daemon and there are 670 routes in the routing table. Libvirt complains

error: Failed to start network Servers
error: Failed to read file '/proc/net/route': Value too large for defined data type

Another virtual network LTSP is running, it was started before the routing table grew up.

Version-Release number of selected component (if applicable):
libvirt-0.8.2

How reproducible:
Start Zebra (and its companions) and then start libvirt. This is the normal start up order in the /etc/init.d

Steps to Reproduce:
1.
2.
3.

Actual results:
Virtual network Servers is not up and not running

Expected results:
Virtual network Servers is up and running

Additional info:
When the Zebra is stopped and RT has only few items, virtual nets start up as expected.

Comment 1 Dave Allan 2013-06-18 14:50:20 UTC
Martin, are you able to determine if this behavior exists with more recent libvirt, either that from RHEL6 or the upstream packages?

Comment 2 Laine Stump 2013-06-18 17:00:35 UTC
This is the problem (in src/network/bridge_driver.c/networkCheckRouteCollision()):

   enum {MAX_ROUTE_SIZE = 1024*64};

This size is used for the buffer allocated to hold the entire contents of /proc/net/route. The above is a bit misleading, though, because it turns out that each line in /proc/net/route is 128 characters long, not 64. So the maximum number of routes supported is 512.

I just posted this simple patch upstream:

https://www.redhat.com/archives/libvir-list/2013-June/msg00719.html

Comment 3 Martin Edlman 2013-06-19 05:02:15 UTC
(In reply to Dave Allan from comment #1)
> Martin, are you able to determine if this behavior exists with more recent
> libvirt, either that from RHEL6 or the upstream packages?

The system is up to date, there is no newer libvirt available. I cannot test it on RHEL6 as we have none with libvirt installed. And it's not possible to install it as these servers are production.

Comment 4 Martin Edlman 2013-06-19 05:04:28 UTC
(In reply to Laine Stump from comment #2)
> This is the problem (in
> src/network/bridge_driver.c/networkCheckRouteCollision()):
> 
>    enum {MAX_ROUTE_SIZE = 1024*64};
> 
> This size is used for the buffer allocated to hold the entire contents of
> /proc/net/route. The above is a bit misleading, though, because it turns out
> that each line in /proc/net/route is 128 characters long, not 64. So the
> maximum number of routes supported is 512.
> 
> I just posted this simple patch upstream:
> 
> https://www.redhat.com/archives/libvir-list/2013-June/msg00719.html

Please let me know when it's accepted and committed to the repository.

Comment 5 Dave Allan 2013-06-19 13:50:03 UTC
(In reply to Martin Edlman from comment #3)
> The system is up to date, there is no newer libvirt available. I cannot test
> it on RHEL6 as we have none with libvirt installed. And it's not possible to
> install it as these servers are production.

Thanks, I wanted to know if it was already fixed upstream, which Laine answered.

Comment 6 Laine Stump 2013-06-20 18:30:09 UTC
This is fixed upstream in the following commit:

commit 2bdf548f5f0f709c9dec8ba1872e4fcaf8c25fbf
Author: Laine Stump <laine>
Date:   Tue Jun 18 12:42:41 2013 -0400

    network: increase max number of routes
    
    This fixes the problem reported in:
    
       https://bugzilla.redhat.com/show_bug.cgi?id=972690
    
    When checking for a collision of a new libvirt network's subnet with
    any existing routes, we read all of /proc/net/route into memory, then
    parse all the entries. The function that we use to read this file
    requires a "maximum length" parameter, which had previously been set
    to 64*1024. As each line in /proc/net/route is 128 bytes, this would
    allow for a maximum of 512 entries in the routing table.
    
    This patch increases that number to 128 * 100000, which allows for
    100,000 routing table entries. This means that it's possible that 12MB
    would be allocated, but that would only happen if there really were
    100,000 route table entries on the system, it's only held for a very
    short time.
    
    Since there is no method of specifying and unlimited max (and that
    would create a potential denial of service anyway) hopefully this
    limit is large enough to accomodate everyone.

Comment 7 Jiri Denemark 2013-12-04 15:39:53 UTC
This bugzilla has been reviewed by Red Hat and is not planned
on being addressed in Red Hat Enterprise Linux 5 because it
does not meet the criteria for Production 3 phase
(https://access.redhat.com/site/support/policy/updates/errata/).
Therefore this bugzilla will be closed. If this bug is critical
to production systems, please contact your Red Hat support
representative and provide sufficient business justification.