Bug 697310 - vif (netback) should take its default MTU from the bridge
vif (netback) should take its default MTU from the bridge
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: xen (Show other bugs)
5.6
x86_64 Linux
medium Severity high
: rc
: ---
Assigned To: Paolo Bonzini
Virtualization Bugs
: ZStream
Depends On: 697021
Blocks: 699611 733417 738608
  Show dependency treegraph
 
Reported: 2011-04-17 12:48 EDT by digimer
Modified: 2013-01-10 22:57 EST (History)
10 users (show)

See Also:
Fixed In Version: xen-3.0.3-134.el5
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 733417 (view as bug list)
Environment:
Last Closed: 2012-02-21 00:54:47 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
patch for vif-bridge (417 bytes, patch)
2011-04-17 12:48 EDT, digimer
no flags Details | Diff
Patch to xen-network-common.sh to enable large MTU on vif's (624 bytes, patch)
2011-04-18 13:17 EDT, digimer
no flags Details | Diff
Patch for /etc/xen/qemu-ifup to enable live migration woth large MTUs (567 bytes, patch)
2011-04-22 19:10 EDT, digimer
no flags Details | Diff
Updated userland patch which works with multiple, out of sync domU ethX to dom0 xenbrY devices (1.50 KB, patch)
2011-04-30 21:42 EDT, digimer
no flags Details | Diff
updated patch (1.29 KB, patch)
2011-06-08 08:30 EDT, Paolo Bonzini
no flags Details | Diff
patch for the SRPM (2.03 KB, patch)
2011-06-08 08:37 EDT, Paolo Bonzini
no flags Details | Diff

  None (edit)
Description digimer 2011-04-17 12:48:15 EDT
Created attachment 492725 [details]
patch for vif-bridge

Description of problem:

If/when the patch for bz #697021 is accepted, this patch will add support in the /etc/xen/scripts/vif-bridge for the arbitrary MTU size of vif's prior to their connection to the bridge.

Version-Release number of selected component (if applicable):

xen-3.0.3-120.el5_6.1

How reproducible:

n/a

Steps to Reproduce:
1. n/a
2.
3.
  
Actual results:

bz 697021, without this patch, will continue to use an MTU of 1500 for new vif's.

Expected results:

Adjusts vif's MTU to match that of the target bridge.

Additional info:

I was unable to sort out how to apply the patch against the xen source RPM. The attached patch's paths will need to be adjusted by the xen RPM maintainer(s).
Comment 1 digimer 2011-04-18 13:17:35 EDT
Created attachment 492964 [details]
Patch to xen-network-common.sh to enable large MTU on vif's

The first patch only worked on domUs, not dom0 itself. This new patch works on both.

Please note; I've not tested with VLANs or bonded links. I don't feel qualified to do those tests, so if someone could it would be appreciated.
Comment 2 digimer 2011-04-21 09:19:38 EDT
Further testing shows that live-migration still causes the bridge(s) to drop to 1500. Not sure on where this needs to be fixed, but it looks like it's the script that creates the tapX interface(s). Anyone able to lend a hand on this?
Comment 3 Miroslav Rezanina 2011-04-22 06:52:14 EDT
tap devices are created directly by qemu itself. They are add to bridge using /etc/xen/qemu-ifup script.
Comment 4 digimer 2011-04-22 19:07:40 EDT
Miroslav, thank you! That was exactly the hint I needed. I'll attach a patch next which fixes live migration with large MTUs. If someone could apply these two patches to their respective RPMs, this should be ready for proper testing.
Comment 5 digimer 2011-04-22 19:10:04 EDT
Created attachment 494328 [details]
Patch for /etc/xen/qemu-ifup to enable live migration woth large MTUs

This patch checks the MTU size of the corresponding 'ethX' device and sets the new 'tapX' device's MTU to match just prior to adding it to the bridge. This allowed me to successfully live-migrate a xen domU using an MTU of 7200.
Comment 6 Paolo Bonzini 2011-04-26 03:24:00 EDT
I agree that patching qemu-ifup is a good idea, but anyway another workaround is to use type=netfront, instead of type=ioemu.  If you care about network performance you're most likely using paravirtualized network interfaces, so the tapX devices are useless for you.
Comment 7 digimer 2011-04-26 09:20:54 EDT
I do use paravirt, but I found that the tapX device was used for the live migration. I'm not sure where to define the 'type=x', but I'll look for it and test it out. 

As an aside, are these scripts copied from the examples directory by the RPM on install? If so, I can try making a proper patch and get a patched script for the xen RPM instead to get that much more done for the next release.
Comment 8 Miroslav Rezanina 2011-04-27 07:58:34 EDT
To use type=netfront, add it to end of vif definition,e.g.:

vif = [ "mac=00:16:36:01:d8:8a,bridge=xenbr0,script=vif-bridge,type=netfront" ]

Most of the script are stored in tools/example directory but some of them can be elsewhere, e.g. qemu-ifup is in tools/ioemu/target-i386-dm directory
Comment 9 digimer 2011-04-30 21:42:43 EDT
Created attachment 496016 [details]
Updated userland patch which works with multiple, out of sync domU ethX to dom0 xenbrY devices

I found a bug where if the user was starting a domU connected to out-of-order
bridges (ie: xenbr0 and xenbr2 which mapped to eth0 and eth1 in domU), the
patched userland tools would try to get the MTU of the 'xenbr' with the
sequence number matching the interface in the domU.

With the help of Paolo and Pasi, I was able to solve this by no longer having
xen setup the bridges (which is not recommended anyway) and altering the
xen-network-common.sh and xen-network-common-bonding.sh scripts to set the MTU
of the vif prior to connecting to the bridge in their add_to_bridge()
functions.

Note that this was tested with xend-config.sxp set to '(network-script
/bin/true)' and the bridge built in
/etc/sysconfig/network-scripts/ifcfg-xenbrX.
Comment 10 digimer 2011-05-02 13:48:45 EDT
Comment on attachment 494328 [details]
Patch for /etc/xen/qemu-ifup to enable live migration woth large MTUs

This patch is now in the remaining active patch.
Comment 11 Paolo Bonzini 2011-06-08 08:30:22 EDT
Created attachment 503673 [details]
updated patch

digimer,

can you please test again with this updated patch?  It picks the tap interface's MTU from the bridge, which is similar to what is done for the vif interfaces and less hackish.
Comment 12 Paolo Bonzini 2011-06-08 08:37:12 EDT
Created attachment 503675 [details]
patch for the SRPM

the previous attachment applies on top of the root filesystem, this one uses the correct paths in the SRPM.
Comment 13 digimer 2011-06-13 17:41:57 EDT
I saved your patch as 'SOURCES/xen-0001-bug-697310.patch' and patched the xen.spec file thusly:

=============
--- xen.spec.orig	2011-06-13 17:14:10.000000000 -0400
+++ xen.spec	2011-06-13 17:33:46.000000000 -0400
@@ -3,7 +3,7 @@
 Summary: Xen is a virtual machine monitor
 Name:    xen
 Version: 3.0.3
-Release: 120%{?dist}.2
+Release: 120%{?dist}.3
 Group:   Development/Libraries
 License: GPLv2+, LGPLv2+
 URL:     http://www.cl.cam.ac.uk/Research/SRG/netos/xen/index.html
@@ -672,6 +672,9 @@
 # Fix integer and buffer overflows in xc_dom_probe_bzimage_kernel() (rhbz 696938)
 Patch1049: xen-fix-overflows-in-xc_dom_probe_bzimage_kernel.patch
 
+# Enables MTUs > 1500 on vifs without causing bridges with MTUs >1500 problems (rhbz 697310)
+Patch1100: xen-0001-bug-697310.patch
+
 BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root
 BuildRequires: transfig libidn-devel zlib-devel texi2html SDL-devel curl-devel
 BuildRequires: libX11-devel python-devel ghostscript tetex-latex 
@@ -1507,6 +1510,9 @@
 # Fix integer and buffer overflows in xc_dom_probe_bzimage_kernel() (rhbz 696938)
 %patch1049 -p1
 
+# Enables MTUs > 1500 on vifs without causing bridges with MTUs >1500 problems (rhbz 697310) 
+%patch1100 -p1
+
 %build
 # libvncserver first
 cat > tools/check/check_libvncserver <<EOF
=============

When I tried to compile, I got this error:

=============
+ echo 'Patch #1100 (xen-0001-bug-697310.patch):'
Patch #1100 (xen-0001-bug-697310.patch):
+ patch -p1 -s
1 out of 1 hunk FAILED -- saving rejects to file tools/ioemu/target-i386-dm/qemu-ifup.rej
error: Bad exit status from /var/tmp/rpm-tmp.56624 (%prep)
=============

The contents of 'tools/ioemu/target-i386-dm/qemu-ifup.rej' are:

=============
***************
*** 10,14 ****
     $(($RANDOM % 256)) $(($RANDOM % 256))`

  ifconfig $1 hw ether ${mac}
  ifconfig $1 0.0.0.0 up
  brctl addif $2 $1
--- 10,20 ----
     $(($RANDOM % 256)) $(($RANDOM % 256))`

  ifconfig $1 hw ether ${mac}
+ 
+ # Set the MTU of the interface to match the bridge
+ if [ -e "/sys/class/net/$2/mtu" ]; then
+     ifconfig $1 mtu `cat /sys/class/net/$2/mtu`
+ fi
+ 
  ifconfig $1 0.0.0.0 up
  brctl addif $2 $1
=============

I'm guessing an earlier patch changed things too much for this to apply. I'll try to fix it and will post an update shortly.
Comment 14 digimer 2011-06-13 17:50:29 EDT
It looks like the latest xen source RPM (xen-3.0.3-120.el5_6.2.src.rpm) has other problems. I get this trying to build it on it's own:

=============
RPM build errors:
    Bad file: /usr/src/redhat/SOURCES/eb-rtl8139-new.zrom: No such file or directory
=============
Comment 15 Miroslav Rezanina 2011-06-14 01:50:47 EDT
(In reply to comment #14)
> It looks like the latest xen source RPM (xen-3.0.3-120.el5_6.2.src.rpm) has
> other problems. I get this trying to build it on it's own:
> 
> =============
> RPM build errors:
>     Bad file: /usr/src/redhat/SOURCES/eb-rtl8139-new.zrom: No such file or
> directory
> =============

Where do you get your srpm from? Do you just rpm -i it? I checked it and zrom file you're missing is in package and should be in SOURCE directory.
Comment 17 digimer 2011-06-16 11:39:58 EDT
I was building on CentOS 5.6. I opened a bug report with them for this RPM as it looks like the issue is with a file being clobbered. I've got RHEL 5.6 though, so if you'd prefer, I can run up a couple of nodes using it to test this patch.
Comment 18 Paolo Bonzini 2011-06-27 06:42:13 EDT
digimer,

just trust us on this part of the work and test with comment 11. :)

Thanks for everything you've done so far on this feature!
Comment 19 digimer 2011-06-27 11:10:20 EDT
Paolo,

  Sorry, I let this slip. I need to reinstall two test nodes using RHEL 5.6 proper before I test this. I will get this done ASAP and report back (with luck, I won't delay too long again... :) ).
Comment 20 digimer 2011-07-09 19:36:38 EDT
I tried the patch in comment 12, but I the patch as-is fails, unable to find the files (applied as patch 1100). I am using '-p 1' to match all the other patches in the xen.spec, and prefix'ed 'a/' or 'xen-3.1.0-src/' to the path names at the top.

====
Patch #1100 (xen_vif_large_mtu.patch):
+ patch -p1 -s
missing header for unified diff at line 1 of patch
File to patch: 
====

Not sure what's going wrong, but I have the sneaking suspicion that it's something truly simple and obvious. :)

Patch as applied:

====
@@ -, +, @@ 
 xen-3.1.0-src/tools/examples/xen-network-common-bonding.sh |    4 ++++
 xen-3.1.0-src/tools/examples/xen-network-common.sh         |    4 ++++
 xen-3.1.0-src/tools/ioemu/target-i386-dm/qemu-ifup         |    6 ++++++
 3 files changed, 14 insertions(+), 0 deletions(-)
--- xen-3.1.0-src/tools/examples/xen-network-common-bonding.sh	
+++ xen-3.1.0-src/tools/examples/xen-network-common-bonding.sh	
@@ -137,6 +137,10 @@ add_to_bridge () {
 	ip link set ${dev} up || true
 	return
     fi
+    # Set the MTU of the vif to match the bridge.
+    if [ -e "/sys/class/net/${bridge}/mtu" ]; then
+        ip link set ${dev} mtu `cat /sys/class/net/${bridge}/mtu`
+    fi
     brctl addif ${bridge} ${dev}
     ip link set ${dev} up
 }
--- xen-3.1.0-src/tools/examples/xen-network-common.sh	
+++ xen-3.1.0-src/tools/examples/xen-network-common.sh	
@@ -139,6 +139,10 @@ add_to_bridge () {
 	ip link set ${dev} up || true
 	return
     fi
+    # Set the MTU of the vif to match the bridge.
+    if [ -e "/sys/class/net/${bridge}/mtu" ]; then
+        ip link set ${dev} mtu `cat /sys/class/net/${bridge}/mtu`
+    fi
     brctl addif ${bridge} ${dev}
     ip link set ${dev} up
 }
--- xen-3.1.0-src/tools/ioemu/target-i386-dm/qemu-ifup	
+++ xen-3.1.0-src/tools/ioemu/target-i386-dm/qemu-ifup	
@@ -10,5 +10,11 @@ mac=`printf "fe:%02x:%02x:%02x:%02x:%02x" \
    $(($RANDOM % 256)) $(($RANDOM % 256))`
====
 
 ifconfig $1 hw ether ${mac}
+
+# Set the MTU of the interface to match the bridge
+if [ -e "/sys/class/net/$2/mtu" ]; then
+    ifconfig $1 mtu `cat /sys/class/net/$2/mtu`
+fi
+
 ifconfig $1 0.0.0.0 up
 brctl addif $2 $1
====
Comment 21 Paolo Bonzini 2011-07-13 08:38:11 EDT
Use https://bugzilla.redhat.com/attachment.cgi?id=503675 instead of the "Raw Unified" link.
Comment 27 Miroslav Rezanina 2011-09-07 07:44:32 EDT
Fix built into xen-3.0.3-134.el5
Comment 32 errata-xmlrpc 2012-02-21 00:54:47 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2012-0160.html

Note You need to log in before you can comment on or make changes to this bug.