Bug 697310

Created attachment 492964 [details]
Patch to xen-network-common.sh to enable large MTU on vif's

The first patch only worked on domUs, not dom0 itself. This new patch works on both.

Please note; I've not tested with VLANs or bonded links. I don't feel qualified to do those tests, so if someone could it would be appreciated.

Further testing shows that live-migration still causes the bridge(s) to drop to 1500. Not sure on where this needs to be fixed, but it looks like it's the script that creates the tapX interface(s). Anyone able to lend a hand on this?

tap devices are created directly by qemu itself. They are add to bridge using /etc/xen/qemu-ifup script.

Miroslav, thank you! That was exactly the hint I needed. I'll attach a patch next which fixes live migration with large MTUs. If someone could apply these two patches to their respective RPMs, this should be ready for proper testing.

Created attachment 494328 [details]
Patch for /etc/xen/qemu-ifup to enable live migration woth large MTUs

This patch checks the MTU size of the corresponding 'ethX' device and sets the new 'tapX' device's MTU to match just prior to adding it to the bridge. This allowed me to successfully live-migrate a xen domU using an MTU of 7200.

I agree that patching qemu-ifup is a good idea, but anyway another workaround is to use type=netfront, instead of type=ioemu.  If you care about network performance you're most likely using paravirtualized network interfaces, so the tapX devices are useless for you.

I do use paravirt, but I found that the tapX device was used for the live migration. I'm not sure where to define the 'type=x', but I'll look for it and test it out. 

As an aside, are these scripts copied from the examples directory by the RPM on install? If so, I can try making a proper patch and get a patched script for the xen RPM instead to get that much more done for the next release.

To use type=netfront, add it to end of vif definition,e.g.:

vif = [ "mac=00:16:36:01:d8:8a,bridge=xenbr0,script=vif-bridge,type=netfront" ]

Most of the script are stored in tools/example directory but some of them can be elsewhere, e.g. qemu-ifup is in tools/ioemu/target-i386-dm directory

Created attachment 496016 [details]
Updated userland patch which works with multiple, out of sync domU ethX to dom0 xenbrY devices

I found a bug where if the user was starting a domU connected to out-of-order
bridges (ie: xenbr0 and xenbr2 which mapped to eth0 and eth1 in domU), the
patched userland tools would try to get the MTU of the 'xenbr' with the
sequence number matching the interface in the domU.

With the help of Paolo and Pasi, I was able to solve this by no longer having
xen setup the bridges (which is not recommended anyway) and altering the
xen-network-common.sh and xen-network-common-bonding.sh scripts to set the MTU
of the vif prior to connecting to the bridge in their add_to_bridge()
functions.

Note that this was tested with xend-config.sxp set to '(network-script
/bin/true)' and the bridge built in
/etc/sysconfig/network-scripts/ifcfg-xenbrX.

Comment on attachment 494328 [details]
Patch for /etc/xen/qemu-ifup to enable live migration woth large MTUs

This patch is now in the remaining active patch.

Created attachment 503673 [details]
updated patch

digimer,

can you please test again with this updated patch?  It picks the tap interface's MTU from the bridge, which is similar to what is done for the vif interfaces and less hackish.

Created attachment 503675 [details]
patch for the SRPM

the previous attachment applies on top of the root filesystem, this one uses the correct paths in the SRPM.

I saved your patch as 'SOURCES/xen-0001-bug-697310.patch' and patched the xen.spec file thusly:

=============
--- xen.spec.orig	2011-06-13 17:14:10.000000000 -0400
+++ xen.spec	2011-06-13 17:33:46.000000000 -0400
@@ -3,7 +3,7 @@
 Summary: Xen is a virtual machine monitor
 Name:    xen
 Version: 3.0.3
-Release: 120%{?dist}.2
+Release: 120%{?dist}.3
 Group:   Development/Libraries
 License: GPLv2+, LGPLv2+
 URL:     http://www.cl.cam.ac.uk/Research/SRG/netos/xen/index.html
@@ -672,6 +672,9 @@
 # Fix integer and buffer overflows in xc_dom_probe_bzimage_kernel() (rhbz 696938)
 Patch1049: xen-fix-overflows-in-xc_dom_probe_bzimage_kernel.patch
 
+# Enables MTUs > 1500 on vifs without causing bridges with MTUs >1500 problems (rhbz 697310)
+Patch1100: xen-0001-bug-697310.patch
+
 BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root
 BuildRequires: transfig libidn-devel zlib-devel texi2html SDL-devel curl-devel
 BuildRequires: libX11-devel python-devel ghostscript tetex-latex 
@@ -1507,6 +1510,9 @@
 # Fix integer and buffer overflows in xc_dom_probe_bzimage_kernel() (rhbz 696938)
 %patch1049 -p1
 
+# Enables MTUs > 1500 on vifs without causing bridges with MTUs >1500 problems (rhbz 697310) 
+%patch1100 -p1
+
 %build
 # libvncserver first
 cat > tools/check/check_libvncserver <<EOF
=============

When I tried to compile, I got this error:

=============
+ echo 'Patch #1100 (xen-0001-bug-697310.patch):'
Patch #1100 (xen-0001-bug-697310.patch):
+ patch -p1 -s
1 out of 1 hunk FAILED -- saving rejects to file tools/ioemu/target-i386-dm/qemu-ifup.rej
error: Bad exit status from /var/tmp/rpm-tmp.56624 (%prep)
=============

The contents of 'tools/ioemu/target-i386-dm/qemu-ifup.rej' are:

=============
***************
*** 10,14 ****
     $(($RANDOM % 256)) $(($RANDOM % 256))`

  ifconfig $1 hw ether ${mac}
  ifconfig $1 0.0.0.0 up
  brctl addif $2 $1
--- 10,20 ----
     $(($RANDOM % 256)) $(($RANDOM % 256))`

  ifconfig $1 hw ether ${mac}
+ 
+ # Set the MTU of the interface to match the bridge
+ if [ -e "/sys/class/net/$2/mtu" ]; then
+     ifconfig $1 mtu `cat /sys/class/net/$2/mtu`
+ fi
+ 
  ifconfig $1 0.0.0.0 up
  brctl addif $2 $1
=============

I'm guessing an earlier patch changed things too much for this to apply. I'll try to fix it and will post an update shortly.

It looks like the latest xen source RPM (xen-3.0.3-120.el5_6.2.src.rpm) has other problems. I get this trying to build it on it's own:

=============
RPM build errors:
    Bad file: /usr/src/redhat/SOURCES/eb-rtl8139-new.zrom: No such file or directory
=============

(In reply to comment #14)
> It looks like the latest xen source RPM (xen-3.0.3-120.el5_6.2.src.rpm) has
> other problems. I get this trying to build it on it's own:
> 
> =============
> RPM build errors:
>     Bad file: /usr/src/redhat/SOURCES/eb-rtl8139-new.zrom: No such file or
> directory
> =============

Where do you get your srpm from? Do you just rpm -i it? I checked it and zrom file you're missing is in package and should be in SOURCE directory.

I was building on CentOS 5.6. I opened a bug report with them for this RPM as it looks like the issue is with a file being clobbered. I've got RHEL 5.6 though, so if you'd prefer, I can run up a couple of nodes using it to test this patch.

digimer,

just trust us on this part of the work and test with comment 11. :)

Thanks for everything you've done so far on this feature!

Paolo,

  Sorry, I let this slip. I need to reinstall two test nodes using RHEL 5.6 proper before I test this. I will get this done ASAP and report back (with luck, I won't delay too long again... :) ).

I tried the patch in comment 12, but I the patch as-is fails, unable to find the files (applied as patch 1100). I am using '-p 1' to match all the other patches in the xen.spec, and prefix'ed 'a/' or 'xen-3.1.0-src/' to the path names at the top.

====
Patch #1100 (xen_vif_large_mtu.patch):
+ patch -p1 -s
missing header for unified diff at line 1 of patch
File to patch: 
====

Not sure what's going wrong, but I have the sneaking suspicion that it's something truly simple and obvious. :)

Patch as applied:

====
@@ -, +, @@ 
 xen-3.1.0-src/tools/examples/xen-network-common-bonding.sh |    4 ++++
 xen-3.1.0-src/tools/examples/xen-network-common.sh         |    4 ++++
 xen-3.1.0-src/tools/ioemu/target-i386-dm/qemu-ifup         |    6 ++++++
 3 files changed, 14 insertions(+), 0 deletions(-)
--- xen-3.1.0-src/tools/examples/xen-network-common-bonding.sh	
+++ xen-3.1.0-src/tools/examples/xen-network-common-bonding.sh	
@@ -137,6 +137,10 @@ add_to_bridge () {
 	ip link set ${dev} up || true
 	return
     fi
+    # Set the MTU of the vif to match the bridge.
+    if [ -e "/sys/class/net/${bridge}/mtu" ]; then
+        ip link set ${dev} mtu `cat /sys/class/net/${bridge}/mtu`
+    fi
     brctl addif ${bridge} ${dev}
     ip link set ${dev} up
 }
--- xen-3.1.0-src/tools/examples/xen-network-common.sh	
+++ xen-3.1.0-src/tools/examples/xen-network-common.sh	
@@ -139,6 +139,10 @@ add_to_bridge () {
 	ip link set ${dev} up || true
 	return
     fi
+    # Set the MTU of the vif to match the bridge.
+    if [ -e "/sys/class/net/${bridge}/mtu" ]; then
+        ip link set ${dev} mtu `cat /sys/class/net/${bridge}/mtu`
+    fi
     brctl addif ${bridge} ${dev}
     ip link set ${dev} up
 }
--- xen-3.1.0-src/tools/ioemu/target-i386-dm/qemu-ifup	
+++ xen-3.1.0-src/tools/ioemu/target-i386-dm/qemu-ifup	
@@ -10,5 +10,11 @@ mac=`printf "fe:%02x:%02x:%02x:%02x:%02x" \
    $(($RANDOM % 256)) $(($RANDOM % 256))`
====
 
 ifconfig $1 hw ether ${mac}
+
+# Set the MTU of the interface to match the bridge
+if [ -e "/sys/class/net/$2/mtu" ]; then
+    ifconfig $1 mtu `cat /sys/class/net/$2/mtu`
+fi
+
 ifconfig $1 0.0.0.0 up
 brctl addif $2 $1
====

Use https://bugzilla.redhat.com/attachment.cgi?id=503675 instead of the "Raw Unified" link.

Fix built into xen-3.0.3-134.el5

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2012-0160.html