Bug 731726

Summary: ipv6: Add GSO support on forwarding path
Product: Red Hat Enterprise Linux 6 Reporter: Tuomo Soini <tis>
Component: kernelAssignee: Jiri Benc <jbenc>
Status: CLOSED DUPLICATE QA Contact: Network QE <network-qe>
Severity: high Docs Contact:
Priority: high    
Version: 6.1CC: davem, haliu, herbert.xu, kzhang, simon.deziel, tgraf
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-10-04 10:47:22 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Backport from linux-2.6 git to fix the issue none

Description Tuomo Soini 2011-08-18 13:33:59 UTC
Created attachment 518872 [details]
Backport from linux-2.6 git to fix the issue

Description of problem:

rhel-6.1 kernel performs very badly with ipv6 when forwarding traffic. Difference in speed being 50KB/s with ipv6 compared to 20MB/s with ipv4.

Version-Release number of selected component (if applicable):

2.6.32-131.6.1.el6

How reproducible:

Testing system is:

kvm-vm-a connected to bridge-a
kvm-vm-b connected to bridge-b
kvm-vm-gw connected to both bridge-a and bridge-b as router.

Steps to Reproduce:
1. ssh kvm-vm-a
2. scp -4 kvm-vm-b:bigfile .
3. rm bigfile
4. scp -6 kvm-vm-b:bigfile .
  
Actual results:

Huge difference in speed.

Expected results:

About same speed.

Additional info:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=630730

Attached patch fixes the issue.

Comment 5 Jiri Benc 2011-10-03 15:58:32 UTC
I cannot reproduce the problem. What is your exact setup? In particular:
- What NIC model do you use in each kvm machine?
- What are the offload parameters in each kvm machine (ethtool -k DEV)?
- How is the routing set up in kvm-vm-gw?

Plus, do you know whether patching just the kvm-vm-gw is enough to fix the problem or whether all three machines have to be patched? If you do, it would help, too.

Comment 6 Tuomo Soini 2011-10-03 16:42:39 UTC
03:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5721 Gigabit Ethernet PCI Express (rev 21)

But I'm quite sure ethernet controllers are not relevant.


setup is like this:

kvm-vm-a connected to bridge-a
kvm-vm-b connected to bridge-b
kvm-vm-gw connected to both bridge-a and bridge-b as router.

Note: everything happens insame same host system so there is no ethernet traffic involved.

All kvm hosts configured to run with virtio ethernet drivers, which I think was the important point.


<kvm-vm-a> - bridge-a - <kvm-vm-gw> - bridge-b - <kvm-vm-b>

<kvm-vm-a> is in 2001:1bc8:101:ea05::/64 network and <kvm-vm-gw> has ip 2001:1bc8:101:ea05::1 being the network gateway.

<kvm-vm-b> is in 2001:1bc8:101:ea06::/64 network and <kvm-vm-gw> has ip 2001:1bc8:101:ea06::1 being the network gateway.


Unfortunately I don't remember if patching kvm-vm-gw virtual machine only made a difference and now the system is in production so I can't easily test with unpatched kernel.


All guests running rhel6.1 codebase (and same for host system).

on vm settings are like this:

ethtool -k eth3
Offload parameters for eth3:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: on
udp-fragmentation-offload: on
generic-segmentation-offload: on
generic-receive-offload: off
large-receive-offload: off

on host system:

ethtool -k bond0.9
Offload parameters for bond0.9:
rx-checksumming: off
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: off
large-receive-offload: off

Comment 7 Tuomo Soini 2011-10-03 16:49:35 UTC
Oh. you asked about three systems. You missed that there were actually four systems when host system is counted too!

Comment 8 Jiri Benc 2011-10-03 19:19:53 UTC
The host should not make a difference, as it is just bridging, i.e. it operates on a layer below IPv4/IPv6. Still, good point, reality doesn't always match one's expectations, definitely something to keep in mind.

Comment 9 Tuomo Soini 2011-10-04 08:13:13 UTC
After digging our logs I'm quite sure adding patch to router vm kernel fixed the issue.

Not two vms connected to same bridge were originally not affedted by the problem. Only forwarded (routed) traffic has the performance problem.

Comment 10 Jiri Benc 2011-10-04 10:47:22 UTC
Thanks! Turns out this is already fixed in kernel-2.6.32-153.el6.

*** This bug has been marked as a duplicate of bug 683455 ***