Red Hat Bugzilla – Bug 620973
TCP performance abysmal on RHEL6 with myri10ge
Last modified: 2011-05-24 10:59:18 EDT
Description of problem: On RHEL5, I was testing the performance of the Myricom 10GbE NICs using netperf, with TCP_SENDFILE and an MTU of 9000, and got around 9400 Mbit/s +/- 100 mbit. I went to test on the RHEL 6 beta, as it will be a number of months before the cards in question are deployed, and RHEL 6 will likely be what we actually run these on, only to find that, with identical settings, I received 1.77 Mbit/s. Version-Release number of selected component (if applicable): 2.6.32-44.2.el6.x86_64 I also tried a stock 2.6.35 kernel, built with make rpm, which achieved similar results. How reproducible: 100% Steps to Reproduce: 1. Use myri10ge card on a RHEL 6 machine 2. Run netperf with any TCP-based test Actual results: Performance on the order of 1.77 Mbit/s (occasionally, I will get a result around 300 Mbit/s back). Expected results: Performance on the order of 9400 Mbit/s. Additional info: UDP_STREAM tests on RHEL 54 using MTU 9000 yielded 8700 Mbit/s +/- 100 Mbit. Tests on RHEL 6 yield 7800-9300 Mbit/s. Using rsync, TCP-based NFS mounts, or any other TCP-based things over the 10GbE link results in connection hangs (though almost never explicitly a drop). This is not true either of using TCP or UDP on the 1Gbit interfaces on the machines or using UDP-based protocols over the 10GbE links. Nothing else is being done on the switch at the time these tests are running. The configuration has not changed from when the RHEL 54 tests were run.
This issue has been proposed when we are only considering blocker issues in the current Red Hat Enterprise Linux release. ** If you would still like this issue considered for the current release, ask your support representative to file as a blocker on your behalf. Otherwise ask that it be considered for the next Red Hat Enterprise Linux release. **
Does "ethtool -K ethX lro off" make something better ?
Thank you for your bug report. This issue was evaluated for inclusion in the current release of Red Hat Enterprise Linux. Unfortunately, we are unable to address this request in the current release. Because we are in the final stage of Red Hat Enterprise Linux 6 development, only significant, release-blocking issues involving serious regressions and data corruption can be considered. If you believe this issue meets the release blocking criteria as defined and communicated to you by your Red Hat Support representative, please ask your representative to file this issue as a blocker for the current release. Otherwise, ask that it be evaluated for inclusion in the next minor release of Red Hat Enterprise Linux.
I have these netperf results when using myri10ge on RHEL6. [root@dhcp-31-100 ~]# netperf -H 192.168.1.2 TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.1.2 (192.168.1.2) port 0 AF_INET Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 87380 65536 65536 10.01 9874.51 Could you elaborate more about the problem (i.e. provide your configuration options)?
Since RHEL 6.1 External Beta has begun, and this bug remains unresolved, it has been rejected as it is not proposed as exception or blocker. Red Hat invites you to ask your support representative to propose this request, if appropriate and relevant, in the next release of Red Hat Enterprise Linux.