Bug 654457
| Summary: | Traffic control (tc) with 10Gig ethernet drops packets when txqueuelen set too low | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 5 | Reporter: | Matthew Whitehead <mwhitehe> | ||||||
| Component: | initscripts | Assignee: | initscripts Maintenance Team <initscripts-maint-list> | ||||||
| Status: | CLOSED WONTFIX | QA Contact: | qe-baseos-daemons | ||||||
| Severity: | medium | Docs Contact: | |||||||
| Priority: | low | ||||||||
| Version: | 5.5 | CC: | notting | ||||||
| Target Milestone: | rc | ||||||||
| Target Release: | --- | ||||||||
| Hardware: | x86_64 | ||||||||
| OS: | Linux | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2010-11-18 02:28:01 UTC | Type: | --- | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
Created attachment 461171 [details]
Patch
Added options for interface (in ifcfg-ethX) configuration:
IFCONFIG_OPTS: ability to modify interface options via "ifconfig -s" (including "txqueuelen").
ETHTOOL_C_OPTS: ability to modify ethernet options via "ethtool -C".
While ETHTOOL_OPTS remains as a legacy supported config mechanism, new config really should be done via interface-specific udev rules (that therefore will get automatically applied as soon as the interface appears.) |
Created attachment 461169 [details] reproducer code, invoked as "send [ip] [port] 1024 [num_cpus]" Description of problem: Traffic control (tc) with 10Gig ethernet drops packets when txqueuelen set too low. /etc/sysconfig/network-scripts/ifup-eth does not provide a parameter to set this elegantly. Version-Release number of selected component (if applicable): initscripts-8.45.30-2.el5 How reproducible: 100% Steps to Reproduce: 1. Increase write buffers on sending machine with 'sysctl -w et.core.wmem_default=4194304 2. Check current traffic control settings with 'tc -s qdisc ls' and note dropped packets. 3. Compile attachment send.c as 'gcc -o send -m64 -lpthread send.c' 4. On second machine with 10G ethernet, start a listener as 'nc -d -l 5000 < /dev/null' 5. Start application with './send [ip] 5000 1024 [num_cpus]". In our test, num_cpus=7. 6. Re-run 'tc -s qdisc ls' and note that 'dropped' increases significantly. Actual results: [ON RECEIVER] [root@mrgdenprdn1 /root]# /usr/bin/nc -d -l 5000 < /dev/null [ON TRANSMITTER] [root@mrgdenprdn2 ~]# ifconfig eth0 eth0 Link encap:Ethernet HWaddr 00:23:7D:5D:E6:E0 inet addr:10.1.23.65 Bcast:10.1.23.255 Mask:255.255.255.0 inet6 addr: fe80::223:7dff:fe5d:e6e0/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:330640 errors:0 dropped:0 overruns:0 frame:0 TX packets:1408100059 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:97168394 (92.6 MiB) TX bytes:1506455772470 (1.3 TiB) Interrupt:169 Memory:fd800000-fdffffff [root@mrgdenprdn2 ~]# /sbin/tc -s qdisc ls qdisc pfifo_fast 0: dev eth0 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1 Sent 1500822564765 bytes 1408090807 pkt (dropped 0, overlimits 0 requeues 571) rate 0bit 0pps backlog 0b 0p requeues 571 [root@mrgdenprdn2 ~] send 10.0.0.1 5000 1024 7 ... lots of output [root@mrgdenprdn2 /root]# /sbin/tc -s qdisc ls qdisc pfifo_fast 0: dev eth0 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1 Sent 1502438590874 bytes 1409606952 pkt (dropped 4222154, overlimits 0 requeues 572) rate 0bit 0pps backlog 0b 0p requeues 572 Expected results: Transmission control should not be dropping packets, but it is because txqueuelen is too low. Additional info: Proposed patch to /etc/sysconfig/network-scripts/ifup-eth.