Description of problem: bare machine: HP DL 380 G5 with two Broadcom NIC, updated to the last firmware revision as found on the HP support web site virtual machine: two RHEL 5.0 with hardware assisted virtualization on the first domU, there's a netperf server listening on the second domU, it's started the tcp_stream_script Results: bandwidth of 2.5 Mbps, and the domU with the netperf server is reporting dropped packets and errors on eth0. Dom0 reports no errors. Things tried to do with no success: - ifconfig eth0 mtu 1400 on both domU - net.ipv4.tcp_{r,w}mem = 4096 1398080 8388608 - net.core.{w,r}mem_max = 8388608 both on domO - txqueuelen set to 3000 for virbr0,xenbr0,vif,etc Note that "normal" networking works, the problem arises only with tcp-intensive workload (the original problem was triggered by MPI benchmark showing poor results. i.e. 2 MFlops sustained for 2 virtual machines, whilst 1 virtual machine shows about 800 MFlops) kernel on virtual machines: 2.6.18-8.1.3.el5 kernel on physical machine: 2.6.18-53.1.4.el5xen on xen ml this problem has hit some peoples (everyone using broadcom NICs, but as they are widely adopted could be unrelated to the real problem), with no working solutions (except: using XenExpress). It seems it could be related to missing PV drivers. On both virtual machines it's aliasing eth0 with 8139cp
-Updated the dom0 with kernel 2.6.18-53.1.13.el5xen - the firmware of the Broadcom BCM5708 is 1.9.6 - disabled the management function as described in http://kb.xensource.com/entry.jspa?externalID=66 now performances are about 45 Mbps from domU to domU in the same domO, the domU with netserver has lots of dropped packets and errors, other domU and domO have no errors.
After using the paravirtualized drivers, as described here: http://www.redhat.com/docs/manuals/enterprise/RHEL-5-manual/en-US/RHEL510/pdf/Para-Virtualized_Drivers.pdf netperf shows a 347 MBps (2912*10^6 bits per second) sustained throughput for a domU->domU. please close.
Closing ticket, at request of reporter.