Bug 846468 - BW_TCP utility yields inconsistent results.
BW_TCP utility yields inconsistent results.
Status: CLOSED ERRATA
Product: Red Hat Hardware Certification Program
Classification: Red Hat
Component: Test Suite (tests) (Show other bugs)
6.1
Unspecified Unspecified
unspecified Severity urgent
: ---
: ---
Assigned To: Greg Nichols
Red Hat Kernel QE team
:
Depends On:
Blocks: 918223 948963
  Show dependency treegraph
 
Reported: 2012-08-07 17:08 EDT by Frank Kuhar
Modified: 2013-08-06 13:44 EDT (History)
5 users (show)

See Also:
Fixed In Version: hwcert-client-1.6.4-R44
Doc Type: Bug Fix
Doc Text:
Cause: Consequence: NETWORK test is yielding inconsistent speed results across multiple network adapters of the same type, and from run to run on the same adapter. Fix: new NETWORK test fixes calls to bw_tcp by setting paralellism to 4 (-P 4), and correctly setting the message size to 1MB (-m 1m). Result: The test results is quite a bit more consistent now.
Story Points: ---
Clone Of:
: 918223 948963 (view as bug list)
Environment:
Last Closed: 2013-08-06 13:44:32 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Results from 8,4,2 thread runs of bw_tcp (1.41 KB, text/plain)
2012-08-07 17:08 EDT, Frank Kuhar
no flags Details
network patch to fix use of bw_tcp (1.90 KB, patch)
2013-04-05 10:58 EDT, Greg Nichols
no flags Details | Diff

  None (edit)
Description Frank Kuhar 2012-08-07 17:08:43 EDT
Created attachment 602858 [details]
Results from 8,4,2 thread runs of bw_tcp

Description of problem:
Network portion of certification test is yielding inconsistent speed results across multiple network adapters of the same type, and from run to run on the same adapter.

Version-Release number of selected component (if applicable):
various versions of cert suite

How reproducible:
Very

Steps to Reproduce:
1.  Run the network portion multiple times on the same 10GB interface and note the results from the bw_tcp portion are highly inconsistent.
2.  Validate network performance via other tools iperf, nfs copies, etc.
3.
  
Actual results:

Random -- or seemingly so

Expected results:

Something consistent good or bad

Additional info:

bw_tcp appears to being called via the network test.  First, the script appears to be passing 1m for a window size and doing so incorrectly.  Second it is invoking the utility to run in a single threaded fashion which yields highly inconsistent results which can be normalized by the use of taskset or numactl to pin the process to a numa domain, by increasing the number of threads.  The same inconsistencies can be seen by running the utility against the localhost interface removing the actual network hardware from the equation.  Attached are runs of bw_tcp with various thread counts, you will note the lower the count the higher the variability and the lower the overall throughput becomes.
Comment 6 Greg Nichols 2013-04-05 10:58:26 EDT
Created attachment 731921 [details]
network patch to fix use of bw_tcp

This patch fixes calls to bw_tcp by setting paralellism to 4 (-P 4), and correctly setting the message size to 1MB (-m 1m).  The results seem to be quite a bit more consistent:

testing bandwidth to gnichols.usersys.redhat.com
bw_tcp -P 4 -m 1m gnichols.usersys.redhat.com
1.048576 114.47 MB/sec
1.048576 114.44 MB/sec
1.048576 114.34 MB/sec
1.048576 115.14 MB/sec
1.048576 114.67 MB/sec

The patch also logs the parameter settings as seen above.
Comment 13 errata-xmlrpc 2013-08-06 13:44:32 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1139.html

Note You need to log in before you can comment on or make changes to this bug.