Bug 1079499 - Review Request: sockperf - network benchmarking utility for testing latency and throughput
Summary: Review Request: sockperf - network benchmarking utility for testing latency a...
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Christopher Meng
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-03-21 17:23 UTC by Michal Schmidt
Modified: 2014-05-17 04:55 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-05-17 04:55:53 UTC
Type: ---
Embargoed:
i: fedora-review+
gwync: fedora-cvs+


Attachments (Terms of Use)

Description Michal Schmidt 2014-03-21 17:23:23 UTC
Spec URL: http://michich.fedorapeople.org/sockperf/sockperf.spec
SRPM URL: http://michich.fedorapeople.org/sockperf/sockperf-2.5.241-1.fc21.src.rpm
Description:
sockperf is a network benchmarking utility over socket API that was designed
for testing performance (latency and throughput) of high-performance systems
(it is also good for testing performance of regular networking systems as
well). It covers most of the socket API calls and options.

Specifically, in addition to the standard throughput tests, sockperf, does the
following:

* Measure latency of each discrete packet at sub-nanosecond resolution (using
  TSC register that counts CPU ticks with very low overhead). 

* Does the above for both ping-pong mode and for latency under load mode. This
  means that we measure latency of single packets even under load of millions
  Packets Per Second (without waiting for reply of packet before sending
  subsequent packet on time) 

* Enable spike analysis by providing histogram, with various percentiles of the
  packets’ latencies (for example: median, min, max, 99% percentile, and more),
  (this is in addition to average and standard deviation). Also, sockperf
  provides full log with all packet’s tx/rx times that can be further analyzed
  with external tools, such as MS-Excel or matplotlib - All this without
  affecting the benchmark itself. 

* Support MANY optional settings for good coverage of socket API and network
  configurations, while still keeping very low overhead in the fast path to
  allow cleanest results.

Fedora Account System Username: michich

Comment 1 Christopher Meng 2014-03-28 09:05:31 UTC
What's the rationale of BR git to apply the patches?

You can try %autosetup macro:

http://www.rpm.org/wiki/PackagerDocs/Autosetup

Comment 2 Michal Schmidt 2014-03-28 12:15:29 UTC
I prefer having the prepped tree as a git tree for the same reasons the wiki page mentions: "The resulting build directory can be used for bisecting problems introduced in patches, and developing new patches from the build directory is more natural than with gendiff."

As to why I am using a sequence of 6 git commands and not simply "%autosetup -S git": I have nicely git formatted patch files, but autosetup applies them internally using "git apply", thus losing the commit messages. Using "git am" gives better results.

Applying patches this way is an often used idiom. systemd or xorg-x11-server are examples of Fedora packages that apply patches this way.

Comment 3 Michal Schmidt 2014-03-28 15:15:39 UTC
By the way, I've proposed a patch to make %autosetup usable for my purposes:
https://bugzilla.redhat.com/show_bug.cgi?id=1082038

Comment 4 Michal Schmidt 2014-04-08 17:54:13 UTC
I've updated the package:

* Tue Apr 08 2014 Michal Schmidt <mschmidt> - 2.5.241-2
- Use %%autosetup.

Spec URL: http://michich.fedorapeople.org/sockperf/sockperf.spec
SRPM URL: http://michich.fedorapeople.org/sockperf/sockperf-2.5.241-2.fc21.src.rpm

Comment 5 Christopher Meng 2014-04-20 12:13:57 UTC
Package Review
==============

Legend:
[x] = Pass, [!] = Fail, [-] = Not applicable, [?] = Not evaluated




===== MUST items =====

C/C++:
[x]: Package does not contain kernel modules.
[x]: Package contains no static executables.
[x]: Package does not contain any libtool archives (.la)
[x]: Rpath absent or only used for internal libs.

Generic:
[x]: Package is licensed with an open-source compatible license and meets
     other legal requirements as defined in the legal section of Packaging
     Guidelines.
[x]: License field in the package spec file matches the actual license.
     Note: Checking patched sources after %prep for licenses. Licenses found:
     "BSD (3 clause)", "Unknown or generated". 2 files have unknown license.
     Detailed output of licensecheck:

BSD (3 clause)
--------------
sockperf-2.5.241/src/Client.cpp
sockperf-2.5.241/src/Client.h
sockperf-2.5.241/src/Defs.cpp
sockperf-2.5.241/src/Defs.h
sockperf-2.5.241/src/IoHandlers.cpp
sockperf-2.5.241/src/IoHandlers.h
sockperf-2.5.241/src/Message.cpp
sockperf-2.5.241/src/Message.h
sockperf-2.5.241/src/PacketTimes.cpp
sockperf-2.5.241/src/PacketTimes.h
sockperf-2.5.241/src/Playback.cpp
sockperf-2.5.241/src/Playback.h
sockperf-2.5.241/src/Server.cpp
sockperf-2.5.241/src/Server.h
sockperf-2.5.241/src/SockPerf.cpp
sockperf-2.5.241/src/Switches.h
sockperf-2.5.241/src/Ticks.cpp
sockperf-2.5.241/src/Ticks.h
sockperf-2.5.241/src/aopt.cpp
sockperf-2.5.241/src/aopt.h
sockperf-2.5.241/src/clock.h
sockperf-2.5.241/src/common.cpp
sockperf-2.5.241/src/common.h
sockperf-2.5.241/src/os_abstract.cpp
sockperf-2.5.241/src/os_abstract.h
sockperf-2.5.241/src/ticks_os.h
sockperf-2.5.241/src/vma-redirect.cpp
sockperf-2.5.241/src/vma-redirect.h
sockperf-2.5.241/tests/vma_multiplexers_test.sh
sockperf-2.5.241/tests/vma_perf_envelope.sh

Unknown or generated
--------------------
sockperf-2.5.241/tests/avner-master-test.sh
sockperf-2.5.241/tests/avner-test.sh

[!]: %build honors applicable compiler flags or justifies otherwise.
[x]: Package contains no bundled libraries without FPC exception.
[x]: Changelog in prescribed format.
[x]: Sources contain only permissible code or content.
[x]: Macros in Summary, %description expandable at SRPM build time.
     Note: Macros in: sockperf (description)
[-]: Package contains desktop file if it is a GUI application.
[-]: Development files must be in a -devel package
[x]: Package uses nothing in %doc for runtime.
[x]: Package consistently uses macros (instead of hard-coded directory names).
[x]: Package is named according to the Package Naming Guidelines.
[x]: Package does not generate any conflict.
[x]: Package obeys FHS, except libexecdir and /usr/target.
[-]: If the package is a rename of another package, proper Obsoletes and
     Provides are present.
[x]: Requires correct, justified where necessary.
[x]: Spec file is legible and written in American English.
[-]: Package contains systemd file(s) if in need.
[!]: Useful -debuginfo package or justification otherwise.
[x]: Package is not known to require an ExcludeArch tag.
[x]: Large documentation must go in a -doc subpackage. Large could be size
     (~1MB) or number of files.
     Note: Documentation size is 235520 bytes in 42 files.
[x]: Package complies to the Packaging Guidelines
[x]: Package successfully compiles and builds into binary rpms on at least one
     supported primary architecture.
[x]: Package installs properly.
[x]: Rpmlint is run on all rpms the build produces.
     Note: There are rpmlint messages (see attachment).
[x]: If (and only if) the source package includes the text of the license(s)
     in its own file, then that file, containing the text of the license(s)
     for the package is included in %doc.
[x]: Package requires other packages for directories it uses.
[x]: Package must own all directories that it creates.
[x]: Package does not own files or directories owned by other packages.
[x]: All build dependencies are listed in BuildRequires, except for any that
     are listed in the exceptions section of Packaging Guidelines.
[x]: Package uses either %{buildroot} or $RPM_BUILD_ROOT
[x]: Package does not run rm -rf %{buildroot} (or $RPM_BUILD_ROOT) at the
     beginning of %install.
[x]: Package does not contain duplicates in %files.
[x]: Permissions on files are set properly.
[x]: Package use %makeinstall only when make install' ' DESTDIR=... doesn't
     work.
[x]: Package is named using only allowed ASCII characters.
[x]: Package do not use a name that already exist
[x]: Package is not relocatable.
[x]: Sources used to build the package match the upstream source, as provided
     in the spec URL.
[x]: Spec file name must match the spec package %{name}, in the format
     %{name}.spec.
[x]: File names are valid UTF-8.
[x]: Packages must not store files under /srv, /opt or /usr/local

===== SHOULD items =====

Generic:
[-]: Sources can be downloaded from URI in Source: tag
     Note: Could not download Source0:
     https://sockperf.googlecode.com/files/sockperf-2.5.241.tar.gz
     See: http://fedoraproject.org/wiki/Packaging:Guidelines#Tags
[x]: If the source package does not include license text(s) as a separate file
     from upstream, the packager SHOULD query upstream to include it.
[x]: Final provides and requires are sane (see attachments).
[x]: Package functions as described.
[x]: Latest version is packaged.
[x]: Package does not include license text files separate from upstream.
[x]: Patches link to upstream bugs/comments/lists or are otherwise justified.
[x]: SourceX tarball generation or download is documented.
     Note: Package contains tarball without URL, check comments
[x]: Description and summary sections in the package spec file contains
     translations for supported Non-English languages, if available.
[x]: Package should compile and build into binary rpms on all supported
     architectures.
[-]: %check is present and all tests pass.
[x]: Packages should try to preserve timestamps of original installed files.
[x]: Packager, Vendor, PreReq, Copyright tags should not be in spec file
[x]: Reviewer should test that the package builds in mock.
[x]: Buildroot is not present
[x]: Package has no %clean section with rm -rf %{buildroot} (or
     $RPM_BUILD_ROOT)
[x]: Dist tag is present (not strictly required in GL).
[x]: No file requires outside of /etc, /bin, /sbin, /usr/bin, /usr/sbin.
[x]: Uses parallel make %{?_smp_mflags} macro.
[x]: SourceX is a working URL.
[x]: Spec use %global instead of %define unless justified.

===== EXTRA items =====

Generic:
[x]: Rpmlint is run on all installed packages.
     Note: There are rpmlint messages (see attachment).
[x]: Large data in /usr/share should live in a noarch subpackage if package is
     arched.
[x]: Package should not use obsolete m4 macros


Rpmlint
-------
Checking: sockperf-2.5.241-2.fc21.i686.rpm
          sockperf-2.5.241-2.fc21.src.rpm
sockperf.i686: W: spelling-error Summary(en_US) benchmarking -> bench marking, bench-marking, benchmark
sockperf.i686: W: spelling-error %description -l en_US benchmarking -> bench marking, bench-marking, benchmark
sockperf.i686: W: spelling-error %description -l en_US latencies -> lateness, enunciates, alienates
sockperf.i686: W: spelling-error %description -l en_US tx -> TX, t, x
sockperf.i686: W: spelling-error %description -l en_US rx -> Rx, ex, r
sockperf.i686: W: spelling-error %description -l en_US matplotlib -> diplomatic
sockperf.i686: E: zero-length /usr/share/doc/sockperf/news
sockperf.src: W: spelling-error Summary(en_US) benchmarking -> bench marking, bench-marking, benchmark
sockperf.src: W: spelling-error %description -l en_US benchmarking -> bench marking, bench-marking, benchmark
sockperf.src: W: spelling-error %description -l en_US latencies -> lateness, enunciates, alienates
sockperf.src: W: spelling-error %description -l en_US tx -> TX, t, x
sockperf.src: W: spelling-error %description -l en_US rx -> Rx, ex, r
sockperf.src: W: spelling-error %description -l en_US matplotlib -> diplomatic
sockperf.src: W: patch-not-applied Patch1: 0001-Don-t-throw-away-Fedora-s-CXXFLAGS.patch
sockperf.src: W: patch-not-applied Patch2: 0002-drop-the-sockperf_-version-copy-of-the-binary.patch
sockperf.src: W: patch-not-applied Patch3: 0003-doc-installation-fixes.patch
sockperf.src: W: invalid-url Source0: https://sockperf.googlecode.com/files/sockperf-2.5.241.tar.gz <urlopen error [Errno 101] Network is unreachable>
2 packages and 0 specfiles checked; 1 errors, 16 warnings.




Rpmlint (installed packages)
----------------------------
# rpmlint sockperf
sockperf.i686: W: spelling-error Summary(en_US) benchmarking -> bench marking, bench-marking, benchmark
sockperf.i686: W: spelling-error %description -l en_US benchmarking -> bench marking, bench-marking, benchmark
sockperf.i686: W: spelling-error %description -l en_US latencies -> lateness, enunciates, alienates
sockperf.i686: W: spelling-error %description -l en_US tx -> TX, t, x
sockperf.i686: W: spelling-error %description -l en_US rx -> Rx, ex, r
sockperf.i686: W: spelling-error %description -l en_US matplotlib -> diplomatic
sockperf.i686: E: zero-length /usr/share/doc/sockperf/news
1 packages and 0 specfiles checked; 1 errors, 6 warnings.
# echo 'rpmlint-done:'



Requires
--------
sockperf (rpmlib, GLIBC filtered):
    libc.so.6
    libdl.so.2
    libgcc_s.so.1
    libgcc_s.so.1(GCC_3.0)
    libm.so.6
    libpthread.so.0
    librt.so.1
    libstdc++.so.6
    libstdc++.so.6(CXXABI_1.3)
    rtld(GNU_HASH)



Provides
--------
sockperf:
    sockperf
    sockperf(x86-32)



Generated by fedora-review 0.5.1 (bb9bf27) last change: 2013-12-13
Command line :/bin/fedora-review -rvn sockperf-2.5.241-2.fc21.src.rpm
Buildroot used: fedora-rawhide-i386
Active plugins: Generic, Shell-api, C/C++
Disabled plugins: Java, Python, fonts, SugarActivity, Ocaml, Perl, Haskell, R, PHP, Ruby
Disabled flags: EXARCH, EPEL5, BATCH, DISTTAG

--------------------------------

1. /builddir/build/BUILD/sockperf-2.5.241/config/aux/missing: Unknown `--is-lightweight' option
Try `/builddir/build/BUILD/sockperf-2.5.241/config/aux/missing --help' for more information
configure: WARNING: 'missing' script is too old or missing

Ask upstream to regenerate files.

2. Same for the doxygen files as #1.

3. g++ -DHAVE_CONFIG_H -I. -I.. -I.    -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches  -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -Werror -Wall  --param inline-unit-growth=200  -D__LINUX__  -DVERSION=2.5.241 -O3 -DNDEBUG -g -O3 -c -o aopt.o aopt.cpp

O3 and NDEBUG are unacceptable. You need to add a new patch.

Others are fine.

Comment 6 Michal Schmidt 2014-04-24 18:36:54 UTC
(In reply to Christopher Meng from comment #5)
> 1. /builddir/build/BUILD/sockperf-2.5.241/config/aux/missing: Unknown
> `--is-lightweight' option
> Try `/builddir/build/BUILD/sockperf-2.5.241/config/aux/missing --help' for
> more information
> configure: WARNING: 'missing' script is too old or missing
> 
> Ask upstream to regenerate files.

I run autoreconf in %prep, but the files in config/aux were not regenerated. Fixed now by calling it with '-f'.

> 2. Same for the doxygen files as #1.

I haven't talked about this with them yet, but yes, it is something to improve in upstream.

> 3. g++ -DHAVE_CONFIG_H -I. -I.. -I.    -O2 -g -pipe -Wall
> -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
> -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches 
> -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -Werror -Wall 
> --param inline-unit-growth=200  -D__LINUX__  -DVERSION=2.5.241 -O3 -DNDEBUG
> -g -O3 -c -o aopt.o aopt.cpp
> 
> O3 and NDEBUG are unacceptable. You need to add a new patch.

According to the packaging guidelies, using higher optimization levels would be acceptable if I could demonstrate significant speedup. Upstream may have had reasons for using these settings, but I do not have the data to show the benefit, so I adjusted patch 0001 to remove the '-O3' and '--param inline-unit-growth=200'.

I disagree about '-DNDEBUG'. All it does is define the NDEBUG macro so that assert() calls are compiled into no code. That's a legitimate upstream's choice to make.

New update:
Spec URL: http://michich.fedorapeople.org/sockperf/sockperf.spec
SRPM URL: http://michich.fedorapeople.org/sockperf/sockperf-2.5.241-3.fc21.src.rpm

Comment 7 Christopher Meng 2014-05-09 11:09:51 UTC
PACKAGE APPROVED.

Comment 8 Michal Schmidt 2014-05-09 11:31:40 UTC
Thank you for the review, Christopher!

New Package SCM Request
=======================
Package Name: sockperf
Short Description: network benchmarking utility for testing latency and throughput
Owners: michich
Branches: f20 el6 epel7
InitialCC:

Comment 9 Gwyn Ciesla 2014-05-09 11:56:23 UTC
Git done (by process-git-requests).


Note You need to log in before you can comment on or make changes to this bug.