Bug 1332470

Hammer Source File :  src/common/obj_bencher.cc

 if (i % 20 == 0) {
      if (i > 0)
        cur_time.localtime(cout) << "min lat: " << data.min_latency
             << " max lat: " << data.max_latency
             << " avg lat: " << data.avg_latency << std::endl;
      //I'm naughty and don't reset the fill
      bencher->out(cout, cur_time) << setfill(' ')
           << setw(5) << "sec"
           << setw(8) << "Cur ops"
           << setw(10) << "started"
           << setw(10) << "finished"
           << setw(10) << "avg MB/s"
           << setw(10) << "cur MB/s"
           << setw(10) << "last lat"
           << setw(10) << "avg lat" << std::endl;
    }

last lat and avg lat here has setw(10) : which is causing the issue.


But in Jewel it was changed to :

  bencher->out(cout, cur_time) << setfill(' ')
          << setw(5) << "sec"
          << setw(8) << "Cur ops"
          << setw(10) << "started"
          << setw(10) << "finished"
          << setw(10) << "avg MB/s"
          << setw(10) << "cur MB/s"
          << setw(12) << "last lat(s)"
          << setw(12) << "avg lat(s)" << std::endl;
    }

*last lat* and *avg lat * is changed to setw(12) :  I hope this will solve the issue.

   << setw(12) << "last lat(s)"
   << setw(12) << "avg lat(s)" << std::endl;

Upstream Hammer backport : http://tracker.ceph.com/issues/15717

https://github.com/ceph/ceph/commit/669b9329 does not fix this problem.

2016-04-27 17:40:30.470978min lat:

Working on making 669b9329 work with hammer and adding an additional patch for the timestampnogapmin issue.

Just so we are clear.

The actual problem with the hammer source code is that we have setw(10) for both the column header and the field itself so they can both be the same size.

<< setw(10) << "last lat"
...
<< setw(10) << (double)data.cur_latency

In the newer code the columns have been made bigger but the field is restricted to one less char than the header so they can't be the same size and a space must always exist between fields *and* we've added a forced space as well in the output.

<< setw(12) << "last lat(s)"
...
<< ' ' << setw(11) << (double)data.cur_latency

That was introduced by this commit, https://github.com/ceph/ceph/commit/474c98961fe0aa85881929fb8e08550ce1666197. I'll need to do a bit more investigation and testing to get the hammer commit right.

I've created https://github.com/ceph/ceph/pull/8960

I'm attaching hammer-rados-bench-formatting.patch which includes all of the patches in that PR. If it applies cleanly (or maybe with a bit of massaging) you could create a test package and you, and the customer, could test it. Since it only requires a client system it should be very low impact.

Created attachment 1154501 [details]
Patch

Diff including all commits from https://github.com/ceph/ceph/pull/8960

(In reply to Brad Hubbard from comment #14)
> Created attachment 1154501 [details]
> Patch
> 
> Diff including all commits from https://github.com/ceph/ceph/pull/8960

Thanks Brad. Sure I will try to test it.

Created attachment 1156637 [details]
rados bench test with fixes in PR : https://github.com/ceph/ceph/pull/8960

I have tested rados bench after applying the patches in above listed PR and now it looks perfect.

(In reply to Brad Hubbard from comment #13)
> I've created https://github.com/ceph/ceph/pull/8960
> 
> I'm attaching hammer-rados-bench-formatting.patch which includes all of the
> patches in that PR. If it applies cleanly (or maybe with a bit of massaging)
> you could create a test package and you, and the customer, could test it.
> Since it only requires a client system it should be very low impact.

Brad, Please find test results in comment#17 attachment.  Now rados bench output looks perfect.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-1972.html