Bug 1475721 - 'gluster volume profile' shouldn't cause negative impact on volume performance
'gluster volume profile' shouldn't cause negative impact on volume performance
Status: CLOSED ERRATA
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: core (Show other bugs)
unspecified
Unspecified Linux
unspecified Severity high
: ---
: RHGS 3.3.1
Assigned To: Krutika Dhananjay
Karan Sandha
: ZStream
Depends On:
Blocks: 1475687
  Show dependency treegraph
 
Reported: 2017-07-27 04:36 EDT by Amar Tumballi
Modified: 2017-11-28 22:29 EST (History)
7 users (show)

See Also:
Fixed In Version: glusterfs-3.8.4-45
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-11-28 22:29:14 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Amar Tumballi 2017-07-27 04:36:52 EDT
Description of problem:
Till now, `gluster volume profile` command is recommended to be used in a production system only when there are issues, and there is a need to understand the I/O pattern. The major reason was the impact on volume performance because of enabling the option.


Version-Release number of selected component (if applicable):
3.3.0

How reproducible:
- this is an hypothesis - 


Actual results:
* Recommended only in few cases and not in actual production setup.

Expected results:
* Always have the option enabled, and use the metrics from filesystem to show the performance pattern over time.

Additional info:

Gluster volume profile can provide a lot of internal information on gluster's performance, and this can be used to plot metrics pretty well.
Comment 4 Krutika Dhananjay 2017-08-03 07:12:21 EDT
I don't have context on this issue except for knowing that we plan to enable profile by default and analyse the perf data. Do we know or have evidence to support the claim that enabling volume profiling does have a negative impact on performance?

-Krutika
Comment 5 Amar Tumballi 2017-08-03 07:37:09 EDT
>  we plan to enable profile by default and analyse the perf data.

Yes, above is the context for this bug to exist.

>  Do we know or have evidence to support the claim that enabling volume profiling does have a negative impact on performance?

Nothing where it was 'analyzed' before. Mostly hypothesis:

1. We don't turn it on by default, but ask customers to give the output when they get into perf issues. 
2. Inside code, if 'measure_latency' flag is set, it is a gettimeofday() syscall for every layer of translator for every fop().
3. Earlier in 'debug/io-stats' translator, we had xlator->lock which was used to update all the counters, which surely caused locking contentions at io-stats.

Hence, we want a performance run with profiling on, so we can see how much we regress, and see if there are anything more to enhance it further.
Comment 7 Atin Mukherjee 2017-09-04 05:54:49 EDT
downstream patch : https://code.engineering.redhat.com/gerrit/#/c/117185
Comment 11 errata-xmlrpc 2017-11-28 22:29:14 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:3276

Note You need to log in before you can comment on or make changes to this bug.