Bug 1348068

Summary: High i/o latency with random write workload from virtual machine(s)
Product: [Community] GlusterFS Reporter: Paul Cuzner <pcuzner>
Component: replicateAssignee: Krutika Dhananjay <kdhananj>
Status: CLOSED EOL QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: 3.7.9CC: bugs, mliyazud, pkarampu
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1392419 (view as bug list) Environment:
Last Closed: 2017-03-08 11:03:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1392419    
Attachments:
Description Flags
iometer config file
none
screenshot showing the max latency at 4secs within the iometer run none

Description Paul Cuzner 2016-06-20 05:21:02 UTC
Created attachment 1169665 [details]
iometer config file

Description of problem:
When using either a replica 3 or sharded replica 3 volume with virtual machines, profiles that exhibit random writes cause high latencies than can sometimes span several seconds.

This has been observed with both fio and iometer as load generators

Sequential writes do NOT exhibit this affect

Version-Release number of selected component (if applicable):


How reproducible:
Every time the workload is run this profile is observed


Steps to Reproduce:
1. use either an fio workload with random write or an iometer profile (attached)
2. track the latency in the vm with pcp
3. 

Actual results:
High latencies are observed which could impact application response times

Expected results:
Spikes in random write workloads are accepted - may be in the 10's of milliseconds - but latencies that are between 600ms 2s are a problem


Additional info:

Comment 1 Paul Cuzner 2016-06-20 05:56:54 UTC
Created attachment 1169677 [details]
screenshot showing the max latency at 4secs within the iometer run

Added a screenshot showing the max latency observed from the iometer run. This is from a single vm running the workload (as per the icf file attached to the case)

Comment 2 Kaushal 2017-03-08 11:03:38 UTC
This bug is getting closed because GlusteFS-3.7 has reached its end-of-life.

Note: This bug is being closed using a script. No verification has been performed to check if it still exists on newer releases of GlusterFS.
If this bug still exists in newer GlusterFS releases, please reopen this bug against the newer release.