Bug 1394895 - Slowness Writing to SAN Disk-backed Storage
Summary: Slowness Writing to SAN Disk-backed Storage
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Storage
Version: 3.2.0
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: ---
Assignee: Jeremy Eder
QA Contact: Wenqi He
URL:
Whiteboard: aos-scalability-34
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-11-14 16:27 UTC by Steven Walter
Modified: 2020-01-17 16:10 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-01-27 16:17:10 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Steven Walter 2016-11-14 16:27:19 UTC
Description of problem:
Customer is seeing significant slowness when writing to disk. This appears to be occurring in OpenShift containers but NOT in vanilla docker containers.
The storage they are using is Gluster-based storage backed by SAN disks. The hosts are also backed by these SAN disks, so when using "ephemeral" storage (non-persistent volumes), it is using the same backing storage. Behavior is consistent regardless if the Gluster frontend is used, so it is not Gluster issue.

i/o tests for the SAN disk backend:

- Writing to the SAN disk from the VMWare host VM has good i/o speeds
- OpenShift pods with SAN disk-backed persistent volumes see very slow i/o writing to the storage
- Containers run with "docker run" see host-like i/o speeds

$ oc describe pvc esm-jenkins
Name:		esm-jenkins
Namespace:	esm
Status:		Bound
Volume:		app-pv-036-1g
Labels:		template=jenkins-persistent-template
Capacity:	1Gi
Access Modes:	RWO
No events.

$ oc describe pv app-pv-036-1g
Name:		app-pv-036-1g
Labels:		<none>
Status:		Bound
Claim:		esm/esm-jenkins
Reclaim Policy:	Recycle
Access Modes:	RWO
Capacity:	1Gi
Message:
Source:
    Type:		Glusterfs (a Glusterfs mount on the host that shares a pod's lifetime)
    EndpointsName:	glusterfs-cluster-app
    Path:		/app-pv-036-1G
    ReadOnly:		false
No events.



Version-Release number of selected component (if applicable):

3.2.0

How reproducible:
Not yet reproduced


Adding more detail in priv comment


Note You need to log in before you can comment on or make changes to this bug.