| Summary: | Slowness Writing to SAN Disk-backed Storage | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Steven Walter <stwalter> |
| Component: | Storage | Assignee: | Jeremy Eder <jeder> |
| Status: | CLOSED NOTABUG | QA Contact: | Wenqi He <wehe> |
| Severity: | urgent | Docs Contact: | |
| Priority: | urgent | ||
| Version: | 3.2.0 | CC: | aos-bugs, bchilds, ekuric, eparis, erich, hamiller, hchen, jeder, shberry |
| Target Milestone: | --- | Keywords: | Performance |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | aos-scalability-34 | ||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-01-27 16:17:10 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
Description of problem: Customer is seeing significant slowness when writing to disk. This appears to be occurring in OpenShift containers but NOT in vanilla docker containers. The storage they are using is Gluster-based storage backed by SAN disks. The hosts are also backed by these SAN disks, so when using "ephemeral" storage (non-persistent volumes), it is using the same backing storage. Behavior is consistent regardless if the Gluster frontend is used, so it is not Gluster issue. i/o tests for the SAN disk backend: - Writing to the SAN disk from the VMWare host VM has good i/o speeds - OpenShift pods with SAN disk-backed persistent volumes see very slow i/o writing to the storage - Containers run with "docker run" see host-like i/o speeds $ oc describe pvc esm-jenkins Name: esm-jenkins Namespace: esm Status: Bound Volume: app-pv-036-1g Labels: template=jenkins-persistent-template Capacity: 1Gi Access Modes: RWO No events. $ oc describe pv app-pv-036-1g Name: app-pv-036-1g Labels: <none> Status: Bound Claim: esm/esm-jenkins Reclaim Policy: Recycle Access Modes: RWO Capacity: 1Gi Message: Source: Type: Glusterfs (a Glusterfs mount on the host that shares a pod's lifetime) EndpointsName: glusterfs-cluster-app Path: /app-pv-036-1G ReadOnly: false No events. Version-Release number of selected component (if applicable): 3.2.0 How reproducible: Not yet reproduced Adding more detail in priv comment