Bug 1298507 - Cinder volume is detached from running pod
Cinder volume is detached from running pod
Product: OpenShift Origin
Classification: Red Hat
Component: Storage (Show other bugs)
Unspecified Unspecified
high Severity medium
: ---
: ---
Assigned To: Jan Safranek
Jianwei Hou
Depends On:
  Show dependency treegraph
Reported: 2016-01-14 04:59 EST by Jan Safranek
Modified: 2016-05-12 13:12 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2016-05-12 13:12:18 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Jan Safranek 2016-01-14 04:59:20 EST
In very rare case, a Cinder volume can be detached from a running pod. The pod is not informed, application running there suddenly cannot read/write any data from/to its volume(s). This can corrupt filesystem on the volume as well as application data there (e.g. database files)!

See https://github.com/kubernetes/kubernetes/issues/19602 for technical details.

It involves probably only Cinder, AWS is safe and I was not able to reproduce it on GCE so far.

Version-Release number of selected component (if applicable):
origin 3.1.0

How reproducible:
rarely (< 10% even in perfect conditions for the reproducer, much less in any real deployment)

Steps to Reproduce:
see the github issue
Comment 1 Mark Turansky 2016-02-04 08:29:54 EST
Fix merged upstream. Awaiting rebase into Origin.

Comment 2 Jan Safranek 2016-02-08 06:20:14 EST
In case there is no rebase I filled Origin PR: https://github.com/openshift/origin/pull/7108
Comment 3 Jan Safranek 2016-02-09 09:38:20 EST
Origin PR merged
Comment 4 Jianwei Hou 2016-02-15 21:32:59 EST
Verified on openshift v1.1.2-260-gf556adc, this bug is not reproducible.

Note You need to log in before you can comment on or make changes to this bug.