Bug 1298507

Summary:	Cinder volume is detached from running pod
Product:	OKD	Reporter:	Jan Safranek <jsafrane>
Component:	Storage	Assignee:	Jan Safranek <jsafrane>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Jianwei Hou <jhou>
Severity:	medium	Docs Contact:
Priority:	high
Version:	3.x	CC:	aos-bugs, bchilds, mturansk
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-05-12 17:12:18 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Jan Safranek 2016-01-14 09:59:20 UTC

In very rare case, a Cinder volume can be detached from a running pod. The pod is not informed, application running there suddenly cannot read/write any data from/to its volume(s). This can corrupt filesystem on the volume as well as application data there (e.g. database files)!

See https://github.com/kubernetes/kubernetes/issues/19602 for technical details.

It involves probably only Cinder, AWS is safe and I was not able to reproduce it on GCE so far.

Version-Release number of selected component (if applicable):
origin 3.1.0

How reproducible:
rarely (< 10% even in perfect conditions for the reproducer, much less in any real deployment)

Steps to Reproduce:
see the github issue

Comment 1 Mark Turansky 2016-02-04 13:29:54 UTC

Fix merged upstream. Awaiting rebase into Origin.

https://github.com/kubernetes/kubernetes/pull/19707

Comment 2 Jan Safranek 2016-02-08 11:20:14 UTC

In case there is no rebase I filled Origin PR: https://github.com/openshift/origin/pull/7108

Comment 3 Jan Safranek 2016-02-09 14:38:20 UTC

Origin PR merged

Comment 4 Jianwei Hou 2016-02-16 02:32:59 UTC

Verified on openshift v1.1.2-260-gf556adc, this bug is not reproducible.