Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1494257

Summary: should prevent the deletion of a PVC that is referenced by an active pod
Product: OpenShift Container Platform Reporter: Josh Foots <jfoots>
Component: StorageAssignee: Jan Safranek <jsafrane>
Status: CLOSED CURRENTRELEASE QA Contact: Liang Xia <lxia>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 3.5.0CC: aos-bugs, aos-storage-staff, bchilds, byount, jliggitt, jsafrane, lxia, pdwyer, tbielawa, xtian
Target Milestone: ---Keywords: UpcomingRelease
Target Release: 3.9.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: atomic-openshift-3.9.14-1.git.0.4efa2ca.el7 Doc Type: Bug Fix
Doc Text:
Cause: OpenShift did not check if a volume is used by a pod and recycled volumes even though they were used by active pods. Consequence: Not all files were recycled and running pod could behave unexpectedly because recycler deleted its data. Fix: Recycler does not start recycling until all pods that use the volume are finished. Result: No data is deleted from underneath of running pods .
Story Points: ---
Clone Of:
: 1533078 1533081 1546086 (view as bug list) Environment:
Last Closed: 2018-04-12 07:37:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1533078, 1533081, 1546086    

Description Josh Foots 2017-09-21 20:38:37 UTC
Description of problem:

The mount/unmount logic is controlled by creation/deletion of the pod. But PV/PVC is controlled by pv controller. As mentioned in issue #44297, in current design, after deleting PVC, the mount point was still exist on the Kubernetes node which the Pod was deployed and also the mount point could be seen from inside the Pod.And also if other Pod reuses the PV, then both old Pod and new Pod could access same mount point.

How reproducible:

Very

Steps to Reproduce:
PreReq: PV created (pv_rwo.yaml provided below)

In Project 1:
1. Claim the persistent volume using a PVC (oc create -f pvc_rwo.yaml)
2. Deploy a pod using the PVC/PV (oc create -f pod.yaml)
3. Write some stuff to the PV (used console terminal, not necessary, but interesting later)
4. Delete the claim (oc delete -f pvc_rwo.yaml)
* Do not delete the Pod

Change into Project 2:
1. Claim the same persistent volume using a PVC (oc create -f pvc_rwo.yaml)
2. Deploy a pod using the PVC/PV (oc create -f pod.yaml)
3. Write some stuff to the PV (used console terminal)

Go back to Project 1:
1. Go to pod's terminal and ls, you'll see pod from project2's data
2. Write some stuff -succeeds


Actual results:

This works

Expected results:

For the PV to be unable to be deleted while an active pod is using it.


Description of problem:

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:

Master Log:

Node Log (of failed PODs):

PV Dump:

PVC Dump:

StorageClass Dump (if StorageClass used by PV/PVC):

Additional info:

Comment 6 Pavel Pospisil 2017-11-21 07:21:14 UTC
Upstream PR is created: https://github.com/kubernetes/kubernetes/pull/55824

Comment 11 Pavel Pospisil 2017-12-18 10:46:50 UTC
Finally, all necessary PRs were merged into Kubernetes.
Particularly into Kubernetes 1.9:
- https://github.com/kubernetes/kubernetes/pull/55873
- https://github.com/kubernetes/kubernetes/pull/55824

Into Kubernetes 1.10:
- https://github.com/kubernetes/kubernetes/pull/55957
- https://github.com/kubernetes/kubernetes/pull/56298

Kubernetes documentation was updated here: https://github.com/kubernetes/website/pull/6415

Comment 26 Jan Safranek 2018-02-05 14:50:01 UTC
Upstream PR: https://github.com/kubernetes/kubernetes/pull/59350

Comment 27 Jan Safranek 2018-02-09 17:22:46 UTC
Origin PR: https://github.com/openshift/origin/pull/18552

Comment 32 Liang Xia 2018-03-08 07:37:32 UTC
pvc-protection is working fine on below version,
openshift v3.9.3

Move bug to verified.

Comment 34 Jan Safranek 2018-04-12 07:37:11 UTC
I wonder why the bug was excluded from errata. Closing manually.