Bug 1608537 - storageclass gp2: reclaimPolicy is empty string and the default behavior is changed
Summary: storageclass gp2: reclaimPolicy is empty string and the default behavior is c...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Storage
Version: 3.11.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 3.11.0
Assignee: Matthew Wong
QA Contact: Chao Yang
URL:
Whiteboard:
: 1618671 (view as bug list)
Depends On: 1618671
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-07-25 18:32 UTC by Hongkai Liu
Modified: 2018-10-11 07:22 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-10-11 07:22:15 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:2652 0 None None None 2018-10-11 07:22:40 UTC

Description Hongkai Liu 2018-07-25 18:32:39 UTC
Description of problem:
# oc get sc gp2 -o yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  annotations:
    storageclass.beta.kubernetes.io/is-default-class: "true"
  creationTimestamp: 2018-07-25T14:50:35Z
  name: gp2
  resourceVersion: "2240"
  selfLink: /apis/storage.k8s.io/v1/storageclasses/gp2
  uid: 16af295d-901a-11e8-ab46-0263d1e683c8
parameters:
  encrypted: "false"
  kmsKeyId: ""
  type: gp2
provisioner: kubernetes.io/aws-ebs
reclaimPolicy: ""
volumeBindingMode: Immediate
==================================
reclaimPolicy is empty string but PVCs based on gp2 is with Retain as reclaim policy.

# oc process -f https://raw.githubusercontent.com/hongkailiu/svt-case-doc/master/files/pvc_template.yaml -p PVC_NAME=pvcaaa | oc create -f -

# oc get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS     CLAIM                    STORAGECLASS   REASON    AGE
pvc-a986a3c7-9033-11e8-ab46-0263d1e683c8   3Gi        RWO            Retain           Released   aaa/pvcaaa               gp2

As a result of "Retain", the PV is still there after the PVC gets deleted.
However, it contradicts the doc "For dynamically provisioned PersistentVolumes, the default reclaim policy is “Delete”."
https://kubernetes.io/docs/tasks/administer-cluster/change-pv-reclaim-policy/


In OCP 3.10 (3.10.18-1.git.0.13dc4a0.el7), we have the expected behavior:
# oc get sc gp2 -o yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  annotations:
    storageclass.beta.kubernetes.io/is-default-class: "true"
  creationTimestamp: 2018-07-25T16:11:25Z
  name: gp2
  resourceVersion: "2229"
  selfLink: /apis/storage.k8s.io/v1/storageclasses/gp2
  uid: 615cf557-9025-11e8-afa0-02f7ed12d48c
parameters:
  encrypted: "false"
  kmsKeyId: ""
  type: gp2
provisioner: kubernetes.io/aws-ebs
reclaimPolicy: Delete
volumeBindingMode: Immediate

Is this change of the default behavior intended?


Version-Release number of selected component (if applicable):
# yum list installed | grep openshift
atomic-openshift.x86_64         3.11.0-0.9.0.git.0.0bffd7b.el7

How reproducible: Always

Steps to Reproduce:
1. 
2.
3.

Actual results:


Expected results:

Master Log:

Node Log (of failed PODs):

PV Dump:

PVC Dump:

StorageClass Dump (if StorageClass used by PV/PVC):

Additional info:

Comment 1 Matthew Wong 2018-07-25 21:21:03 UTC
what happens if a storageClass with `reclaimPolicy: ""` is created manually? It should be rejected. Or one with no reclaimPolicy at all? It should default to Delete. Defaulting/validation code has not changed in between 3.10/3.11, it should not be possible for a storageClass with `reclaimPolicy: ""` to exist. 

PV being set to retain is OK in this case because it is a fail-safe for when storageClass has `reclaimPolicy: ""` which, again, shouldn't even be possible!

Comment 2 Matthew Wong 2018-07-25 21:26:04 UTC
Actually, I am wrong: this line looks like a mistake https://github.com/kubernetes/kubernetes/blob/master/pkg/apis/storage/validation/validation.go#L117 , it is allowing storageClasses to have reclaimPolicy=="" which is bad. I will make a PR upstream.

Comment 3 Matthew Wong 2018-07-25 21:39:37 UTC
So there's 2 issues:
1) https://github.com/openshift/openshift-ansible/issues/9340 , openshift-anisble was updated to support setting the default storage class's reclaimPolicy but there is a minor error where it defaults to "" instead of None/nil.
2) reclaimPolicy of "" shouldn't be allowed in the first place.

My PR will fix 2), 1) is probably trivial to fix as well.

Comment 4 Matthew Wong 2018-07-26 21:33:18 UTC
Upstream PR to make `reclaimPolicy=""` invalid opened https://github.com/kubernetes/kubernetes/pull/66682

Comment 5 Matthew Wong 2018-08-24 14:44:02 UTC
*** Bug 1618671 has been marked as a duplicate of this bug. ***

Comment 6 Matthew Wong 2018-08-28 20:20:23 UTC
https://github.com/openshift/openshift-ansible/pull/9781 has merged

Comment 8 Hongkai Liu 2018-09-05 17:57:11 UTC
Verified with
# git log --oneline -1
6ead45b Merge pull request #9919 from rdoxenham/master

# yum list installed | grep openshift
atomic-openshift.x86_64         3.11.0-0.21.0.git.0.abddb61.el7

# oc get sc gp2 -o yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  annotations:
    storageclass.beta.kubernetes.io/is-default-class: "true"
  creationTimestamp: 2018-09-05T14:03:55Z
  name: gp2
  resourceVersion: "1786"
  selfLink: /apis/storage.k8s.io/v1/storageclasses/gp2
  uid: 86b835f4-b114-11e8-b251-024152172b16
parameters:
  encrypted: "false"
  kmsKeyId: ""
  type: gp2
provisioner: kubernetes.io/aws-ebs
reclaimPolicy: Delete
volumeBindingMode: Immediate


Delete is back in the sc definition.
Thanks for the fix.

Comment 10 errata-xmlrpc 2018-10-11 07:22:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2652


Note You need to log in before you can comment on or make changes to this bug.