Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1532512

Summary: Expand PVC successfully but have a "VolumeResizeFailed" event
Product: OpenShift Container Platform Reporter: Qin Ping <piqin>
Component: StorageAssignee: Hemant Kumar <hekumar>
Status: CLOSED ERRATA QA Contact: Qin Ping <piqin>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.9.0CC: aos-bugs, aos-storage-staff, bchilds, hchiramm, hekumar, pprakash
Target Milestone: ---   
Target Release: 3.9.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-03-28 14:18:26 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Qin Ping 2018-01-09 07:55:14 UTC
Description of problem:
Expand PVC successfully, but if using oc describe pvc cmd will see a strange evnent. 
Warning  VolumeResizeFailed     8m    volume_expand                Operation cannot be fulfilled on persistentvolumeclaims "glusterc": the object has been modified; please apply your changes to the latest version and try again

Version-Release number of selected component (if applicable):
oc v3.9.0-0.16.0
openshift v3.9.0-0.16.0
kubernetes v1.9.0-beta1


How reproducible:
Always

Steps to Reproduce:
1. Deploy CNS on OCP v3.9
2. Enable feature gates ExpandPersistentVolumes and admission-control PersistentVolumeClaimResize
3. Create a storageclass glusterfs with allowVolumeExpansion=true
4. Create a dynamic provisioning PVC using the storageclass
5. Expand PVC from 1Gi to 3Gi
6. When the PVC status is Resizing, expand PVC from 3Gi to 5Gi
7. Using oc get pvc -o yaml and oc describe pvc to check expand result.

Actual results:
Expand finished, but report a strange event.

Expected results:
Expand finished and report expand PVC successfully.

PVC Dump:
# oc get pvc -o yaml
apiVersion: v1
items:
- apiVersion: v1
  kind: PersistentVolumeClaim
  metadata:
    annotations:
      pv.kubernetes.io/bind-completed: "yes"
      pv.kubernetes.io/bound-by-controller: "yes"
      volume.beta.kubernetes.io/storage-class: glusterfs
      volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/glusterfs
    creationTimestamp: 2018-01-09T07:29:28Z
    name: glusterc
    namespace: piqin
    resourceVersion: "460320"
    selfLink: /api/v1/namespaces/piqin/persistentvolumeclaims/glusterc
    uid: d37c3063-f50e-11e7-8ffd-0050569f5abb
  spec:
    accessModes:
    - ReadWriteOnce
    resources:
      requests:
        storage: 5Gi
    volumeName: pvc-d37c3063-f50e-11e7-8ffd-0050569f5abb
  status:
    accessModes:
    - ReadWriteOnce
    capacity:
      storage: 8G
    phase: Bound
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

StorageClass Dump (if StorageClass used by PV/PVC):
allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  creationTimestamp: 2018-01-04T10:23:06Z
  name: glusterfs
  resourceVersion: "453137"
  selfLink: /apis/storage.k8s.io/v1/storageclasses/glusterfs
  uid: 41157a28-f139-11e7-9c0c-0050569f5abb
parameters:
  resturl: http://172.30.102.160:8080
provisioner: kubernetes.io/glusterfs
reclaimPolicy: Delete


Additional info:
# oc describe pvc
Name:          glusterc
Namespace:     piqin
StorageClass:  glusterfs
Status:        Bound
Volume:        pvc-d37c3063-f50e-11e7-8ffd-0050569f5abb
Labels:        <none>
Annotations:   pv.kubernetes.io/bind-completed=yes
               pv.kubernetes.io/bound-by-controller=yes
               volume.beta.kubernetes.io/storage-class=glusterfs
               volume.beta.kubernetes.io/storage-provisioner=kubernetes.io/glusterfs
Finalizers:    []
Capacity:      8G
Access Modes:  RWO
Events:
  Type     Reason                 Age   From                         Message
  ----     ------                 ----  ----                         -------
  Normal   ProvisioningSucceeded  14m   persistentvolume-controller  Successfully provisioned volume pvc-d37c3063-f50e-11e7-8ffd-0050569f5abb using kubernetes.io/glusterfs
  Warning  VolumeResizeFailed     8m    volume_expand                Operation cannot be fulfilled on persistentvolumeclaims "glusterc": the object has been modified; please apply your changes to the latest version and try again

Comment 1 Hemant Kumar 2018-01-16 16:33:47 UTC
alright this is expected, I will reduce severity of error message in this case and rename `VolumeResizeFailed` to something less severe.

Comment 2 Hemant Kumar 2018-01-17 21:20:09 UTC
Fixed https://github.com/kubernetes/kubernetes/pull/58415 .

The resizeFailed event will still be recorded but when resize is successful, another resize successful message will be recorded with the pvc.

Comment 3 Hemant Kumar 2018-02-07 18:45:21 UTC
https://github.com/openshift/origin/pull/18509

Comment 5 Qin Ping 2018-02-27 06:43:50 UTC
Verified in:
openshift v3.9.0-0.53.0
glusterfs-3.8.4-54.el7rhgs.x86_64
heketi-5.0.0-19.el7rhgs.x86_64

Comment 8 errata-xmlrpc 2018-03-28 14:18:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0489