Bug 1532512 - Expand PVC successfully but have a "VolumeResizeFailed" event
Summary: Expand PVC successfully but have a "VolumeResizeFailed" event
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Storage
Version: 3.9.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 3.9.0
Assignee: Hemant Kumar
QA Contact: Qin Ping
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-01-09 07:55 UTC by Qin Ping
Modified: 2018-03-28 14:18 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-03-28 14:18:26 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1535939 0 unspecified CLOSED [RFE] Support online resize of GlusterFS PVs. 2023-09-14 04:15:41 UTC
Red Hat Product Errata RHBA-2018:0489 0 None None None 2018-03-28 14:18:45 UTC

Internal Links: 1535939

Description Qin Ping 2018-01-09 07:55:14 UTC
Description of problem:
Expand PVC successfully, but if using oc describe pvc cmd will see a strange evnent. 
Warning  VolumeResizeFailed     8m    volume_expand                Operation cannot be fulfilled on persistentvolumeclaims "glusterc": the object has been modified; please apply your changes to the latest version and try again

Version-Release number of selected component (if applicable):
oc v3.9.0-0.16.0
openshift v3.9.0-0.16.0
kubernetes v1.9.0-beta1


How reproducible:
Always

Steps to Reproduce:
1. Deploy CNS on OCP v3.9
2. Enable feature gates ExpandPersistentVolumes and admission-control PersistentVolumeClaimResize
3. Create a storageclass glusterfs with allowVolumeExpansion=true
4. Create a dynamic provisioning PVC using the storageclass
5. Expand PVC from 1Gi to 3Gi
6. When the PVC status is Resizing, expand PVC from 3Gi to 5Gi
7. Using oc get pvc -o yaml and oc describe pvc to check expand result.

Actual results:
Expand finished, but report a strange event.

Expected results:
Expand finished and report expand PVC successfully.

PVC Dump:
# oc get pvc -o yaml
apiVersion: v1
items:
- apiVersion: v1
  kind: PersistentVolumeClaim
  metadata:
    annotations:
      pv.kubernetes.io/bind-completed: "yes"
      pv.kubernetes.io/bound-by-controller: "yes"
      volume.beta.kubernetes.io/storage-class: glusterfs
      volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/glusterfs
    creationTimestamp: 2018-01-09T07:29:28Z
    name: glusterc
    namespace: piqin
    resourceVersion: "460320"
    selfLink: /api/v1/namespaces/piqin/persistentvolumeclaims/glusterc
    uid: d37c3063-f50e-11e7-8ffd-0050569f5abb
  spec:
    accessModes:
    - ReadWriteOnce
    resources:
      requests:
        storage: 5Gi
    volumeName: pvc-d37c3063-f50e-11e7-8ffd-0050569f5abb
  status:
    accessModes:
    - ReadWriteOnce
    capacity:
      storage: 8G
    phase: Bound
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

StorageClass Dump (if StorageClass used by PV/PVC):
allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  creationTimestamp: 2018-01-04T10:23:06Z
  name: glusterfs
  resourceVersion: "453137"
  selfLink: /apis/storage.k8s.io/v1/storageclasses/glusterfs
  uid: 41157a28-f139-11e7-9c0c-0050569f5abb
parameters:
  resturl: http://172.30.102.160:8080
provisioner: kubernetes.io/glusterfs
reclaimPolicy: Delete


Additional info:
# oc describe pvc
Name:          glusterc
Namespace:     piqin
StorageClass:  glusterfs
Status:        Bound
Volume:        pvc-d37c3063-f50e-11e7-8ffd-0050569f5abb
Labels:        <none>
Annotations:   pv.kubernetes.io/bind-completed=yes
               pv.kubernetes.io/bound-by-controller=yes
               volume.beta.kubernetes.io/storage-class=glusterfs
               volume.beta.kubernetes.io/storage-provisioner=kubernetes.io/glusterfs
Finalizers:    []
Capacity:      8G
Access Modes:  RWO
Events:
  Type     Reason                 Age   From                         Message
  ----     ------                 ----  ----                         -------
  Normal   ProvisioningSucceeded  14m   persistentvolume-controller  Successfully provisioned volume pvc-d37c3063-f50e-11e7-8ffd-0050569f5abb using kubernetes.io/glusterfs
  Warning  VolumeResizeFailed     8m    volume_expand                Operation cannot be fulfilled on persistentvolumeclaims "glusterc": the object has been modified; please apply your changes to the latest version and try again

Comment 1 Hemant Kumar 2018-01-16 16:33:47 UTC
alright this is expected, I will reduce severity of error message in this case and rename `VolumeResizeFailed` to something less severe.

Comment 2 Hemant Kumar 2018-01-17 21:20:09 UTC
Fixed https://github.com/kubernetes/kubernetes/pull/58415 .

The resizeFailed event will still be recorded but when resize is successful, another resize successful message will be recorded with the pvc.

Comment 3 Hemant Kumar 2018-02-07 18:45:21 UTC
https://github.com/openshift/origin/pull/18509

Comment 5 Qin Ping 2018-02-27 06:43:50 UTC
Verified in:
openshift v3.9.0-0.53.0
glusterfs-3.8.4-54.el7rhgs.x86_64
heketi-5.0.0-19.el7rhgs.x86_64

Comment 8 errata-xmlrpc 2018-03-28 14:18:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0489


Note You need to log in before you can comment on or make changes to this bug.