Bug 1919291
| Summary: | [Cinder-csi-driver] Filesystem did not expand when volume resize | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Wei Duan <wduan> |
| Component: | Storage | Assignee: | Mike Fedosin <mfedosin> |
| Storage sub component: | OpenStack CSI Drivers | QA Contact: | Wei Duan <wduan> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | high | ||
| Priority: | high | CC: | aos-bugs, m.andre, mbooth, piqin, pprinett |
| Version: | 4.7 | Keywords: | Triaged |
| Target Milestone: | --- | ||
| Target Release: | 4.8.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: |
Cause: During resizing in Cinder CSI Driver "findmnt" command got several volume mounts and couldn't pick up the right one. So, it stopped the resizing.
Consequence: After PV resizing filesystem on the volume had the same size, so users had to extend it manually.
Fix: Use the first mount for "findmnt" command.
Result: Filesystem is resized along with the PV.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-07-27 22:36:39 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Is this a regression from 4.6? Was volume resizing available in 4.6? We might not be able to fix within 4.7; targeting 4.8 and definitely planning on backporting. It is not a regression as we just have the cinder-csi-driver. Maybe you want to ask about cinder in-tree, it doesn't support on-line expandtion, so agree to change targeting 4.8 and backporting. Thanks. It is an upstream issue, the fix is on review *** Bug 1906391 has been marked as a duplicate of this bug. *** Verified pass on 4.8.0-0.nightly-2021-03-30-160509. $ oc get pvc -w NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE mypvc Bound pvc-c2cbf54e-07df-4e14-870e-ec1de20175b5 2Gi RWO standard-csi 22m mypvc Bound pvc-c2cbf54e-07df-4e14-870e-ec1de20175b5 3Gi RWO standard-csi 23m # df -h Filesystem Size Used Avail Use% Mounted on ... /dev/vdb 2.9G 1.9G 1008M 66% /mnt/local ... Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:2438 |
[Cinder-csi-driver] Filesystem did not expand for on-line volume resize storage/Openstack CSI Driver H Description of problem: Filesystem did not expand for on-line volume resizing although the pvc is expanded Version-Release number of selected component (if applicable): 4.7.0-0.nightly-2021-01-21-215614 How reproducible: Always Steps to Reproduce: 1. Create cluster on OSP with Cinder-csi-driver installed 2. Create pod(mypod) and pvc(mypvc) with sc standard-csi $ oc get pod NAME READY STATUS RESTARTS AGE mypod 1/1 Running 0 21m $ oc get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE mypvc Bound pvc-0e27b8cb-ed0e-4291-8b74-a7faf29e7c6d 2Gi RWO standard-csi 21m 3. Resize pvc from 2Gi to 4Gi: $ oc patch pvc mypvc -p '{"spec": {"resources": {"requests": {"storage": "4Gi"}}}}' persistentvolumeclaim/mypvc patched 4. Check the pvc status is expanded and MountVolume.NodeExpandVolume succeeded for volume from the log $ oc get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE mypvc Bound pvc-0e27b8cb-ed0e-4291-8b74-a7faf29e7c6d 4Gi RWO standard-csi 22m $ oc describe pvc Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal WaitForPodScheduled 10m persistentvolume-controller waiting for pod mypod to be scheduled Normal Provisioning 10m cinder.csi.openstack.org_wduan-0122a-ccdlk-master-0_3aaa58ac-9c3a-4390-af9d-bc38c21ca0ae External provisioner is provisioning volume for claim "wduan-resize/mypvc" Normal ExternalProvisioning 10m persistentvolume-controller waiting for a volume to be created, either by external provisioner "cinder.csi.openstack.org" or manually created by system administrator Normal ProvisioningSucceeded 10m cinder.csi.openstack.org_wduan-0122a-ccdlk-master-0_3aaa58ac-9c3a-4390-af9d-bc38c21ca0ae Successfully provisioned volume pvc-0e27b8cb-ed0e-4291-8b74-a7faf29e7c6d Warning ExternalExpanding <invalid> volume_expand Ignoring the PVC: didn't find a plugin capable of expanding the volume; waiting for an external controller to process this PVC. Normal Resizing <invalid> external-resizer cinder.csi.openstack.org External resizer is resizing volume pvc-0e27b8cb-ed0e-4291-8b74-a7faf29e7c6d Normal FileSystemResizeRequired <invalid> external-resizer cinder.csi.openstack.org Require file system resize of volume on node Normal FileSystemResizeSuccessful <invalid> kubelet, wduan-0122a-ccdlk-worker-0-z8z56 MountVolume.NodeExpandVolume succeeded for volume "pvc-0e27b8cb-ed0e-4291-8b74-a7faf29e7c6d" 5. Check in pod the volume is 4Gi but the filesysyem is still 2Gi: $ oc rsh mypod # lsblk | grep vdb vdb 252:16 0 4G 0 disk /mnt/local # df -h | grep vdb /dev/vdb 2.0G 6.0M 1.9G 1% /mnt/local Actual results: Filesystem did not be expanded. Expected results: Filesystem should be expanded. Master Log: Node Log (of failed PODs): PV Dump: PVC Dump: StorageClass Dump (if StorageClass used by PV/PVC): Additional info: