Bug 2170689 - Extension of preallocated COW disks is broken
Summary: Extension of preallocated COW disks is broken
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm
Version: 4.5.3
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ovirt-4.5.3-async
: ---
Assignee: Albert Esteve
QA Contact: Shir Fishbain
URL:
Whiteboard:
Depends On:
Blocks: 2210036
TreeView+ depends on / blocked
 
Reported: 2023-02-16 22:40 UTC by Germano Veit Michel
Modified: 2023-05-25 14:32 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Previously, when requesting an extension on a cow format disk, the Logical Volume was not resized. As a result, the Virtual Size on the extended preallocated cow disk was greater than the Actual size. In this release, Actual Size and Virtual Size are equal after the extension of cow preallocated volumes.
Clone Of:
: 2210036 (view as bug list)
Environment:
Last Closed: 2023-03-28 19:48:38 UTC
oVirt Team: Storage
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github oVirt vdsm pull 379 0 None open volume: allow extend cow preallocated 2023-02-20 14:01:56 UTC
Github oVirt vdsm pull 380 0 None open volume: allow extend cow preallocated 2023-02-21 13:47:14 UTC
Red Hat Knowledge Base (Solution) 6999137 0 None None None 2023-02-20 19:53:56 UTC
Red Hat Product Errata RHBA-2023:1501 0 None None None 2023-03-28 19:48:39 UTC

Description Germano Veit Michel 2023-02-16 22:40:17 UTC
Description of problem:

With incremental enabled, a preallocated disk is not RAW, it is COW format.

However the logic to extend these disks is just going through the normal 'COW/Thin' extension code, not resizing the LV

Version-Release number of selected component (if applicable):
4.5.3

How reproducible:
100%

Steps to Reproduce:
1. Use block storage
2. Create a 10G disk, preallocated with incremental enabled.
3. This is the state:

   image:    66f05d6b-a932-4a52-bad9-2c4514a6e076

             - 0fd9eebb-2e57-4f7b-9d19-04c8c73ce4c5
               status: OK, voltype: LEAF, format: COW, legality: LEGAL, type: PREALLOCATED, capacity: 10737418240, truesize: 10737418240

3. Extend the disk to 15G in the UI

4. This is the state. Note only capacity increased

   image:    66f05d6b-a932-4a52-bad9-2c4514a6e076

             - 0fd9eebb-2e57-4f7b-9d19-04c8c73ce4c5
               status: OK, voltype: LEAF, format: COW, legality: LEGAL, type: PREALLOCATED, capacity: 16106127360, truesize: 10737418240

The LV is still 10G after the extension:

  0fd9eebb-2e57-4f7b-9d19-04c8c73ce4c5 ae235e7c-bccc-4499-93ea-2ab0648e9fe0 -wi-------  10.00g                                                    


Actual results:
* Capacity is now 15G
* LV size is still 10G

Expected results:
* Capacity is now 15G
* LV size is also 15G

Additional info:

2023-02-17 08:03:46,816+1000 INFO  (jsonrpc/2) [vdsm.api] START extendVolumeSize(spUUID='bae9dac4-a68f-11ed-a0ba-022734000005', sdUUID='ae235e7c-bccc-4499-93ea-2ab0648e9fe0', imgUUID='66f05d6b-a932-4a52-bad9-2c4514a6e076', volUUID='0fd9eebb-2e57-4f7b-9d19-04c8c73ce4c5', newSize='16106127360') from=::ffff:192.168.2.147,35050, flow_id=2ec8b717-cb7e-4f18-92a0-b89d2afa25a8, task_id=3fbdadf6-dff6-4384-a542-82c7284a3ed1 (api:31)


Volume Extension on COW does nothing on the LV, it just stops here and return, doing nothing on the LV size:

lib/vdsm/storage/volume.py

    def extendSize(self, new_capacity):
        ....
        if volFormat == sc.COW_FORMAT:
            self.log.debug("skipping cow size extension for volume %s to "
                           "capacity %s", self.volUUID, new_capacity)
            return

Comment 1 Arik 2023-02-17 10:45:49 UTC
yeah, we could argue it's a feature request to extend disks on block storage for which incremental backup is enabled but we now enable incremental backup for all disks (from the UI).. Albert, isn't it just a matter of changing the assumption that for QCOW volumes we don't need to extend the LVs so lvm.extendLV would be called for QCOW+preallocated on block storage?

Comment 2 Germano Veit Michel 2023-02-19 20:54:02 UTC
Yes, please consider fixing this, and not as an RFE.
It was picked up by the discrepancy tool actually, but one of the main issues is that it can make VMs with presumably preallocated disks pause, as those disks now new extensions to reach their full size.

There is a customer behind this, Bimal could you please attach your case here?

Comment 3 Albert Esteve 2023-02-20 13:59:11 UTC
Yes, iinm this is just a matter of relaxing the condition in the `extendSize` method.

I was able to reproduce the issue in a test and posted a PR with the fix. I will verify the fix with my local setup too.
That and an OST run should give us some fail-safe in case there is any non-expected side effect.

Comment 4 Casper (RHV QE bot) 2023-02-20 14:30:39 UTC
This bug has low overall severity and is not going to be further verified by QE.

Comment 6 Arik 2023-02-21 08:33:23 UTC
missing a backport to the 4.5.3.z branch

Comment 7 Albert Esteve 2023-02-27 16:36:23 UTC
> one of the main issues is that it can make VMs with presumably preallocated disks pause, as those disks now new extensions to reach their full size

We discussed this, and it should not be a big issue, as we extend fast and early. If the VM writes beyond the LV size but within its max capacity,
we get a block threshold event from libvirt, and the disk is extended by one chunk. Same as for thin disks.

Regardless, we reworked the skip condition so that all preallocated volumes do get extended, to match user expectations. Therefore, this should not happen anymore.

Comment 8 Germano Veit Michel 2023-02-28 21:09:42 UTC
(In reply to Albert Esteve from comment #7)
> > one of the main issues is that it can make VMs with presumably preallocated disks pause, as those disks now new extensions to reach their full size
> 
> We discussed this, and it should not be a big issue, as we extend fast and
> early. If the VM writes beyond the LV size but within its max capacity,
> we get a block threshold event from libvirt, and the disk is extended by one
> chunk. Same as for thin disks.

Indeed, not a big issue in the latest versions with the improve extension mechanism.
 
> Regardless, we reworked the skip condition so that all preallocated volumes
> do get extended, to match user expectations. Therefore, this should not
> happen anymore.

Thank you. It's good to keep preallocated disks as preallocated.

Comment 18 Shir Fishbain 2023-03-27 14:53:00 UTC
Verified 
LV size and capacity size are the same after extension. 


# lvs 30ccf7d1-884b-4cb3-9062-35373cbfc2ec/3d7c0233-e12d-418d-923f-5d9567453352 --devicesfile ""
  LV                                   VG                                   Attr       LSize  Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  3d7c0233-e12d-418d-923f-5d9567453352 30ccf7d1-884b-4cb3-9062-35373cbfc2ec -wi-XX--X- 15.00g

# lvdisplay 30ccf7d1-884b-4cb3-9062-35373cbfc2ec/3d7c0233-e12d-418d-923f-5d9567453352 --devicesfile ""
  --- Logical volume ---
  LV Path                /dev/30ccf7d1-884b-4cb3-9062-35373cbfc2ec/3d7c0233-e12d-418d-923f-5d9567453352
  LV Name                3d7c0233-e12d-418d-923f-5d9567453352
  VG Name                30ccf7d1-884b-4cb3-9062-35373cbfc2ec
  LV UUID                AR413I-5Ml3-rOsi-p8BN-QDrw-p0TS-EnSLck
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                15.00 GiB
  Current LE             120
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     8192
  Block device           253:40

version:
vdsm-4.50.3.7-1.el8ev.x86_64

Comment 20 errata-xmlrpc 2023-03-28 19:48:38 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (vdsm bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:1501


Note You need to log in before you can comment on or make changes to this bug.