RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1073305 - resize volume can go beyond the pool capacity
Summary: resize volume can go beyond the pool capacity
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: John Ferlan
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-03-06 07:37 UTC by yanbing du
Modified: 2015-11-19 05:45 UTC (History)
8 users (show)

Fixed In Version: libvirt-1.2.16-1.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-11-19 05:45:01 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:2202 0 normal SHIPPED_LIVE libvirt bug fix and enhancement update 2015-11-19 08:17:58 UTC

Description yanbing du 2014-03-06 07:37:57 UTC
Description of problem:
Resize a volume to reach the pool capacity, then resize it again to a large value(bigger than pool capacity) can succeed. 

Version-Release number of selected component (if applicable):
libvirt-1.1.1-26.el7.x86_64


How reproducible:
100%

Steps to Reproduce:
1.
# virsh pool-info tmp
Name:           tmp
UUID:           5958f480-7452-4343-b2c6-019ba4f181f4
State:          running
Persistent:     yes
Autostart:      no
Capacity:       98.31 GiB
Allocation:     14.08 GiB
Available:      84.23 GiB

# virsh vol-list tmp
Name                 Path
-----------------------------------------
vol1                 /tmp/test2/vol1

# virsh vol-info /tmp/test2/vol1
Name:           vol1
Type:           file
Capacity:       10.00 MiB
Allocation:     10.00 MiB

# virsh vol-resize /tmp/test2/vol1 100G
error: Failed to change size of volume 'vol1' to 100G

error: operation failed: Not enough space left on storage pool

--> except fail

# virsh vol-resize /tmp/test2/vol1 85G
Size of volume 'vol1' successfully changed to 85G

# virsh vol-info /tmp/test2/vol1
Name:           vol1
Type:           file
Capacity:       85.00 GiB
Allocation:     10.00 MiB

# virsh vol-resize /tmp/test2/vol1 100G
Size of volume 'vol1' successfully changed to 100G

--> except fail, but succeed

# virsh vol-resize /tmp/test2/vol1 150G
Size of volume 'vol1' successfully changed to 150G

# virsh vol-info /tmp/test2/vol1
Name:           vol1
Type:           file
Capacity:       150.00 GiB
Allocation:     10.00 MiB 

2.
3.

Actual results:
error: Failed to change size of volume 'vol1' to XX
error: operation failed: Not enough space left on storage pool 

Expected results:
As reproduce steps show. 

Additional info:

Comment 3 John Ferlan 2015-03-27 11:48:14 UTC
ugh!  Seems the resize is adjusting sizes of pool when it shouldn't necessarily be doing that.  The way pools are set up and creation is handled, the "allocation" and "available" are adjusted based on the size of the allocation. Thus the following each results in the same output before and after:

Before:

$ virsh pool-info default
Name:           default
UUID:           462942be-bd42-4b68-b315-e7fa3a4fdebc
State:          running
Persistent:     yes
Autostart:      no
Capacity:       134.45 GiB
Allocation:     62.04 GiB
Available:      72.41 GiB

$ virsh vol-create-as default bz1073305 10G 10G
$ virsh vol-create-as default bz1073305 20G 10G
$ virsh vol-create-as default bz1073305 100G 10G

$ virsh pool-info default
Name:           default
UUID:           462942be-bd42-4b68-b315-e7fa3a4fdebc
State:          running
Persistent:     yes
Autostart:      no
Capacity:       134.45 GiB
Allocation:     72.04 GiB
Available:      62.41 GiB

Theory being you are creating a sparse file and in order to adjust the size, the virsh vol-resize would be called with the --allocate value set (eg virsh vol-resize default 20G --allocate 20G).

The "capacity" value is for the volume.  Perhaps what "may" useful because of the confusion is for the pool to display whether the pool is "overcommitted" if every volume in the pool had it's allocation == capacity, but that's a different issue. Note we cannot change the "available" to be volume capacity based as that would change things in an unexpected manner perhaps for applications which use the available value to determine the size of what they "can" allocate in the pool.

Anyway, unlike the 'Create' option, the 'Resize' option checks whether the incoming capacity value (not allocation value) would result in "enough space" in the pool for the capacity change:

    if (incoming capacity > existing capacity + pool available)
        fail

So for my example, if I resize using 100G as my capacity and not allocation, things will fail because the following check passes:

    if (100 > 10 + 62)

Now, if I successfully change my volume capacity and not allocation to 50G, then retry the same 100 I have:

    if (100 > 50 + 62)

which allows the code to set the volume capacity to 100.

NOTE: Prior to commit id '4f3287a41' (0.9.10), the check was "incoming capacity > existing allocation + available", which perhaps closer to what was necessary, but still not quite right.

The issue being (following the Create logic) it doesn't matter what "capacity" changes to as long as the "allocation" (eg, what really takes space away) doesn't grow beyond the bounds of the pool.

Whether being "overcommitted" with respect to the capacity of any volume(s) in the pool compared to the actual space available and "how" pool volumes perhaps may be allowed to grow past allocation to reach capacity without the libvirt pool logic realizing it (being notified) is a separate issue. That is, if I create a volume with a capacity of 20G and an allocation of 10G, if writes reach 10G, then since it's a sparse volume I still need to check if can we work our way to 20G without having libvirt do a resize that includes a larger allocation...

I do think the fix to this "resize" issue involves an extra check regarding whether the incoming resize includes an allocation change or if it's just changing the allocation allowed value. It also requires a few more words in the vol-create-as and vol-resize virsh man page to describe the "interactions".

Comment 4 John Ferlan 2015-03-27 16:11:42 UTC
Patch posted upstream:

http://www.redhat.com/archives/libvir-list/2015-March/msg01477.html

Comment 5 John Ferlan 2015-04-09 23:14:15 UTC
Patches reviewed and pushed:

git describe 1095230dee8e1bf184248b087695f45f867a4143
v1.2.14-81-g1095230


Lots of detail in the commit message:

commit 1095230dee8e1bf184248b087695f45f867a4143
Author: John Ferlan <jferlan>
Date:   Fri Mar 27 09:48:59 2015 -0400

    storage: Fix issues in storageVolResize
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1073305
    
    When creating a volume in a pool, the creation allows the 'capacity'
    value to be larger than the available space in the pool. As long as
    the 'allocation' value will fit in the space, the volume will be created.
    
    However, resizing the volume checks were made with the new absolute
    capacity value against existing capacity + the available space without
    regard for whether the new absolute capacity was actually allocating
    space or not.  For example, a pool with 75G of available space creates
    a volume of 10G using a capacity of 100G and allocation of 10G will succeed;
    however, if the allocation used a capacity of 10G instead and then tried
    to resize the allocation to 100G the code would fail to allow the backend
    to try the resize.
    
    Furthermore, when updating the pool "available" and "allocation" values,
    the resize code would just "blindly" adjust them regardless of whether
    space was "allocated" or just "capacity" was being adjusted.  This left
    a scenario whereby a resize to 100G would fail; however, a resize to 50G
    followed by one to 100G would both succeed.  Again, neither was adjusting
    the allocation value, just the "capacity" value.
    
    This patch adds more logic to the resize code to understand whether the
    new capacity value is actually "allocating" space as well and whether it
    shrinking or expanding. Since unsigned arithmatic is involved, the possibility
    that we adjust the pool size values incorrectly is probable.
    
    This patch also ensures that updates to the pool values only occur if we
    actually performed the allocation.
    
    NB: The storageVolDelete, storageVolCreateXML, and storageVolCreateXMLFrom
    each only updates the pool allocation/availability values by the target
    volume allocation value.

Comment 7 Ján Tomko 2015-05-28 12:32:09 UTC
The check in the above commit could be bypassed by using --shrink.
That should be fixed by:
commit 7211f66ad71066dc1bf9c428c70d8ba87ab791e9
Author:     Ján Tomko <jtomko>
CommitDate: 2015-05-28 14:10:09 +0200

    Simplify allocation check in storageVolResize
    
    Since shrinking a volume below existing allocation is not allowed,
    it is not possible for a successful resize with VOL_RESIZE_ALLOCATE
    to increase the pool's available value.
    
    Even with the SHRINK flag it is possible to extend the current
    allocation or even the capacity. Remove the overflow when
    computing delta with this flag and do the check even if the
    flag was specified.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1073305

git describe: v1.2.16-rc1-8-g7211f66

Comment 8 yisun 2015-06-23 07:24:23 UTC
Verified:
ON libvirt-1.2.16-1.el7.x86_64

prepare a pool with around 8G capacity

# virsh pool-dumpxml pcap
<pool type='dir'>
  <name>pcap</name>
  <uuid>3a74d746-1551-4b5b-b8c1-a7ec7c5cd274</uuid>
  <capacity unit='bytes'>8320901120</capacity>
  <allocation unit='bytes'>37720064</allocation>
  <available unit='bytes'>8283181056</available>
  <source>
  </source>
  <target>
    <path>/test</path>
    <permissions>
      <mode>0755</mode>
      <owner>0</owner>
      <group>0</group>
      <label>system_u:object_r:unlabeled_t:s0</label>
    </permissions>
  </target>
</pool>

# virsh pool-info pcap
Name:           pcap
UUID:           3a74d746-1551-4b5b-b8c1-a7ec7c5cd274
State:          running
Persistent:     yes
Autostart:      no
Capacity:       7.75 GiB
Allocation:     35.97 MiB
Available:      7.71 GiB


Scenario 1: only vol-resize capability, libvirt should allow the operation no matter how much capability-size provided. And pool's capability, allocation and available should not be changed.


1. # virsh vol-create-as pcap 1.img 1G 1G
Vol 1.img created

2. # virsh pool-refresh pcap; virsh pool-info pcap
Pool pcap refreshed
Name:           pcap
UUID:           3a74d746-1551-4b5b-b8c1-a7ec7c5cd274
State:          running
Persistent:     no
Autostart:      no
Capacity:       7.75 GiB
Allocation:     1.04 GiB
Available:      6.71 GiB

3. # virsh vol-resize /test/1.img 100G
Size of volume '1.img' successfully changed to 100G

4. # virsh vol-info /test/1.img
Name:           1.img
Type:           file
Capacity:       100.00 GiB
Allocation:     1.00 GiB

5. # virsh pool-refresh pcap; virsh pool-info pcap
Pool pcap refreshed

Name:           pcap
UUID:           3a74d746-1551-4b5b-b8c1-a7ec7c5cd274
State:          running
Persistent:     no
Autostart:      no
Capacity:       7.75 GiB
Allocation:     1.04 GiB
Available:      6.71 GiB
<================================ pool info not modified as expect.

 Scenario 2: vol-resize capability with --allocate, libvirt should allow the operation only if vol's newly allocated size is smaller than pool's available size.

1. clear the pool and prepare a vol again
# virsh vol-create-as pcap 1.img 1G 1G
Vol 1.img created

2. # virsh pool-info pcap
Name:           pcap
UUID:           3a74d746-1551-4b5b-b8c1-a7ec7c5cd274
State:          running
Persistent:     no
Autostart:      no
Capacity:       7.75 GiB
Allocation:     1.04 GiB
Available:      6.71 GiB


3. # virsh vol-resize /test/1.img 8G --allocate
error: Failed to change size of volume '1.img' to 8G
error: operation failed: Not enough space left in storage pool <==== failed as expect. since allocation exceeds pool's available space.


4. # virsh vol-resize /test/1.img 7G --allocate
Size of volume '1.img' successfully changed to 7G  <====  successfully when left space is enough

5. # virsh pool-info pcap
Name:           pcap
UUID:           3a74d746-1551-4b5b-b8c1-a7ec7c5cd274
State:          running
Persistent:     yes
Autostart:      no
Capacity:       7.75 GiB
Allocation:     7.04 GiB
Available:      731.46 MiB


Scenario3: test with --shrink but try to extend capacity and allocation
clear the env and create a 1G volume
1. # virsh vol-create-as pcap 1.img 1G 1G
Vol 1.img created

2. # virsh pool-info pcap
Name:           pcap
UUID:           3a74d746-1551-4b5b-b8c1-a7ec7c5cd274
State:          running
Persistent:     yes
Autostart:      no
Capacity:       7.75 GiB
Allocation:     1.04 GiB
Available:      6.71 GiB

3. # virsh vol-resize --allocate --shrink /test/1.img 8G
error: Failed to change size of volume '1.img' to 8G

error: operation failed: Not enough space left in storage pool <====== expected

4. # virsh vol-resize --shrink /test/1.img 18G
Size of volume '1.img' successfully changed to 18G   <==== expected
5. # virsh vol-info /test/1.img
Name:           1.img
Type:           file
Capacity:       18.00 GiB     <=== not limited, as expect
Allocation:     1.00 GiB

Comment 10 errata-xmlrpc 2015-11-19 05:45:01 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2202.html


Note You need to log in before you can comment on or make changes to this bug.