RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1371892 - vol-info get incorrect size info for luks encrypted volume
Summary: vol-info get incorrect size info for luks encrypted volume
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.4
Hardware: x86_64
OS: Linux
low
low
Target Milestone: rc
: ---
Assignee: John Ferlan
QA Contact: lijuan men
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-08-31 11:40 UTC by yisun
Modified: 2018-04-10 10:36 UTC (History)
5 users (show)

Fixed In Version: libvirt-3.7.0-1.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-04-10 10:36:45 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2018:0704 0 normal SHIPPED_LIVE libvirt bug fix and enhancement update 2018-04-10 15:57:35 UTC

Description yisun 2016-08-31 11:40:05 UTC
decription:
vol-info get incorrect size info for luks encrypted volume

versions:
libvirt-2.0.0-6.el7.x86_64
qemu-kvm-rhev-2.6.0-22.el7.x86_64
qemu-img-rhev-2.6.0-22.el7.x86_64

How reproducible:
100%

Steps:
1. prepare a libvirt secret
# virsh secret-list
 UUID                                  Usage
--------------------------------------------------------------------------------
...
 4ff78f8c-6ee6-4a8d-b638-2b59d5d49279  volume /var/lib/libvirt/images/luks/luks_5.img

2. create a luks encrypted vm with size=1M

# cat luks_5.vol 
<volume>
        <name>luks_5.img</name>
        <capacity unit='M'>1</capacity>
        <target>
          <path>/var/lib/libvirt/images/luks/luks_5.img</path>
          <format type='raw'/>
          <encryption format='luks'>
             <secret type='passphrase' uuid='4ff78f8c-6ee6-4a8d-b638-2b59d5d49279'/>
             <cipher name='aes' size='256' mode='cbc' hash='sha256'/>
             <ivgen name='plain64' hash='sha256'/>
          </encryption>        
</target>
</volume>

# virsh vol-create luks luks_5.vol 
Vol luks_5.img created from luks_5.vol

3. use vol-info to get the volume's info
# virsh vol-info /var/lib/libvirt/images/luks/luks_5.img  --bytes
Name:           luks_5.img
Type:           file
Capacity:       2101248 bytes   <====== the size is not 1M
Allocation:     135168 bytes 

4. use "qemu-img info"
# qemu-img info /var/lib/libvirt/images/luks/luks_5.img
image: /var/lib/libvirt/images/luks/luks_5.img
file format: luks
virtual size: 1.0M (1048576 bytes)  <======= this one is correct
disk size: 132K
encrypted: yes


Expected result:
vol-info should return correct vol's capacity size, we should now treat the luks encryption image as raw file, since "qemu-img info" can get correct capacity. 

Actual result:
It returns the image's file size.

Comment 1 yisun 2017-03-20 04:13:20 UTC
qemu-kvm-rhev-2.8.0-6.el7.x86_64
libvirt-3.1.0-2.el7.x86_64

hmm, currently with latest libvirt and qemu-kvm-rhev. Behaviours changed. 
Libvirt and qemu seem use different ways to calculate virtual disk size.
** created vol with qemu, and check size with virsh ** or
** created vol with virsh, and check size with qemu **
libvirt will give incorrect results, as follow:

1. create vol with qemu and check with virsh.
# qemu-img create -f luks --object secret,id=sec0,data=`printf %s "redhat" | base64`,format=base64 -o key-secret=sec0 /var/lib/libvirt/images/luks/qemu_create.luks 10M   **10M here**


## qemu-img info /var/lib/libvirt/images/luks/qemu_create.luks
image: /var/lib/libvirt/images/luks/qemu_create.luks
file format: luks
virtual size: 10M (10485760 bytes) **qemu-img info has correct size**



## virsh vol-info /var/lib/libvirt/images/luks/qemu_create.luks --bytes
Name:           qemu_create.luks
Type:           file
Capacity:       12554240 bytes   **here, the virsh vol-info gives a incorrect size**
Allocation:     262144 bytes


2. create vol with virsh vol-create
## cat vol.xml 
<volume>
        <name>luks_1.img</name>
        <capacity unit='M'>10</capacity>
        <target>
          <path>/var/lib/libvirt/images/luks/luks_1.img</path>
          <format type='raw'/>
          <encryption format='luks'>
             <secret type='passphrase' uuid='f981dd17-143f-45bc-88e6-ed1fe20ce9da'/>
          </encryption>        
         </target>
</volume>


## virsh vol-create luks vol.xml
virsh pool-Vol luks_1.img created from vol.xml


## virsh vol-info /var/lib/libvirt/images/luks/luks_1.img
Name:           luks_1.img
Type:           file
Capacity:       11.97 MiB   **not 10m as xml defined**
Allocation:     256.00 KiB

## qemu-img info /var/lib/libvirt/images/luks/luks_1.img 
image: /var/lib/libvirt/images/luks/luks_1.img
file format: luks
virtual size: 10M (10485760 bytes) **qemu gives right size**
disk size: 256K
encrypted: yes

Comment 2 John Ferlan 2017-03-24 14:59:12 UTC
Typically libvirt would use a stat() return of 'st_size' as the capacity of a raw file (such as what LUKS file is described as).  For qcow* encrypted files, further "peeking" would alter the capacity value based up the encrypted volumes view of the logical size - IOW: a metadata capacity is provided. For a LUKS volume there isn't a metadata capacity provided, instead the LUKS metadata provides the value of the "payload offset" - IOW: the size of the header for LUKS metadata. That value is provided as a number of 512 byte blocks that needs to be subtracted from the calculated capacity.


Posted a patch upstream:

http://www.redhat.com/archives/libvir-list/2017-March/msg01178.html

Comment 3 John Ferlan 2017-04-03 20:27:27 UTC
Patch has been pushed upstream

commit b7d44f450c06803df7df3ad380f7a5c97425c1e6
Author: John Ferlan <jferlan>
Date:   Fri Mar 24 09:26:17 2017 -0400

    storage: Fix capacity value for LUKS encrypted volumes
...
    The 'capacity' value (e.g. guest logical size) for a LUKS volume is
    smaller than the 'physical' value of the file in the file system, so
    we need to account for that.
    
    When peeking at the encryption information about the volume add a fetch
    of the payload_offset which is described as the offset to the start of
    the volume data (in 512 byte sectors) in QEMU's QCryptoBlockLUKSHeader.
    
    Then adjust the ->capacity appropriately when we determine that the
    volume target encryption has a payload_offset value.


$ git describe b7d44f450c06803df7df3ad380f7a5c97425c1e6
v3.2.0-21-gb7d44f4
$

Comment 6 lijuan men 2017-06-15 08:02:42 UTC
testing summary:the vol-info is still not the same as 'qemu-img info'

version:
libvirt-3.2.0-10.el7.x86_64
qemu-kvm-rhev-2.9.0-10.el7.x86_64

steps:
1.create vol with 'virsh vol-create '

[root@lmen1 ~]# virsh secret-list 
 UUID                                  Usage
--------------------------------------------------------------------------------
 d48bf526-62c7-4a8b-88d6-ba8ad6286073  iscsi libvirtiscsi
 f981dd17-143f-45bc-88e6-ed1fe20ce9da  volume /var/lib/libvirt/images/luk.img


[root@lmen1 ~]# cat luk.vol 
<volume>
        <name>luk.img</name>
        <capacity unit='M'>1</capacity>
        <target>
          <path>/var/lib/libvirt/images/luk.img</path>
          <format type='raw'/>
          <encryption format='luks'>
             <secret type='passphrase' uuid='f981dd17-143f-45bc-88e6-ed1fe20ce9da'/>
             <cipher name='aes' size='256' mode='cbc' hash='sha256'/>
             <ivgen name='plain64' hash='sha256'/>
          </encryption>        
</target>
</volume>

[root@lmen1 ~]# virsh vol-create default luk.vol 
Vol luk.img created from luk.vol


2.check the size of the volume

[root@lmen1 ~]# virsh vol-info /var/lib/libvirt/images/luk.img
Name:           luk.img
Type:           file
Capacity:       2.00 MiB     --->2M
Allocation:     132.00 KiB


[root@lmen1 ~]# qemu-img info /var/lib/libvirt/images/luk.img
image: /var/lib/libvirt/images/luk.img
file format: luks
virtual size: 1.0M (1048576 bytes)    -->1M
disk size: 132K
encrypted: yes

3.check the size in the guest
start a guest with the disk xml:
 <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/var/lib/libvirt/images/luk.img'/>
      <target dev='sdb' bus='sata'/>
      <encryption format='luks'>
        <secret type='passphrase' uuid='f981dd17-143f-45bc-88e6-ed1fe20ce9da'/>
      </encryption>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>

[root@lmen1 ~]# virsh destroy q35;virsh start q35
Domain q35 destroyed
Domain q35 started

in the guest,
#fdisk -l
  Disk /dev/sdb: 1 MB, 1048576 bytes, 2048 sectors   --->1M
  Units = sectors of 1 * 512 = 512 bytes
  Sector size (logical/physical): 512 bytes / 512 bytes
  I/O size (minimum/optimal): 512 bytes / 512 bytes

So,I want to know,why the result of 'virsh vol-info' is different from 'qemu-img info' and the info in the guest.

PS:
If I create a 20M volume using 'virsh vol-create',the result of 'virsh vol-info' is 21M,'qemu-img info' is 20M,the info in guest is 20M

Comment 7 John Ferlan 2017-06-15 19:33:50 UTC
I'm curious too, because I did test this before throwing it over the wall.

So I've added some debug code to the two places I changed:

virStorageSourceUpdateCapacity:
...
    if (src->encryption && src->encryption->payload_offset != -1) {
        VIR_WARN("src cap=%llu offset=%d",
                  src->capacity, src->encryption->payload_offset);
        src->capacity -= src->encryption->payload_offset * 512;
        VIR_WARN("src cap=%llu", src->capacity);
    }
...

storageBackendProbeTarget:
...
    if (encryption && meta->encryption) {
        VIR_WARN("encryption");
        if (meta->encryption->payload_offset != -1) {
            VIR_WARN("target cap=%llu offset=%d",
                     target->capacity, meta->encryption->payload_offset);
            target->capacity -= meta->encryption->payload_offset * 512;
            VIR_WARN("target cap=%llu", target->capacity);
        }
...


Then run...

# virsh vol-create default luk.xml
...
2017-06-15 15:30:51.240+0000: 16910: warning : virStorageSourceUpdateCapacity:3460 : src cap=2101248 offset=0
2017-06-15 15:30:51.240+0000: 16910: warning : virStorageSourceUpdateCapacity:3462 : src cap=2101248
...
Vol luk.img created from luk.xml

#  virsh vol-create default luk.xml
...
2017-06-15 15:30:55.619+0000: 16914: warning : virStorageSourceUpdateCapacity:3460 : src cap=2101248 offset=0
2017-06-15 15:30:55.619+0000: 16914: warning : virStorageSourceUpdateCapacity:3462 : src cap=2101248
...

Name:           luk.img
Type:           file
Capacity:       2101248 bytes
Allocation:     135168 bytes

#

< very strange, but interestingly enough... and I have more than one disk here, but the end result is the curious part >

# virsh pool-refresh default
...
2017-06-15 15:32:50.481+0000: 16912: warning : virStorageFileGetEncryptionPayloadOffset:953 : payload_offset=2056
2017-06-15 15:32:50.481+0000: 16912: warning : storageBackendProbeTarget:3489 : encryption
2017-06-15 15:32:50.481+0000: 16912: warning : storageBackendProbeTarget:3492 : target cap=22024192 offset=2056
2017-06-15 15:32:50.481+0000: 16912: warning : storageBackendProbeTarget:3494 : target cap=20971520
2017-06-15 15:32:50.481+0000: 16912: warning : virStorageFileGetEncryptionPayloadOffset:953 : payload_offset=2056
2017-06-15 15:32:50.481+0000: 16912: warning : storageBackendProbeTarget:3489 : encryption
2017-06-15 15:32:50.481+0000: 16912: warning : storageBackendProbeTarget:3492 : target cap=2101248 offset=2056
2017-06-15 15:32:50.481+0000: 16912: warning : storageBackendProbeTarget:3494 : target cap=1048576
2017-06-15 15:32:50.481+0000: 16912: warning : virStorageFileGetEncryptionPayloadOffset:953 : payload_offset=4040
2017-06-15 15:32:50.481+0000: 16912: warning : storageBackendProbeTarget:3489 : encryption
2017-06-15 15:32:50.481+0000: 16912: warning : storageBackendProbeTarget:3492 : target cap=23040000 offset=4040
2017-06-15 15:32:50.481+0000: 16912: warning : storageBackendProbeTarget:3494 : target cap=20971520
2017-06-15 15:32:50.481+0000: 16912: warning : virStorageFileGetEncryptionPayloadOffset:953 : payload_offset=4040
2017-06-15 15:32:50.481+0000: 16912: warning : storageBackendProbeTarget:3489 : encryption
2017-06-15 15:32:50.481+0000: 16912: warning : storageBackendProbeTarget:3492 : target cap=1075810304 offset=4040
2017-06-15 15:32:50.481+0000: 16912: warning : storageBackendProbeTarget:3494 : target cap=1073741824
2017-06-15 15:32:50.481+0000: 16912: warning : virStorageFileGetEncryptionPayloadOffset:953 : payload_offset=-1
2017-06-15 15:32:50.481+0000: 16912: warning : storageBackendProbeTarget:3489 : encryption
...
Pool default refreshed

# virsh vol-info /home/vm-images/luk.img --bytes 
...
2017-06-15 15:35:25.573+0000: 16913: warning : virStorageSourceUpdateCapacity:3460 : src cap=2101248 offset=2056
2017-06-15 15:35:25.573+0000: 16913: warning : virStorageSourceUpdateCapacity:3462 : src cap=1048576
...
Name:           luk.img
Type:           file
Capacity:       1048576 bytes
Allocation:     135168 bytes

#

Well that's the right answer!

So what change (needle in the haystack) caused the difference? Or did I perhaps run my tests using a debug session second and because of that causing the refresh to magically happen.

So short term, there is a work-around - just refresh the pool...

After a bit digging - it seems it is possible to fetch the correct value for the create, build, refresh volume code path. Typically for a RAW defined volume we'd just assign physical to capacity; however, for a RAW and Encrypted volume we can do better.

I've proposed a patch upstream:

https://www.redhat.com/archives/libvir-list/2017-June/msg00676.html

I didn't (yet) set this back to assigned - it's where we are in the release cycle that causes me to pause before ensuring I can get a quick approval and then backport appropriately (since there is a workaround of sorts).

Comment 8 lijuan men 2017-06-22 08:33:40 UTC
do you have the schedule when to put the patch into libvirt pkg?
Will it be too late in 7.4 phase? If it is too late,it is suggested to move this bug into 7.5

Comment 9 John Ferlan 2017-06-22 10:16:00 UTC
At this point we need special flags for 7.4, but I see other patches still getting included so who knows. All we can do is ask/try.  Still there is a work around at least w/r/t checking directly after creation (to do the refresh). Restarts, reboots, finding the volume in a pool "later" would also show the right size. It's just the case of checking the size after create before any sort of refresh occurs.

Comment 10 Xuesong Zhang 2017-06-22 10:35:37 UTC
(In reply to John Ferlan from comment #9)
> At this point we need special flags for 7.4, but I see other patches still
> getting included so who knows. All we can do is ask/try.  Still there is a
> work around at least w/r/t checking directly after creation (to do the
> refresh). Restarts, reboots, finding the volume in a pool "later" would also
> show the right size. It's just the case of checking the size after create
> before any sort of refresh occurs.

Move back to Assign per above comment 6 and 7, and move to RHEL7.5, since it seems the patch in comment 7 is still in reviewing phase till now, can not be backport to the downstream in current release cycle.

Comment 11 John Ferlan 2017-06-24 10:47:27 UTC
Pushed upstream:

commit 5431055d2b833463b25269cdb486dd08ef790097
Author: John Ferlan <jferlan>
Date:   Thu Jun 15 14:32:43 2017 -0400

    util: Force reading of meta data to get encryption capacity value
    
...

    As it turns out the volume create, build, and refresh path was not peeking
    at the meta data, so immediately after a create operation the value displayed
    for capacity was still incorrect. However, if a pool refresh was done the
    correct value was fetched as a result of a meta data peek.
    
    The reason is it seems historically if the file type is RAW then peeking
    at the file just took the physical value for the capacity. However, since
    we know if it's an encrypted file, then peeking at the meta data will be
    required in order to get a true capacity value.
    
    So check for encryption in the source and if present, use the meta data
    in order to fill in the capacity value and set the payload_offset.

...

$ git describe 5431055d2b833463b25269cdb486dd08ef790097
v3.4.0-168-g5431055
$

Comment 14 lijuan men 2017-12-06 09:04:24 UTC
verify the bug

version:
libvirt-3.9.0-5.el7.x86_64
qemu-kvm-rhev-2.10.0-11.el7.x86_64


scenario1: create 1M vol with virsh vol-create

[root@localhost ~]# cat luks.xml 
<secret ephemeral='no' private='yes'>
  <uuid>f981dd17-143f-45bc-88e6-ed1fe20ce9da</uuid>
  <description>LUKS Sample Secret</description>
  <usage type='volume'>
    <volume>/var/libvirt/images/luks.img</volume>
  </usage>
</secret>

[root@localhost ~]# virsh secret-define luks.xml 
Secret f981dd17-143f-45bc-88e6-ed1fe20ce9da created


[root@localhost ~]# virsh secret-list
 UUID                                  Usage
--------------------------------------------------------------------------------
 a1d92516-1f84-4be8-9cf8-d84efbfdd4b1  iscsi libvirtiscsi
 f981dd17-143f-45bc-88e6-ed1fe20ce9da  volume /var/libvirt/images/luks.img

[root@localhost ~]#  MYSECRET=`echo "redhat" | base64`
[root@localhost ~]# virsh secret-set-value f981dd17-143f-45bc-88e6-ed1fe20ce9da $MYSECRET
Secret value set


[root@localhost ~]# cat luks-vol.xml 
<volume>
        <name>luks.img</name>
        <capacity unit='M'>1</capacity>
        <target>
          <path>/var/lib/libvirt/images/luks.img</path>
          <format type='raw'/>
          <encryption format='luks'>
             <secret type='passphrase' uuid='f981dd17-143f-45bc-88e6-ed1fe20ce9da'/>
             <cipher name='aes' size='256' mode='cbc' hash='sha256'/>
             <ivgen name='plain64' hash='sha256'/>
          </encryption>        
</target>
</volume>


[root@localhost ~]# virsh vol-create default luks-vol.xml
Vol luks.img created from luks-vol.xml


[root@localhost ~]# virsh vol-info /var/lib/libvirt/images/luks.img --bytes
Name:           luks.img
Type:           file
Capacity:       1048576 bytes
Allocation:     135168 bytes


[root@localhost ~]# qemu-img info /var/lib/libvirt/images/luks.img
image: /var/lib/libvirt/images/luks.img
file format: luks
virtual size: 1.0M (1048576 bytes)
disk size: 132K



scenario2:create 10M vol with virsh vol-create

[root@localhost ~]# cat luks-vol1.xml
<volume>
        <name>luks1.img</name>
        <capacity unit='M'>10</capacity>
        <target>
          <path>/var/lib/libvirt/images/luks1.img</path>
          <format type='raw'/>
          <encryption format='luks'>
             <secret type='passphrase' uuid='f981dd17-143f-45bc-88e6-ed1fe20ce9da'/>
             <cipher name='aes' size='256' mode='cbc' hash='sha256'/>
             <ivgen name='plain64' hash='sha256'/>
          </encryption>        
</target>
</volume>


[root@localhost ~]# virsh vol-create default luks-vol1.xml
Vol luks1.img created from luks-vol1.xml


[root@localhost ~]# virsh vol-info /var/lib/libvirt/images/luks1.img --bytes
Name:           luks1.img
Type:           file
Capacity:       10485760 bytes
Allocation:     135168 bytes

[root@localhost ~]# qemu-img info /var/lib/libvirt/images/luks1.img
image: /var/lib/libvirt/images/luks1.img
file format: luks
virtual size: 10M (10485760 bytes)
disk size: 132K



scenario3: create 10M vol with qemu and check with virsh



[root@localhost ~]# qemu-img create -f luks --object secret,id=sec0,data=`printf %s "redhat" | base64`,format=base64 -o key-secret=sec0 /var/lib/libvirt/images/luks2.img 10M
Formatting '/var/lib/libvirt/images/luks2.img', fmt=luks size=10485760 key-secret=sec0


[root@localhost ~]# qemu-img info /var/lib/libvirt/images/luks2.img
image: /var/lib/libvirt/images/luks2.img
file format: luks
virtual size: 10M (10485760 bytes)
disk size: 256K

[root@localhost ~]# virsh pool-refresh default
Pool default refreshed


[root@localhost ~]# virsh vol-info /var/lib/libvirt/images/luks2.img 
Name:           luks2.img
Type:           file
Capacity:       10.00 MiB
Allocation:     256.00 KiB

in the above 3 scenarios,I also check the disks' size in the guest,the size is right.

Comment 18 errata-xmlrpc 2018-04-10 10:36:45 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:0704


Note You need to log in before you can comment on or make changes to this bug.