Bug 1534951 - RFE: Support preallocation mode for luks format
Summary: RFE: Support preallocation mode for luks format
Keywords:
Status: ASSIGNED
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: qemu-kvm
Version: ---
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: Maxim Levitsky
QA Contact: Tingting Mao
URL:
Whiteboard:
: 1594864 1651872 (view as bug list)
Depends On:
Blocks: 1594864
TreeView+ depends on / blocked
 
Reported: 2018-01-16 10:48 UTC by Ping Li
Modified: 2019-08-23 18:36 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Clone Of:
: 1594864 (view as bug list)
Environment:
Last Closed:
Type: Feature Request


Attachments (Terms of Use)

Description Ping Li 2018-01-16 10:48:47 UTC
Description of problem:
Preallocation mode should be supported by luks format.

Version-Release number of selected component (if applicable):
qemu-kvm-rhev-2.10.0-16.el7

How reproducible:
100%

Steps to Reproduce:
1. Check the support options by create luks image
# qemu-img create -f luks -o ?
Supported options:
size             Virtual disk size
key-secret       ID of the secret that provides the keyslot passphrase
cipher-alg       Name of encryption cipher algorithm
cipher-mode      Name of encryption cipher mode
ivgen-alg        Name of IV generator algorithm
ivgen-hash-alg   Name of IV generator hash algorithm
hash-alg         Name of encryption hash algorithm
iter-time        Time to spend in PBKDF in milliseconds

2. Create luks image with full mode
# qemu-img create -f luks --object secret,id=sec0,data=base -o key-secret=sec0 base.luks 1G -o preallocation=full
Formatting 'base.luks', fmt=luks size=1073741824 key-secret=sec0 preallocation=full
qemu-img: base.luks: Parameter 'preallocation' is unexpected

2. Create luks image with falloc mode
# qemu-img create -f luks --object secret,id=sec0,data=base -o key-secret=sec0 base.luks 1G -o preallocation=falloc
Formatting 'base.luks', fmt=luks size=1073741824 key-secret=sec0 preallocation=falloc
qemu-img: base.luks: Parameter 'preallocation' is unexpected

3. Create luks image with off mode
# qemu-img create -f luks --object secret,id=sec0,data=base -o key-secret=sec0 -o preallocation=full test.luks 1G
Formatting 'test.luks', fmt=luks size=1073741824 key-secret=sec0 preallocation=full
qemu-img: test.luks: Parameter 'preallocation' is unexpected

Actual results:
Preallocation is not supported by luks format

Expected results:
Preallocation should be supported by luks format

Additional info:

Comment 2 Ademar Reis 2018-06-21 14:49:15 UTC
(In reply to Ping Li from comment #0)
> Description of problem:
> Preallocation mode should be supported by luks format.
> 
> Version-Release number of selected component (if applicable):
> qemu-kvm-rhev-2.10.0-16.el7
> 
> How reproducible:
> 100%
> 
> Steps to Reproduce:
> 1. Check the support options by create luks image
> # qemu-img create -f luks -o ?
> Supported options:
> size             Virtual disk size
> key-secret       ID of the secret that provides the keyslot passphrase
> cipher-alg       Name of encryption cipher algorithm
> cipher-mode      Name of encryption cipher mode
> ivgen-alg        Name of IV generator algorithm
> ivgen-hash-alg   Name of IV generator hash algorithm
> hash-alg         Name of encryption hash algorithm
> iter-time        Time to spend in PBKDF in milliseconds
> 
> 2. Create luks image with full mode
> # qemu-img create -f luks --object secret,id=sec0,data=base -o
> key-secret=sec0 base.luks 1G -o preallocation=full
> Formatting 'base.luks', fmt=luks size=1073741824 key-secret=sec0
> preallocation=full
> qemu-img: base.luks: Parameter 'preallocation' is unexpected
> 
> 2. Create luks image with falloc mode
> # qemu-img create -f luks --object secret,id=sec0,data=base -o
> key-secret=sec0 base.luks 1G -o preallocation=falloc
> Formatting 'base.luks', fmt=luks size=1073741824 key-secret=sec0
> preallocation=falloc
> qemu-img: base.luks: Parameter 'preallocation' is unexpected
> 
> 3. Create luks image with off mode
> # qemu-img create -f luks --object secret,id=sec0,data=base -o
> key-secret=sec0 -o preallocation=full test.luks 1G
> Formatting 'test.luks', fmt=luks size=1073741824 key-secret=sec0
> preallocation=full
> qemu-img: test.luks: Parameter 'preallocation' is unexpected
> 

All these work in current upstream, so can you please retest with 7.6 packages?

Comment 3 Ademar Reis 2018-06-21 14:52:45 UTC
(In reply to Ping Li from comment #0)
> Description of problem:
> Preallocation mode should be supported by luks format.
> 
> Version-Release number of selected component (if applicable):
> qemu-kvm-rhev-2.10.0-16.el7
> 
> How reproducible:
> 100%
> 
> Steps to Reproduce:
> 1. Check the support options by create luks image
> # qemu-img create -f luks -o ?
> Supported options:
> size             Virtual disk size
> key-secret       ID of the secret that provides the keyslot passphrase
> cipher-alg       Name of encryption cipher algorithm
> cipher-mode      Name of encryption cipher mode
> ivgen-alg        Name of IV generator algorithm
> ivgen-hash-alg   Name of IV generator hash algorithm
> hash-alg         Name of encryption hash algorithm
> iter-time        Time to spend in PBKDF in milliseconds
> 

Even though preallocation options do work in current upstream, -o ? is still missing them. Daniel, what do you think?

Comment 4 Daniel Berrangé 2018-06-21 15:05:25 UTC
There's no support for preallocation with the luks driver at this time, but if using JSON syntax you can still use preallocation with the layer underneath the luks format.

It would be reasonable to add support for prealloc with luks though.

Comment 5 Ademar Reis 2018-06-21 15:45:56 UTC
(In reply to Daniel Berrange from comment #4)
> There's no support for preallocation with the luks driver at this time, but
> if using JSON syntax you can still use preallocation with the layer
> underneath the luks format.
> 

Indeed... The commands are working now (they were originally reported as broken, see comment #0), but the image is not preallocated:

$ qemu-img create -f luks --object secret,id=sec0,data=base -o key-secret=sec0 base.luks 1G -o preallocation=full
Formatting 'base.luks', fmt=luks size=1073741824 key-secret=sec0 preallocation=full

[ademar@optimus ~]$ qemu-img info base.luks | head -n 5
image: base.luks
file format: luks
virtual size: 1.0G (1073741824 bytes)
disk size: 256K
encrypted: yes

$ qemu-img --version
qemu-img version 2.12.0 (qemu-2.12.0-1.fc27)

Comment 6 Daniel Berrangé 2018-06-21 15:58:26 UTC
Odd, QEMU should reject any attempt to use them if they're not registered  as valid options in help, so I wonder what changed.

Comment 8 Ping Li 2018-06-22 03:30:00 UTC
After re-tested the scenarios on RHEL-7.6, qemu-img could create images with preallocation mode, but the space is not allocated.

Packages tested:
qemu-kvm-rhev-2.12.0-4.el7
kernel-3.10.0-862.el7

Test steps:
1. Check the support options by create luks image
# qemu-img create -f luks -o ?
Supported options:
size             Virtual disk size
key-secret       ID of the secret that provides the keyslot passphrase
cipher-alg       Name of encryption cipher algorithm
cipher-mode      Name of encryption cipher mode
ivgen-alg        Name of IV generator algorithm
ivgen-hash-alg   Name of IV generator hash algorithm
hash-alg         Name of encryption hash algorithm
iter-time        Time to spend in PBKDF in milliseconds

2. Create luks image with full mode
# qemu-img create -f luks --object secret,id=sec0,data=base -o key-secret=sec0 full.luks 1G -o preallocation=full
Formatting 'full.luks', fmt=luks size=1073741824 key-secret=sec0 preallocation=full
# qemu-img info full.luks | head -4
image: full.luks
file format: luks
virtual size: 1.0G (1073741824 bytes)
disk size: 256K


2. Create luks image with falloc mode
# qemu-img create -f luks --object secret,id=sec0,data=base -o key-secret=sec0 falloc.luks 1G -o preallocation=falloc
Formatting 'falloc.luks', fmt=luks size=1073741824 key-secret=sec0 preallocation=falloc
# qemu-img info falloc.luks | head -4
image: falloc.luks
file format: luks
virtual size: 1.0G (1073741824 bytes)
disk size: 256K

3. Create luks image with off mode
# qemu-img create -f luks --object secret,id=sec0,data=base -o key-secret=sec0 off.luks 1G -o preallocation=off
Formatting 'off.luks', fmt=luks size=1073741824 key-secret=sec0 preallocation=off
# qemu-img info off.luks | head -4
image: off.luks
file format: luks
virtual size: 1.0G (1073741824 bytes)
disk size: 256K

Comment 9 Ping Li 2018-06-22 03:36:36 UTC
(In reply to Daniel Berrange from comment #4)
> There's no support for preallocation with the luks driver at this time, but
> if using JSON syntax you can still use preallocation with the layer

Json syntax only works for existing files such as specifying the backing file. Mostly, we need it for network protocols. I think it can't be used when creating an image. Could you help to share the detail information about using JSON syntax to allocate a new image?

> underneath the luks format.
> 
> It would be reasonable to add support for prealloc with luks though.

Comment 10 Ademar Reis 2018-06-25 16:50:50 UTC
In the meanwhile, I opened a simpler BZ to handle the parameter correctly (return error if -o preallocation is used with -f luks), see Bug 1594864.

Comment 11 Daniel Berrangé 2019-05-13 14:43:23 UTC
The error reporting regression arrived in this change:

commit 1ec4f4160a1a94cf3d13b43551fff2792bd5056e 
Author: Kevin Wolf <kwolf@redhat.com>
Date:   Fri Mar 2 14:16:36 2018 +0100

    luks: Create block_crypto_co_create_generic()
    
    Everything that refers to the protocol layer or QemuOpts is moved out of
    block_crypto_create_generic(), so that the remaining function is
    suitable to be called by a .bdrv_co_create implementation.
    
    LUKS is the only driver that actually implements the old interface, and
    we don't intend to use it in any new drivers, so put the moved out code
    directly into a LUKS function rather than creating a generic
    intermediate one.
    
    Signed-off-by: Kevin Wolf <kwolf@redhat.com>
    Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
    Reviewed-by: Eric Blake <eblake@redhat.com>


Before this change, we complained if 'preallocation' flag was requested with luks. After this change it is silently ignored.

Comment 12 Maxim Levitsky 2019-07-10 17:17:59 UTC
Patch posted upstream

https://lists.gnu.org/archive/html/qemu-devel/2019-07/msg02521.html

Comment 13 Maxim Levitsky 2019-07-11 09:25:47 UTC
V2 of the patch posted:

https://lists.nongnu.org/archive/html/qemu-block/2019-07/msg00475.html

Comment 14 Tingting Mao 2019-07-11 11:34:46 UTC
Tested this issue with the latest qemu packages.


Tested with:
qemu-kvm-4.0.0-5.module+el8.1.0+3622+5812d9bf


Steps:
1. Check the supported options --------------------- No preallocation option!
# qemu-img create -f luks -o ?
Supported options:
  cipher-alg=<str>       - Name of encryption cipher algorithm
  cipher-mode=<str>      - Name of encryption cipher mode
  hash-alg=<str>         - Name of encryption hash algorithm
  iter-time=<num>        - Time to spend in PBKDF in milliseconds
  ivgen-alg=<str>        - Name of IV generator algorithm
  ivgen-hash-alg=<str>   - Name of IV generator hash algorithm
  key-secret=<str>       - ID of the secret that provides the keyslot passphrase
  size=<size>            - Virtual disk size

2. Create with preallocation=full
# time qemu-img create -f luks --object secret,id=sec0,data=base -o key-secret=sec0 base.luks 1G -o preallocation=full
Formatting 'base.luks', fmt=luks size=1073741824 key-secret=sec0 preallocation=full

real	0m4.395s
user	0m4.327s
sys	0m0.026s
# qemu-img info base.luks 
image: base.luks
file format: luks
virtual size: 1.0G (1073741824 bytes)
disk size: 256K ------------------------------------ No preallocation!
encrypted: yes
Format specific information:
    ivgen alg: plain64
    hash alg: sha256
    cipher alg: aes-256
    uuid: dd3cd723-2c59-4968-a42a-45044db4084e
    cipher mode: xts
    slots:
        [0]:
            active: true
            iters: 1147740
            key offset: 4096
            stripes: 4000
        [1]:
            active: false
            key offset: 262144
        [2]:
            active: false
            key offset: 520192
        [3]:
            active: false
            key offset: 778240
        [4]:
            active: false
            key offset: 1036288
        [5]:
            active: false
            key offset: 1294336
        [6]:
            active: false
            key offset: 1552384
        [7]:
            active: false
            key offset: 1810432
    payload offset: 2068480
    master key iters: 286720

3. Create with preallocation=falloc
# time qemu-img create -f luks --object secret,id=sec0,data=base -o key-secret=sec0 base.luks 1G -o preallocation=falloc
Formatting 'base.luks', fmt=luks size=1073741824 key-secret=sec0 preallocation=falloc

real	0m4.473s
user	0m4.327s
sys	0m0.017s
# qemu-img info base.luks 
image: base.luks
file format: luks
virtual size: 1.0G (1073741824 bytes)
disk size: 256K -------------------------------------- No preallocation!
encrypted: yes
Format specific information:
    ivgen alg: plain64
    hash alg: sha256
    cipher alg: aes-256
    uuid: 67d2216e-eb1f-461d-90e9-39279aa9ef8c
    cipher mode: xts
    slots:
        [0]:
            active: true
            iters: 1145734
            key offset: 4096
            stripes: 4000
        [1]:
            active: false
            key offset: 262144
        [2]:
            active: false
            key offset: 520192
        [3]:
            active: false
            key offset: 778240
        [4]:
            active: false
            key offset: 1036288
        [5]:
            active: false
            key offset: 1294336
        [6]:
            active: false
            key offset: 1552384
        [7]:
            active: false
            key offset: 1810432
    payload offset: 2068480
    master key iters: 286720

3. Create with preallocation=off
# time qemu-img create -f luks --object secret,id=sec0,data=base -o key-secret=sec0 base.luks 1G -o preallocation=off
Formatting 'base.luks', fmt=luks size=1073741824 key-secret=sec0 preallocation=off

real	0m4.481s
user	0m4.319s
sys	0m0.025s
# qemu-img info base.luks 
image: base.luks
file format: luks
virtual size: 1.0G (1073741824 bytes)
disk size: 256K
encrypted: yes
Format specific information:
    ivgen alg: plain64
    hash alg: sha256
    cipher alg: aes-256
    uuid: 09005187-bdfb-46f3-8d46-88f62c400d89
    cipher mode: xts
    slots:
        [0]:
            active: true
            iters: 1145734
            key offset: 4096
            stripes: 4000
        [1]:
            active: false
            key offset: 262144
        [2]:
            active: false
            key offset: 520192
        [3]:
            active: false
            key offset: 778240
        [4]:
            active: false
            key offset: 1036288
        [5]:
            active: false
            key offset: 1294336
        [6]:
            active: false
            key offset: 1552384
        [7]:
            active: false
            key offset: 1810432
    payload offset: 2068480
    master key iters: 286720

Comment 15 Tingting Mao 2019-07-11 11:37:39 UTC
*** Bug 1594864 has been marked as a duplicate of this bug. ***

Comment 16 Tingting Mao 2019-07-16 07:47:47 UTC
*** Bug 1651872 has been marked as a duplicate of this bug. ***

Comment 17 John Ferlan 2019-08-02 13:17:21 UTC
v5 was posted and queued for upstream qemu 4.2:

https://lists.nongnu.org/archive/html/qemu-devel/2019-07/msg03870.html

Comment 18 John Ferlan 2019-08-23 18:36:53 UTC
Commit pushed to qemu-4.2:

https://git.qemu.org/?p=qemu.git;a=commit;h=672de729a1f93d84e7597652b1125ab5d62421d8


Note You need to log in before you can comment on or make changes to this bug.