Bug 1534951
Summary: | RFE: Support preallocation mode for luks format | |||
---|---|---|---|---|
Product: | Red Hat Enterprise Linux Advanced Virtualization | Reporter: | Ping Li <pingl> | |
Component: | qemu-kvm | Assignee: | Maxim Levitsky <mlevitsk> | |
Status: | CLOSED ERRATA | QA Contact: | Tingting Mao <timao> | |
Severity: | medium | Docs Contact: | ||
Priority: | medium | |||
Version: | --- | CC: | areis, berrange, chayang, coli, ddepaula, juzhang, knoel, lolyu, mlevitsk, mtessun, ngu, rbalakri, timao, virt-maint, yihyu | |
Target Milestone: | rc | Keywords: | FutureFeature | |
Target Release: | --- | |||
Hardware: | Unspecified | |||
OS: | Unspecified | |||
Whiteboard: | ||||
Fixed In Version: | qemu-kvm-4.1.0-18.module+el8.1.1+5150+45ce6c40 | Doc Type: | Enhancement | |
Doc Text: | Story Points: | --- | ||
Clone Of: | ||||
: | 1594864 (view as bug list) | Environment: | ||
Last Closed: | 2020-02-04 18:28:48 UTC | Type: | Feature Request | |
Regression: | --- | Mount Type: | --- | |
Documentation: | --- | CRM: | ||
Verified Versions: | Category: | --- | ||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
Cloudforms Team: | --- | Target Upstream Version: | ||
Embargoed: | ||||
Bug Depends On: | ||||
Bug Blocks: | 1594864 |
Description
Ping Li
2018-01-16 10:48:47 UTC
(In reply to Ping Li from comment #0) > Description of problem: > Preallocation mode should be supported by luks format. > > Version-Release number of selected component (if applicable): > qemu-kvm-rhev-2.10.0-16.el7 > > How reproducible: > 100% > > Steps to Reproduce: > 1. Check the support options by create luks image > # qemu-img create -f luks -o ? > Supported options: > size Virtual disk size > key-secret ID of the secret that provides the keyslot passphrase > cipher-alg Name of encryption cipher algorithm > cipher-mode Name of encryption cipher mode > ivgen-alg Name of IV generator algorithm > ivgen-hash-alg Name of IV generator hash algorithm > hash-alg Name of encryption hash algorithm > iter-time Time to spend in PBKDF in milliseconds > > 2. Create luks image with full mode > # qemu-img create -f luks --object secret,id=sec0,data=base -o > key-secret=sec0 base.luks 1G -o preallocation=full > Formatting 'base.luks', fmt=luks size=1073741824 key-secret=sec0 > preallocation=full > qemu-img: base.luks: Parameter 'preallocation' is unexpected > > 2. Create luks image with falloc mode > # qemu-img create -f luks --object secret,id=sec0,data=base -o > key-secret=sec0 base.luks 1G -o preallocation=falloc > Formatting 'base.luks', fmt=luks size=1073741824 key-secret=sec0 > preallocation=falloc > qemu-img: base.luks: Parameter 'preallocation' is unexpected > > 3. Create luks image with off mode > # qemu-img create -f luks --object secret,id=sec0,data=base -o > key-secret=sec0 -o preallocation=full test.luks 1G > Formatting 'test.luks', fmt=luks size=1073741824 key-secret=sec0 > preallocation=full > qemu-img: test.luks: Parameter 'preallocation' is unexpected > All these work in current upstream, so can you please retest with 7.6 packages? (In reply to Ping Li from comment #0) > Description of problem: > Preallocation mode should be supported by luks format. > > Version-Release number of selected component (if applicable): > qemu-kvm-rhev-2.10.0-16.el7 > > How reproducible: > 100% > > Steps to Reproduce: > 1. Check the support options by create luks image > # qemu-img create -f luks -o ? > Supported options: > size Virtual disk size > key-secret ID of the secret that provides the keyslot passphrase > cipher-alg Name of encryption cipher algorithm > cipher-mode Name of encryption cipher mode > ivgen-alg Name of IV generator algorithm > ivgen-hash-alg Name of IV generator hash algorithm > hash-alg Name of encryption hash algorithm > iter-time Time to spend in PBKDF in milliseconds > Even though preallocation options do work in current upstream, -o ? is still missing them. Daniel, what do you think? There's no support for preallocation with the luks driver at this time, but if using JSON syntax you can still use preallocation with the layer underneath the luks format. It would be reasonable to add support for prealloc with luks though. (In reply to Daniel Berrange from comment #4) > There's no support for preallocation with the luks driver at this time, but > if using JSON syntax you can still use preallocation with the layer > underneath the luks format. > Indeed... The commands are working now (they were originally reported as broken, see comment #0), but the image is not preallocated: $ qemu-img create -f luks --object secret,id=sec0,data=base -o key-secret=sec0 base.luks 1G -o preallocation=full Formatting 'base.luks', fmt=luks size=1073741824 key-secret=sec0 preallocation=full [ademar@optimus ~]$ qemu-img info base.luks | head -n 5 image: base.luks file format: luks virtual size: 1.0G (1073741824 bytes) disk size: 256K encrypted: yes $ qemu-img --version qemu-img version 2.12.0 (qemu-2.12.0-1.fc27) Odd, QEMU should reject any attempt to use them if they're not registered as valid options in help, so I wonder what changed. After re-tested the scenarios on RHEL-7.6, qemu-img could create images with preallocation mode, but the space is not allocated. Packages tested: qemu-kvm-rhev-2.12.0-4.el7 kernel-3.10.0-862.el7 Test steps: 1. Check the support options by create luks image # qemu-img create -f luks -o ? Supported options: size Virtual disk size key-secret ID of the secret that provides the keyslot passphrase cipher-alg Name of encryption cipher algorithm cipher-mode Name of encryption cipher mode ivgen-alg Name of IV generator algorithm ivgen-hash-alg Name of IV generator hash algorithm hash-alg Name of encryption hash algorithm iter-time Time to spend in PBKDF in milliseconds 2. Create luks image with full mode # qemu-img create -f luks --object secret,id=sec0,data=base -o key-secret=sec0 full.luks 1G -o preallocation=full Formatting 'full.luks', fmt=luks size=1073741824 key-secret=sec0 preallocation=full # qemu-img info full.luks | head -4 image: full.luks file format: luks virtual size: 1.0G (1073741824 bytes) disk size: 256K 2. Create luks image with falloc mode # qemu-img create -f luks --object secret,id=sec0,data=base -o key-secret=sec0 falloc.luks 1G -o preallocation=falloc Formatting 'falloc.luks', fmt=luks size=1073741824 key-secret=sec0 preallocation=falloc # qemu-img info falloc.luks | head -4 image: falloc.luks file format: luks virtual size: 1.0G (1073741824 bytes) disk size: 256K 3. Create luks image with off mode # qemu-img create -f luks --object secret,id=sec0,data=base -o key-secret=sec0 off.luks 1G -o preallocation=off Formatting 'off.luks', fmt=luks size=1073741824 key-secret=sec0 preallocation=off # qemu-img info off.luks | head -4 image: off.luks file format: luks virtual size: 1.0G (1073741824 bytes) disk size: 256K (In reply to Daniel Berrange from comment #4) > There's no support for preallocation with the luks driver at this time, but > if using JSON syntax you can still use preallocation with the layer Json syntax only works for existing files such as specifying the backing file. Mostly, we need it for network protocols. I think it can't be used when creating an image. Could you help to share the detail information about using JSON syntax to allocate a new image? > underneath the luks format. > > It would be reasonable to add support for prealloc with luks though. In the meanwhile, I opened a simpler BZ to handle the parameter correctly (return error if -o preallocation is used with -f luks), see Bug 1594864. The error reporting regression arrived in this change: commit 1ec4f4160a1a94cf3d13b43551fff2792bd5056e Author: Kevin Wolf <kwolf> Date: Fri Mar 2 14:16:36 2018 +0100 luks: Create block_crypto_co_create_generic() Everything that refers to the protocol layer or QemuOpts is moved out of block_crypto_create_generic(), so that the remaining function is suitable to be called by a .bdrv_co_create implementation. LUKS is the only driver that actually implements the old interface, and we don't intend to use it in any new drivers, so put the moved out code directly into a LUKS function rather than creating a generic intermediate one. Signed-off-by: Kevin Wolf <kwolf> Reviewed-by: Daniel P. Berrangé <berrange> Reviewed-by: Eric Blake <eblake> Before this change, we complained if 'preallocation' flag was requested with luks. After this change it is silently ignored. Patch posted upstream https://lists.gnu.org/archive/html/qemu-devel/2019-07/msg02521.html V2 of the patch posted: https://lists.nongnu.org/archive/html/qemu-block/2019-07/msg00475.html Tested this issue with the latest qemu packages. Tested with: qemu-kvm-4.0.0-5.module+el8.1.0+3622+5812d9bf Steps: 1. Check the supported options --------------------- No preallocation option! # qemu-img create -f luks -o ? Supported options: cipher-alg=<str> - Name of encryption cipher algorithm cipher-mode=<str> - Name of encryption cipher mode hash-alg=<str> - Name of encryption hash algorithm iter-time=<num> - Time to spend in PBKDF in milliseconds ivgen-alg=<str> - Name of IV generator algorithm ivgen-hash-alg=<str> - Name of IV generator hash algorithm key-secret=<str> - ID of the secret that provides the keyslot passphrase size=<size> - Virtual disk size 2. Create with preallocation=full # time qemu-img create -f luks --object secret,id=sec0,data=base -o key-secret=sec0 base.luks 1G -o preallocation=full Formatting 'base.luks', fmt=luks size=1073741824 key-secret=sec0 preallocation=full real 0m4.395s user 0m4.327s sys 0m0.026s # qemu-img info base.luks image: base.luks file format: luks virtual size: 1.0G (1073741824 bytes) disk size: 256K ------------------------------------ No preallocation! encrypted: yes Format specific information: ivgen alg: plain64 hash alg: sha256 cipher alg: aes-256 uuid: dd3cd723-2c59-4968-a42a-45044db4084e cipher mode: xts slots: [0]: active: true iters: 1147740 key offset: 4096 stripes: 4000 [1]: active: false key offset: 262144 [2]: active: false key offset: 520192 [3]: active: false key offset: 778240 [4]: active: false key offset: 1036288 [5]: active: false key offset: 1294336 [6]: active: false key offset: 1552384 [7]: active: false key offset: 1810432 payload offset: 2068480 master key iters: 286720 3. Create with preallocation=falloc # time qemu-img create -f luks --object secret,id=sec0,data=base -o key-secret=sec0 base.luks 1G -o preallocation=falloc Formatting 'base.luks', fmt=luks size=1073741824 key-secret=sec0 preallocation=falloc real 0m4.473s user 0m4.327s sys 0m0.017s # qemu-img info base.luks image: base.luks file format: luks virtual size: 1.0G (1073741824 bytes) disk size: 256K -------------------------------------- No preallocation! encrypted: yes Format specific information: ivgen alg: plain64 hash alg: sha256 cipher alg: aes-256 uuid: 67d2216e-eb1f-461d-90e9-39279aa9ef8c cipher mode: xts slots: [0]: active: true iters: 1145734 key offset: 4096 stripes: 4000 [1]: active: false key offset: 262144 [2]: active: false key offset: 520192 [3]: active: false key offset: 778240 [4]: active: false key offset: 1036288 [5]: active: false key offset: 1294336 [6]: active: false key offset: 1552384 [7]: active: false key offset: 1810432 payload offset: 2068480 master key iters: 286720 3. Create with preallocation=off # time qemu-img create -f luks --object secret,id=sec0,data=base -o key-secret=sec0 base.luks 1G -o preallocation=off Formatting 'base.luks', fmt=luks size=1073741824 key-secret=sec0 preallocation=off real 0m4.481s user 0m4.319s sys 0m0.025s # qemu-img info base.luks image: base.luks file format: luks virtual size: 1.0G (1073741824 bytes) disk size: 256K encrypted: yes Format specific information: ivgen alg: plain64 hash alg: sha256 cipher alg: aes-256 uuid: 09005187-bdfb-46f3-8d46-88f62c400d89 cipher mode: xts slots: [0]: active: true iters: 1145734 key offset: 4096 stripes: 4000 [1]: active: false key offset: 262144 [2]: active: false key offset: 520192 [3]: active: false key offset: 778240 [4]: active: false key offset: 1036288 [5]: active: false key offset: 1294336 [6]: active: false key offset: 1552384 [7]: active: false key offset: 1810432 payload offset: 2068480 master key iters: 286720 *** Bug 1594864 has been marked as a duplicate of this bug. *** *** Bug 1651872 has been marked as a duplicate of this bug. *** v5 was posted and queued for upstream qemu 4.2: https://lists.nongnu.org/archive/html/qemu-devel/2019-07/msg03870.html Commit pushed to qemu-4.2: https://git.qemu.org/?p=qemu.git;a=commit;h=672de729a1f93d84e7597652b1125ab5d62421d8 Verified this bug as below. Tested with: qemu-kvm-4.1.0-18.module+el8.1.1+5150+45ce6c40 kernel-4.18.0-147.0.3.el8_1 Steps: * Check the supported options. # qemu-img create -f luks -o ? test.luks Supported options: cipher-alg=<str> - Name of encryption cipher algorithm cipher-mode=<str> - Name of encryption cipher mode hash-alg=<str> - Name of encryption hash algorithm iter-time=<num> - Time to spend in PBKDF in milliseconds ivgen-alg=<str> - Name of IV generator algorithm ivgen-hash-alg=<str> - Name of IV generator hash algorithm key-secret=<str> - ID of the secret that provides the keyslot passphrase nocow=<bool (on/off)> - Turn off copy-on-write (valid only on btrfs) preallocation=<str> - Preallocation mode (allowed values: off, falloc, full) ----------------- There is. size=<size> - Virtual disk size Scenario1 (Create with preallocation=off) 1. Created # qemu-img create -f luks --object secret,id=sec0,data=base -o key-secret=sec0,preallocation=off test.luks 5G 2. Check the image info. # qemu-img info test.luks image: test.luks file format: luks virtual size: 5 GiB (5368709120 bytes) disk size: 256 KiB -------------------------- No allocated. encrypted: yes Format specific information: ivgen alg: plain64 hash alg: sha256 cipher alg: aes-256 uuid: 4d98f079-77b3-4786-bcc6-d88f0248a914 cipher mode: xts slots: [0]: active: true iters: 989968 key offset: 4096 stripes: 4000 [1]: active: false key offset: 262144 [2]: active: false key offset: 520192 [3]: active: false key offset: 778240 [4]: active: false key offset: 1036288 [5]: active: false key offset: 1294336 [6]: active: false key offset: 1552384 [7]: active: false key offset: 1810432 payload offset: 2068480 master key iters: 209961 # ls -lash test.luks 256K -rw-r--r--. 1 root root 5.1G Dec 11 03:30 test.luks Scenario2 (Create with preallocation=falloc) 1. Created # strace -e trace=fallocate -f qemu-img create -f luks --object secret,id=sec0,data=base -o key-secret=sec0,preallocation=falloc test.luks 5G strace: Process 2313 attached Formatting 'test.luks', fmt=luks size=5368709120 key-secret=sec0 preallocation=falloc strace: Process 2314 attached [pid 2314] fallocate(9, 0, 0, 5370777600) = 0 ----------------------------- Invoked fallocate syscall. [pid 2313] +++ exited with 0 +++ [pid 2314] +++ exited with 0 +++ +++ exited with 0 +++ 2. Check image info. # qemu-img info test.luks image: test.luks file format: luks virtual size: 5 GiB (5368709120 bytes) disk size: 5 GiB encrypted: yes Format specific information: ivgen alg: plain64 hash alg: sha256 cipher alg: aes-256 uuid: 215ca94a-b295-489b-80cf-ffbd05ab60d1 cipher mode: xts slots: [0]: active: true iters: 1005152 key offset: 4096 stripes: 4000 [1]: active: false key offset: 262144 [2]: active: false key offset: 520192 [3]: active: false key offset: 778240 [4]: active: false key offset: 1036288 [5]: active: false key offset: 1294336 [6]: active: false key offset: 1552384 [7]: active: false key offset: 1810432 payload offset: 2068480 master key iters: 209659 # ls -alsh test.luks 5.1G -rw-r--r--. 1 root root 5.1G Dec 11 03:33 test.luks Scenario3 (Create with preallocation=full) 1. Created # qemu-img create -f luks --object secret,id=sec0,data=base -o key-secret=sec0,preallocation=full test.luks 5G 2. Check the image info. # qemu-img info test.luks image: test.luks file format: luks virtual size: 5 GiB (5368709120 bytes) disk size: 5 GiB -------------------------- Fully allocated. encrypted: yes Format specific information: ivgen alg: plain64 hash alg: sha256 cipher alg: aes-256 uuid: 6da46fd8-fd30-4776-a7d5-3b4e4c5ea462 cipher mode: xts slots: [0]: active: true iters: 1048576 key offset: 4096 stripes: 4000 [1]: active: false key offset: 262144 [2]: active: false key offset: 520192 [3]: active: false key offset: 778240 [4]: active: false key offset: 1036288 [5]: active: false key offset: 1294336 [6]: active: false key offset: 1552384 [7]: active: false key offset: 1810432 payload offset: 2068480 master key iters: 209676 # ls -lash test.luks 5.1G -rw-r--r--. 1 root root 5.1G Dec 11 03:35 test.luks Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0404 |