Description of problem: LUKS passphrase secrets that are a multiple of 16 characters in length result in incorrectly padded encrypted base64 data being passed to QEMU. - Create a domain for the test : qemu-img create -f raw /var/lib/libvirt/images/test.img 1G virt-install --import --name luks-attach-test --ram 1024 \ --vcpus 1 --cpu host -w network=default,model=virtio \ --disk path=/var/lib/libvirt/images/test.img,format=raw \ --noautoconsole - The following uses a 64 character length passphrase : PASS=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 64 | head -n 1) - Format an example LUKS disk using qemu-img : openssl rand -base64 16 > sec0-iv.b64 printf "$PASS" | openssl enc -aes-256-cbc -a -K $(cat /var/lib/libvirt/qemu/domain-6-luks-attach-test/master-key.aes | hexdump -v -e '/1 "%02X"') -iv $(base64 -d sec0-iv.b64 | hexdump -v -e '/1 "%02X"') > luks-sec.b64 qemu-img create --object secret,id=secmaster,format=raw,file=/var/lib/libvirt/qemu/domain-6-luks-attach-test/master-key.aes \ --object secret,id=sec0,keyid=secmaster,format=base64,file=luks-sec.b64,iv=$(<sec0-iv.b64) \ -f luks -o key-secret=sec0 /var/lib/libvirt/images/test.luks 1G - Create the required secrect and disk XML : cat >> sec.xml << EOF <secret ephemeral="no" private="no"> <usage type="volume"> <volume>var/lib/libvirt/images/test.luks</volume> </usage> </secret> EOF virsh secret-define sec.xml Secret 1ba9af5d-45a5-4ca5-8a0c-07f4fc46984c created cat >> disk.xml << EOF <disk> <driver name="qemu" type="raw"/> <source file="/var/lib/libvirt/images/test.luks"/> <target bus="virtio" dev="vdb"/> <encryption format="luks"> <secret type="passphrase" uuid="1ba9af5d-45a5-4ca5-8a0c-07f4fc46984c"/> </encryption> </disk> EOF - Set the value of the secret within libvirt : MYSECRET=$(printf %s "${PASS}" | base64 -w 0) virsh secret-set-value 1ba9af5d-45a5-4ca5-8a0c-07f4fc46984c ${MYSECRET} - Attempt to attach the device : virsh attach-device luks-attach-test disk.xml error: Failed to attach device from disk.xml error: internal error: unable to execute QEMU command 'object-add': Incorrect number of padding bytes (89) found on decrypted data - The following trace is seen within libvirtd.log with DEBUG enabled : 2017-05-19 10:31:12.567+0000: 3394: debug : qemuDomainObjEnterMonitorInternal:3176 : Entering monitor (mon=0x7f381400f1f0 vm=0x7f382000d510 name=luks-attach-test) 2017-05-19 10:31:12.567+0000: 3394: debug : qemuMonitorAddObject:3060 : type=secret objalias=virtio-disk1-luks-secret0 props=0x7f38300071f0 2017-05-19 10:31:12.567+0000: 3394: debug : qemuMonitorAddObject:3062 : mon:0x7f381400f1f0 vm:0x7f382000d510 json:1 fd:28 2017-05-19 10:31:12.567+0000: 3394: debug : qemuMonitorJSONCommandWithFd:296 : Send command '{"execute":"object-add","arguments":{"qom-type":"secret","id":"virtio-disk1-luks-secret0","props":{"data":"3fTvuaZivMOFN3BjSTdZtIW/sA7mwFVyJ1MpauZrcMJlfDCGzP19KCZA7MiQMKvMHs88QJstgweqwf9aulfqdQ==","keyid":"masterKey0","iv":"RMRlju/G5E9qu1/2/5ctKQ==","format":"base64"}},"id":"libvirt-20"}' for write with FD -1 2017-05-19 10:31:12.567+0000: 3394: info : qemuMonitorSend:1009 : QEMU_MONITOR_SEND_MSG: mon=0x7f381400f1f0 msg={"execute":"object-add","arguments":{"qom-type":"secret","id":"virtio-disk1-luks-secret0","props":{"data":"3fTvuaZivMOFN3BjSTdZtIW/sA7mwFVyJ1MpauZrcMJlfDCGzP19KCZA7MiQMKvMHs88QJstgweqwf9aulfqdQ==","keyid":"masterKey0","iv":"RMRlju/G5E9qu1/2/5ctKQ==","format":"base64"}},"id":"libvirt-20"}^M fd=-1 2017-05-19 10:31:12.568+0000: 3391: info : qemuMonitorIOWrite:534 : QEMU_MONITOR_IO_WRITE: mon=0x7f381400f1f0 buf={"execute":"object-add","arguments":{"qom-type":"secret","id":"virtio-disk1-luks-secret0","props":{"data":"3fTvuaZivMOFN3BjSTdZtIW/sA7mwFVyJ1MpauZrcMJlfDCGzP19KCZA7MiQMKvMHs88QJstgweqwf9aulfqdQ==","keyid":"masterKey0","iv":"RMRlju/G5E9qu1/2/5ctKQ==","format":"base64"}},"id":"libvirt-20"}^M len=290 ret=290 errno=0 2017-05-19 10:31:12.568+0000: 3391: info : qemuMonitorIOProcess:429 : QEMU_MONITOR_IO_PROCESS: mon=0x7f381400f1f0 buf={"id": "libvirt-20", "error": {"class": "GenericError", "desc": "Incorrect number of padding bytes (89) found on decrypted data"}}^M len=132 2017-05-19 10:31:12.568+0000: 3391: debug : qemuMonitorJSONIOProcessLine:191 : Line [{"id": "libvirt-20", "error": {"class": "GenericError", "desc": "Incorrect number of padding bytes (89) found on decrypted data"}}] 2017-05-19 10:31:12.568+0000: 3391: info : qemuMonitorJSONIOProcessLine:211 : QEMU_MONITOR_RECV_REPLY: mon=0x7f381400f1f0 reply={"id": "libvirt-20", "error": {"class": "GenericError", "desc": "Incorrect number of padding bytes (89) found on decrypted data"}} 2017-05-19 10:31:12.568+0000: 3391: debug : qemuMonitorJSONIOProcess:260 : Total used 132 bytes out of 132 available in buffer 2017-05-19 10:31:12.568+0000: 3394: debug : qemuMonitorJSONCommandWithFd:301 : Receive command reply ret=0 rxObject=0x55569cb433f0 2017-05-19 10:31:12.568+0000: 3394: debug : qemuMonitorJSONCheckError:376 : unable to execute QEMU command {"execute":"object-add","arguments":{"qom-type":"secret","id":"virtio-disk1-luks-secret0","props":{"data":"3fTvuaZivMOFN3BjSTdZtIW/sA7mwFVyJ1MpauZrcMJlfDCGzP19KCZA7MiQMKvMHs88QJstgweqwf9aulfqdQ==","keyid":"masterKey0","iv":"RMRlju/G5E9qu1/2/5ctKQ==","format":"base64"}},"id":"libvirt-20"}: {"id":"libvirt-20","error":{"class":"GenericError","desc":"Incorrect number of padding bytes (89) found on decrypted data"}} 2017-05-19 10:31:12.568+0000: 3394: error : qemuMonitorJSONCheckError:387 : internal error: unable to execute QEMU command 'object-add': Incorrect number of padding bytes (89) found on decrypted data 2017-05-19 10:31:12.568+0000: 3394: debug : qemuDomainObjExitMonitorInternal:3199 : Exited monitor (mon=0x7f381400f1f0 vm=0x7f382000d510 name=luks-attach-test) - This is reproducible using the same values via monitor commands directly with QEMU : $QEMU -object secret,id=secmaster,format=raw,file=/var/lib/libvirt/qemu/domain-6-luks-attach-test/master-key.aes -nographic -serial mon:stdio Ctrl+A-c QEMU 2.7.1 monitor - type 'help' for more information (qemu) object_add secret,id=sec0,keyid=secmaster,format=base64,data=3fTvuaZivMOFN3BjSTdZtIW/sA7mwFVyJ1MpauZrcMJlfDCGzP19KCZA7MiQMKvMHs88QJstgweqwf9aulfqdQ==,iv=RMRlju/G5E9qu1/2/5ctKQ== Incorrect number of padding bytes (89) found on decrypted data - However manually generating the encrypted base64 data for the secret again with openssl produces a different valid result : printf "$PASS" | openssl enc -aes-256-cbc -a -K $(cat /var/lib/libvirt/qemu/domain-6-luks-attach-test/master-key.aes | hexdump -v -e '/1 "%02X"') -iv $(echo "RMRlju/G5E9qu1/2/5ctKQ==" | base64 -d | hexdump -v -e '/1 "%02X"') 3fTvuaZivMOFN3BjSTdZtIW/sA7mwFVyJ1MpauZrcMJlfDCGzP19KCZA7MiQMKvMHs88QJstgweqwf9aulfqdaJgS46ZIEm9HmMvxZiTgxw= Libvirt - 3fTvuaZivMOFN3BjSTdZtIW/sA7mwFVyJ1MpauZrcMJlfDCGzP19KCZA7MiQMKvMHs88QJstgweqwf9aulfqdQ== openssl - 3fTvuaZivMOFN3BjSTdZtIW/sA7mwFVyJ1MpauZrcMJlfDCGzP19KCZA7MiQMKvMHs88QJstgweqwf9aulfqdaJgS46ZIEm9HmMvxZiTgxw= - Using this value things now work as expected, we can also attach the LUKS device : $QEMU -object secret,id=secmaster,format=raw,file=/var/lib/libvirt/qemu/domain-6-luks-attach-test/master-key.aes -nographic -serial mon:stdio Ctrl+A-c QEMU 2.7.1 monitor - type 'help' for more information (qemu) object_add secret,id=sec0,keyid=secmaster,format=base64,data=3fTvuaZivMOFN3BjSTdZtIW/sA7mwFVyJ1MpauZrcMJlfDCGzP19KCZA7MiQMKvMHs88QJstgweqwf9aulfqdaJgS46ZIEm9HmMvxZiTgxw=,iv=RMRlju/G5E9qu1/2/5ctKQ== (qemu) drive_add dummy file=/var/lib/libvirt/images/test.luks,key-secret=sec0,format=luks,if=none,id=drive0 OK Version-Release number of selected component (if applicable): libvirt-2.2.0-2.fc25.x86_64 How reproducible: Always Steps to Reproduce: See above. Actual results: Secret object fails to be added due to incorrect padding within the encrypted base64 data. Expected results: Secret object added successfully allowing a LUKS device to be decrypted correctly. Additional info:
commit 71890992daf37ec78b00b4ce873369421dc99731 Author: Daniel P. Berrange <berrange> Date: Tue May 2 11:32:43 2017 +0100 Fix padding of encrypted data If we are encoding a block of data that is 16 bytes in length, we cannot leave it as 16 bytes, we must pad it out to the next block boundary, 32 bytes. Without this padding, the decoder will incorrectly treat the last byte of plain text as the padding length, as it can't distinguish padded from non-padded data. The problem exhibited itself when using a 16 byte passphrase for a LUKS volume $ virsh secret-set-value 55806c7d-8e93-456f-829b-607d8c198367 \ $(echo -n 1234567812345678 | base64) Secret value set $ virsh start demo error: Failed to start domain demo error: internal error: process exited while connecting to monitor: >>>>>>>>>>Len 16 2017-05-02T10:35:40.016390Z qemu-system-x86_64: -object \ secret,id=virtio-disk1-luks-secret0,data=SEtNi5vDUeyseMKHwc1c1Q==,\ keyid=masterKey0,iv=zm7apUB1A6dPcH53VW960Q==,format=base64: \ Incorrect number of padding bytes (56) found on decrypted data Notice how the padding '56' corresponds to the ordinal value of the character '8'. Signed-off-by: Daniel P. Berrange <berrange>
libvirt-2.2.1-2.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2017-ad6a31ebe1
libvirt-2.2.1-2.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-ad6a31ebe1
libvirt-2.2.1-2.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.