Description of problem: Libvirt currently does not support providing of the secret for decrypting the TLS client key for disks.
This bug is going to be addressed in the next major release.
Fixed upstream: commit ab7b3167b383b2af9bace432c17b5301ce4b8bc2 Author: Peter Krempa <pkrempa> Date: Tue Jun 30 17:08:44 2020 +0200 tests: qemuxml2argv: Test encrypted TLS key for nbd/vxhs disks Add a dummy secret so that we see what command line is generated. Signed-off-by: Peter Krempa <pkrempa> Reviewed-by: Eric Blake <eblake> commit 423711aef266377f1bfc5557ad8a364b0b03a4da Author: Peter Krempa <pkrempa> Date: Tue Jun 30 15:39:41 2020 +0200 qemu: domain: Setup secret for TLS key for nbd/vxhs disks Setup the TLS secret when preparing a virStorageSource for use. https://bugzilla.redhat.com/show_bug.cgi?id=1602328 Signed-off-by: Peter Krempa <pkrempa> Reviewed-by: Eric Blake <eblake> commit 470d322e0cc3d45eccf54c6e4fa13bf1bd40428f Author: Peter Krempa <pkrempa> Date: Mon Jun 29 17:12:03 2020 +0200 qemu: conf: Add configuration of TLS key encryption for 'vxhs' and 'nbd' disks Until now libvirt didn't allow using encrypted TLS key for disk clients. Add fields for configuring the secret and propagate defaults. Signed-off-by: Peter Krempa <pkrempa> Reviewed-by: Eric Blake <eblake> commit 7fa772bfd719ca8cf07ec5482266572fd21d6f5d Author: Peter Krempa <pkrempa> Date: Mon Jun 29 15:11:00 2020 +0200 qemu block: Add internals for handling 'secret' corresponding to TLS key Add infrastructure for hot- and cold-plug of the secret object holding decryption key for the TLS key. Signed-off-by: Peter Krempa <pkrempa> Reviewed-by: Eric Blake <eblake> commit a3cbbc5afb4a9ad447c42776063351274eb2b7a6 Author: Peter Krempa <pkrempa> Date: Mon Jun 29 15:10:42 2020 +0200 qemu: domain: Add infrastructure passing in TLS key's decryption key via 'secret' Store the required data in the private data of a storage source and ensure that the 'alias' of the secret is formatted in the status XML. Signed-off-by: Peter Krempa <pkrempa> Reviewed-by: Eric Blake <eblake> commit cd0dc1c19feab350a235185cdf38d21d6c155216 Author: Peter Krempa <pkrempa> Date: Tue Jun 30 17:56:08 2020 +0200 virQEMUDriverConfigLoadSpecificTLSEntry: Split up fetching of server-only config options The '*_tls_x509_verify' options are relevant only when we are going to expose a server socket as client sockets always enable verification. Split up the macro to separate the common bits from the server bits so that when we'll later extend support of 'nbd' and 'vxhs' disks which are client only we can reuse the existing macros. Signed-off-by: Peter Krempa <pkrempa> Reviewed-by: Eric Blake <eblake> commit 94b5e9ebf62be0292cda9b3c31f2be1e385316cb Author: Peter Krempa <pkrempa> Date: Tue Jun 30 17:51:50 2020 +0200 virQEMUDriverConfigLoadSpecificTLSEntry: Move fetching of 'chardev_tls' above macro Move the extraction of the config value so that it makes more sense after upcoming refactors. Signed-off-by: Peter Krempa <pkrempa> Reviewed-by: Eric Blake <eblake> commit 473b97abac5f4cc898f397be12a8fbcc0302a36a Author: Peter Krempa <pkrempa> Date: Mon Jun 29 17:05:35 2020 +0200 qemu: conf: Move 'nbd' and 'vxhs' tls config variables together with rest of tls setup Signed-off-by: Peter Krempa <pkrempa> Reviewed-by: Eric Blake <eblake> commit 72fdba7ffd82e7f4c44ea239d001c51a54fb6285 Author: Peter Krempa <pkrempa> Date: Mon Jun 29 12:05:03 2020 +0200 qemu.conf: Remove misleading mention of 'migrate_tls' There's no such parameter. Reword the sentence to account for enabling TLS-encrypted migration using API flags. Signed-off-by: Peter Krempa <pkrempa> Reviewed-by: Eric Blake <eblake> commit 7bfb85cf1d138c4cee92a776bcda267468304a43 Author: Peter Krempa <pkrempa> Date: Mon Jun 29 16:00:03 2020 +0200 qemuDomainDiskHasEncryptionSecret: unexport Signed-off-by: Peter Krempa <pkrempa> Reviewed-by: Eric Blake <eblake> commit 47be72571915e3b920dce316a8375987c8c44223 Author: Peter Krempa <pkrempa> Date: Fri May 22 12:29:10 2020 +0200 qemu: domain: Introduce helper for always fetching virStorageSource private data Add a helper which will always return the storage source private data even if it was not allocated before. Signed-off-by: Peter Krempa <pkrempa> Reviewed-by: Eric Blake <eblake>
Test on v6.5.0-234-gc4cd0ef917 qemu-kvm-core-5.0.0-2.module+el8.3.0+7379+0505d6ca.x86_64 but FAILED: Libvirt compiling options: ../configure --without-libssh --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-qemu --without-openvz --without-lxc --without-vbox --without-libxl --with-sasl --with-polkit --with-libvirtd --without-phyp --with-esx --without-hyperv --without-vmware --without-xenapi --without-vz --without-bhyve --without-interface --with-network --with-storage-fs --with-storage-lvm --with-storage-iscsi --with-storage-iscsi-direct --with-storage-scsi --with-storage-disk --with-storage-mpath --without-storage-sheepdog --with-storage-gluster --without-storage-zfs --without-storage-vstorage --with-numactl --with-numad --with-capng --without-fuse --without-netcf --with-selinux --without-apparmor --without-hal --with-udev --with-yajl --without-sanlock --with-libpcap --with-macvtap --with-audit --with-dtrace --with-driver-modules --with-firewalld --with-firewalld-zone --without-wireshark-dissector --without-pm-utils --with-nss-plugin '--with-packager=Unknown, 2019-08-19-12:13:01, lab.rhel8.me' --with-packager-version=1.el8 --with-qemu-user=qemu --with-qemu-group=qemu --with-tls-priority=@LIBVIRT,SYSTEM --enable-werror --enable-expensive-tests --with-init-script=systemd --without-login-shell Setup: 1. Generate keys and certs of CA, server, client. For client keys, encrypted with passphrase "redhat" 2. Copy all keys and certs to /etc/pki/libvirt-nbd/ 3. Prepare a image /tmp/test. Prepare the nbd tls server: # qemu-nbd --object tls-creds-x509,id=tls0,endpoint=server,dir=/etc/pki/libvirt-nbd /tmp/test -x new -t -v Steps: 1. Prepare /etc/libvirt/qemu.conf and /etc/libvirt/libvirtd.conf : qemu.conf: nbd_tls = 1 nbd_tls_x509_cert_dir = "/etc/pki/libvirt-nbd" max_core = "unlimited" nbd_tls_x509_secret_uuid = "567e84ae-e792-4a74-b22d-f01445281862" libvirtd.conf: log_level = 3 log_filters = "1:qemu 4:object 4:json" log_outputs = "1:file:/tmp/libvirtd.log" 2. Start libvirtd and virtlogd # LD_LIBRARY_PATH="./src/.libs" ./src/.libs/libvirtd # LD_LIBRARY_PATH="./src/.libs" ./src/.libs/virtlogd 3. Define a libvirt tls secret with uuid 567e84ae-e792-4a74-b22d-f01445281862 Set the password to "redhat": # virsh secret-set-value 567e84ae-e792-4a74-b22d-f01445281862 `printf redhat|base64` 4. Start a vm with nbd tls disk Disk xml: <disk type='network' device='disk'> <driver name='qemu' type='raw'/> <source protocol='nbd' name='new' tls='yes'> <host name='XXXX' port='10809'/> </source> <target dev='sdb' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> # virsh start fedora31 error: Failed to start domain fedora31 error: internal error: process exited while connecting to monitor: 2020-07-21T07:57:33.126804Z qemu-kvm: -blockdev {"driver":"nbd","server":{"type":"inet","host":"XXXX","port":"10809"},"export":"new","tls-creds":"objlibvirt-1-storage_tls0","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}: Denied by server for option 5 (starttls) server reported: TLS not configured However the qemu-img info with passphrase object works: # qemu-img info --object secret,id=sec,data=redhat,format=raw --object tls-creds-x509,id=tls0,endpoint=client,dir=/etc/pki/libvirt-nbd/,verify-peer=yes,passwordid=sec nbd://XXXX/new image: nbd://XXXX:10809/new file format: raw virtual size: 100 MiB (104857600 bytes) disk size: unavailable The qemu log: -chardev spicevmc,id=charredir1,name=usbredir \ -device usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3 \ -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 \ -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -msg timestamp=on 2020-07-21 07:54:21.749+0000: Domain id=2 is tainted: host-cpu char device redirected to /dev/pts/23 (label charserial0) 2020-07-21T07:54:21.916845Z qemu-kvm: -blockdev {"driver":"nbd","server":{"type":"inet","host":"XXXX","port":"10809"},"export":"new","tls-creds":"objlibvirt-1-storage_tls0","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}: Denied by server for option 5 (starttls) server reported: TLS not configured 2020-07-21 07:54:21.969+0000: shutting down, reason=failed 2020-07-21 07:57:32.953+0000: starting up libvirt version: 6.6.0, package: 1.el8 (Unknown, 2019-08-19-12:13:01, lab.rhel8.me), qemu version: 5.0.50qemu-kvm-5.0.0-2.module+el8.3.0+7379+0505d6ca, kernel: 4.18.0-221.el8.x86_64, hostname: XXXX LC_ALL=C \ LD_LIBRARY_PATH=./src/.libs \ PATH=/root/.gem/ruby/2.4.0/bin:/root/.cargo/bin:/root/.local/bin:/root/.gem/ruby/2.4.0/bin:/root/.cargo/bin:/root/.local/bin:/usr/share/Modules/bin:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin:/root/bin \ HOME=/var/lib/libvirt/qemu/domain-3-fedora31 \ USER=root \ LOGNAME=root \ XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-3-fedora31/.local/share \ XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-3-fedora31/.cache \ XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-3-fedora31/.config \ QEMU_AUDIO_DRV=spice \ /usr/libexec/qemu-kvm \ -name guest=fedora31,debug-threads=on \ -S \ -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-fedora31/master-key.aes \ -machine pc-i440fx-rhel7.6.0,accel=kvm,usb=off,vmport=off,dump-guest-core=off \ -cpu host,migratable=on \ -m 1024 \ -overcommit mem-lock=off \ -smp 1,sockets=1,cores=1,threads=1 \ -uuid 9e49557d-7326-4cfc-8d37-6207bb17e7f5 \ -no-user-config \ -nodefaults \ -chardev socket,id=charmonitor,fd=29,server,nowait \ -mon chardev=charmonitor,id=monitor,mode=control \ -rtc base=utc,driftfix=slew \ -global kvm-pit.lost_tick_policy=delay \ -no-hpet \ -no-shutdown \ -global PIIX4_PM.disable_s3=1 \ -global PIIX4_PM.disable_s4=1 \ -boot strict=on \ -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7 \ -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x5 \ -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1 \ -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2 \ -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x6 \ -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x7 \ -blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/disk.qcow2","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-2-format","read-only":false,"driver":"qcow2","file":"libvirt-2-storage","backing":null}' \ -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=libvirt-2-format,id=scsi0-0-0-0,bootindex=1 \ -object secret,id=objlibvirt-1-storage_tls0-secret0,data=Et94qHmL96TI59qIrqOsDw==,keyid=masterKey0,iv=/a/LFpyIaJudfSbiEUdq5w==,format=base64 \ -object tls-creds-x509,id=objlibvirt-1-storage_tls0,dir=/etc/pki/libvirt-nbd,endpoint=client,verify-peer=yes,passwordid=objlibvirt-1-storage_tls0-secret0 \ -blockdev '{"driver":"nbd","server":{"type":"inet","host":"XXXX","port":"10809"},"export":"new","tls-creds":"objlibvirt-1-storage_tls0","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"raw","file":"libvirt-1-storage"}' \ -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,device_id=drive-scsi0-0-0-1,drive=libvirt-1-format,id=scsi0-0-0-1 \ -netdev tap,fd=34,id=hostnet0 \ -device e1000,netdev=hostnet0,id=net0,mac=52:54:00:b0:15:11,bus=pci.0,addr=0x3 \ -chardev pty,id=charserial0 \ -device isa-serial,chardev=charserial0,id=serial0 \ -chardev spicevmc,id=charchannel0,name=vdagent \ -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 \ -device usb-tablet,id=input0,bus=usb.0,port=1 \ -spice port=0,disable-ticketing,image-compression=off,seamless-migration=on \ -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2 \ -device intel-hda,id=sound0,bus=pci.0,addr=0x4 \ -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 \ -chardev spicevmc,id=charredir0,name=usbredir \ -device usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2 \ -chardev spicevmc,id=charredir1,name=usbredir \ -device usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3 \ -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 \ -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -msg timestamp=on 2020-07-21 07:57:32.953+0000: Domain id=3 is tainted: host-cpu char device redirected to /dev/pts/24 (label charserial0) 2020-07-21T07:57:33.126804Z qemu-kvm: -blockdev {"driver":"nbd","server":{"type":"inet","host":"XXXX","port":"10809"},"export":"new","tls-creds":"objlibvirt-1-storage_tls0","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}: Denied by server for option 5 (starttls) server reported: TLS not configured 2020-07-21 07:57:33.176+0000: shutting down, reason=failed Libvirtd log: 2020-07-21 07:54:21.994+0000: 1600129: debug : qemuDomainObjEndAsyncJob:1168 : Stopping async job: start (vm=0x7fcbd00deae0 name=fedora31) 2020-07-21 07:54:21.994+0000: 1617872: debug : qemuProcessHandleMonitorEOF:295 : Received EOF on 0x7fcbd00deae0 'fedora31' 2020-07-21 07:54:21.994+0000: 1617872: debug : qemuMonitorDispose:225 : mon=0x7fcc180430b0 2020-07-21 07:54:21.995+0000: 1600323: debug : qemuProcessEventHandler:4898 : vm=0x7fcbd00deae0, event=7 2020-07-21 07:54:21.995+0000: 1600323: debug : qemuProcessKill:7227 : vm=0x7fcbd00deae0 name=fedora31 pid=-1 flags=0x1 2020-07-21 07:54:21.995+0000: 1600323: debug : qemuProcessKill:7231 : VM 'fedora31' not active 2020-07-21 07:54:21.995+0000: 1600323: debug : qemuDomainObjBeginJobInternal:830 : Starting job: job=destroy agentJob=none asyncJob=none (vm=0x7fcbd00deae0 name=fedora31, current job=none agentJob=none async=none) 2020-07-21 07:54:21.995+0000: 1600323: debug : qemuDomainObjBeginJobInternal:877 : Started job: destroy (async=none vm=0x7fcbd00deae0 name=fedora31) 2020-07-21 07:54:21.995+0000: 1600323: debug : processMonitorEOFEvent:4790 : Domain 0x7fcbd00deae0 'fedora31' is not active, ignoring EOF 2020-07-21 07:54:21.995+0000: 1600323: debug : qemuDomainObjEndJob:1130 : Stopping job: destroy (async=none vm=0x7fcbd00deae0 name=fedora31) 2020-07-21 07:57:32.871+0000: 1600130: debug : qemuDomainObjBeginJobInternal:830 : Starting job: job=none agentJob=none asyncJob=start (vm=0x7fcbd00deae0 name=fedora31, current job=none agentJob=none async=none) 2020-07-21 07:57:32.871+0000: 1600130: debug : qemuDomainObjBeginJobInternal:885 : Started async job: start (vm=0x7fcbd00deae0 name=fedora31) 2020-07-21 07:57:32.871+0000: 1600130: debug : qemuProcessStart:7086 : conn=0x7fcc1c005aa0 driver=0x7fcbd003d710 vm=0x7fcbd00deae0 name=fedora31 id=-1 asyncJob=start migrateFrom=<null> migrateFd=-1 migratePath=<null> snapshot=(nil) vmop=0 flags=0x1 2020-07-21 07:57:32.871+0000: 1600130: debug : qemuProcessInit:5517 : vm=0x7fcbd00deae0 name=fedora31 id=-1 migration=0 2020-07-21 07:57:32.871+0000: 1600130: debug : qemuProcessInit:5519 : Beginning VM startup process 2020-07-21 07:57:32.871+0000: 1600130: debug : qemuProcessInit:5537 : Determining emulator version 2020-07-21 07:57:32.883+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.883+0000: 1600130: debug : qemuProcessStartValidate:5326 : Checking for KVM availability 2020-07-21 07:57:32.883+0000: 1600130: debug : qemuProcessStartValidate:5337 : Checking domain and device security labels 2020-07-21 07:57:32.883+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.883+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.884+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.884+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.884+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.884+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.884+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.884+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.884+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.884+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.884+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.885+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.885+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.885+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.885+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.885+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.885+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.885+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.885+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.885+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.886+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.886+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.886+0000: 1600130: debug : virQEMUCapsCacheLookup:5644 : Returning caps 0x7fcbd0026250 for /usr/libexec/qemu-kvm 2020-07-21 07:57:32.886+0000: 1600130: debug : qemuProcessStartValidate:5392 : Checking for any possible (non-fatal) issues 2020-07-21 07:57:32.886+0000: 1600130: debug : qemuProcessInit:5551 : Setting current domain def as transient 2020-07-21 07:57:32.887+0000: 1600130: debug : qemuDomainAssignVirtioSerialAddresses:136 : Finished reserving existing ports 2020-07-21 07:57:32.887+0000: 1600130: debug : qemuDomainUSBAddressAddHubs:3034 : Found 3 USB devices and 6 provided USB ports; adding 0 hubs 2020-07-21 07:57:32.887+0000: 1600130: debug : qemuDomainAssignUSBAddresses:3173 : Existing USB addresses have been reserved 2020-07-21 07:57:32.887+0000: 1600130: debug : qemuDomainAssignUSBAddresses:3181 : Finished assigning USB ports 2020-07-21 07:57:32.887+0000: 1600130: debug : qemuProcessPrepareDomain:6203 : Generating domain security label (if required) 2020-07-21 07:57:32.888+0000: 1600130: debug : qemuProcessPrepareDomain:6233 : Assigning domain PCI addresses 2020-07-21 07:57:32.888+0000: 1600130: debug : qemuDomainAssignVirtioSerialAddresses:136 : Finished reserving existing ports 2020-07-21 07:57:32.888+0000: 1600130: debug : qemuDomainUSBAddressAddHubs:3034 : Found 3 USB devices and 6 provided USB ports; adding 0 hubs 2020-07-21 07:57:32.888+0000: 1600130: debug : qemuDomainAssignUSBAddresses:3173 : Existing USB addresses have been reserved 2020-07-21 07:57:32.888+0000: 1600130: debug : qemuDomainAssignUSBAddresses:3181 : Finished assigning USB ports 2020-07-21 07:57:32.888+0000: 1600130: debug : qemuProcessPrepareDomain:6242 : Setting graphics devices 2020-07-21 07:57:32.888+0000: 1600130: debug : qemuProcessPrepareDomain:6246 : Create domain masterKey 2020-07-21 07:57:32.888+0000: 1600130: debug : qemuProcessPrepareDomain:6250 : Setting up storage 2020-07-21 07:57:32.889+0000: 1600130: debug : qemuProcessPrepareDomain:6254 : Prepare chardev source backends for TLS 2020-07-21 07:57:32.889+0000: 1600130: debug : qemuProcessPrepareDomain:6257 : Prepare device secrets 2020-07-21 07:57:32.889+0000: 1600130: debug : qemuProcessPrepareDomain:6261 : Prepare bios/uefi paths 2020-07-21 07:57:32.889+0000: 1600130: debug : qemuProcessPrepareDomain:6267 : Preparing external devices 2020-07-21 07:57:32.889+0000: 1600130: debug : qemuProcessPrepareDomain:6280 : Preparing monitor state 2020-07-21 07:57:32.889+0000: 1600130: debug : qemuProcessPrepareDomain:6289 : Updating guest CPU definition 2020-07-21 07:57:32.889+0000: 1600130: debug : qemuProcessPrepareHost:6466 : Preparing network devices 2020-07-21 07:57:32.890+0000: 1600130: debug : qemuProcessPrepareHost:6471 : Preparing host devices 2020-07-21 07:57:32.890+0000: 1600130: debug : qemuProcessPrepareHost:6480 : Preparing chr devices 2020-07-21 07:57:32.890+0000: 1600130: debug : qemuProcessPrepareHost:6492 : Ensuring no historical cgroup is lying around 2020-07-21 07:57:32.905+0000: 1600130: debug : qemuProcessPrepareHost:6525 : Write domain masterKey 2020-07-21 07:57:32.913+0000: 1600130: debug : qemuProcessPrepareHost:6529 : Preparing disks (host) 2020-07-21 07:57:32.919+0000: 1600130: debug : qemuProcessPrepareHost:6533 : Preparing external devices 2020-07-21 07:57:32.919+0000: 1600130: debug : qemuProcessLaunch:6672 : conn=0x7fcc1c005aa0 driver=0x7fcbd003d710 vm=0x7fcbd00deae0 name=fedora31 if=3 asyncJob=6 incoming.launchURI=<null> incoming.deferredURI=<null> incoming.fd=-1 incoming.path=<null> snapshot=(nil) vmop=0 flags=0x11 2020-07-21 07:57:32.919+0000: 1600130: debug : qemuProcessLaunch:6702 : Creating domain log file 2020-07-21 07:57:32.919+0000: 1600130: debug : qemuDomainLogContextNew:6048 : Context new 0x7fcc20047000 stdioLogD=1 2020-07-21 07:57:32.920+0000: 1600130: debug : qemuProcessLaunch:6718 : Building emulator command line 2020-07-21 07:57:32.920+0000: 1600130: debug : qemuBuildCommandLine:9889 : driver=0x7fcbd003d710 def=0x7fcc28042a80 mon=0x7fcc20046060 qemuCaps=0x7fcbd0026eb0 migrateURI=(null) snapshot=(nil) vmop=0 flags=0x0 2020-07-21 07:57:32.953+0000: 1600130: warning : qemuDomainObjTaint:5915 : Domain id=3 name='fedora31' uuid=9e49557d-7326-4cfc-8d37-6207bb17e7f5 is tainted: host-cpu 2020-07-21 07:57:32.953+0000: 1600130: debug : qemuProcessLaunch:6745 : Building mount namespace 2020-07-21 07:57:32.953+0000: 1600130: debug : qemuProcessLaunch:6750 : Setting up raw IO 2020-07-21 07:57:32.953+0000: 1600130: debug : qemuProcessLaunch:6760 : Setting up security labelling 2020-07-21 07:57:32.965+0000: 1600130: debug : qemuProcessLaunch:6786 : QEMU vm=0x7fcbd00deae0 name=fedora31 running with pid=1618364 2020-07-21 07:57:32.965+0000: 1600130: debug : qemuProcessLaunch:6793 : Writing early domain status to disk 2020-07-21 07:57:32.966+0000: 1600130: debug : qemuProcessLaunch:6797 : Waiting for handshake from child 2020-07-21 07:57:32.968+0000: 1600130: debug : qemuProcessLaunch:6805 : Setting up domain cgroup (if required) 2020-07-21 07:57:33.015+0000: 1600130: debug : qemuSetupImagePathCgroup:74 : Allow path /var/lib/libvirt/images/disk.qcow2, perms: rw 2020-07-21 07:57:33.015+0000: 1600130: debug : qemuSetupImageCgroupInternal:134 : Not updating cgroups for disk path 'new', type: network 2020-07-21 07:57:33.016+0000: 1600130: debug : qemuProcessLaunch:6823 : Setting emulator tuning/settings 2020-07-21 07:57:33.019+0000: 1600130: debug : qemuProcessLaunch:6827 : Setting cgroup for external devices (if required) 2020-07-21 07:57:33.019+0000: 1600130: debug : qemuProcessLaunch:6831 : Setting up resctrl 2020-07-21 07:57:33.019+0000: 1600130: debug : qemuProcessLaunch:6835 : Setting up managed PR daemon 2020-07-21 07:57:33.019+0000: 1600130: debug : qemuProcessLaunch:6840 : Setting domain security labels 2020-07-21 07:57:33.028+0000: 1600130: debug : qemuProcessLaunch:6872 : Labelling done, completing handshake to child 2020-07-21 07:57:33.028+0000: 1600130: debug : qemuProcessLaunch:6875 : Handshake complete, child running 2020-07-21 07:57:33.029+0000: 1600130: debug : qemuProcessLaunch:6886 : Waiting for monitor to show up 2020-07-21 07:57:33.029+0000: 1600130: debug : qemuProcessWaitForMonitor:2393 : Connect monitor to vm=0x7fcbd00deae0 name='fedora31' retry=0 2020-07-21 07:57:33.030+0000: 1600130: info : qemuMonitorOpenInternal:717 : QEMU_MONITOR_NEW: mon=0x7fcc180430b0 fd=34 2020-07-21 07:57:33.030+0000: 1600130: debug : qemuDomainObjBeginJobInternal:830 : Starting job: job=async nested agentJob=none asyncJob=none (vm=0x7fcbd00deae0 name=fedora31, current job=none agentJob=none async=start) 2020-07-21 07:57:33.030+0000: 1600130: debug : qemuDomainObjBeginJobInternal:877 : Started job: async nested (async=start vm=0x7fcbd00deae0 name=fedora31) 2020-07-21 07:57:33.030+0000: 1600130: debug : qemuDomainObjEnterMonitorInternal:5424 : Entering monitor (mon=0x7fcc180430b0 vm=0x7fcbd00deae0 name=fedora31) 2020-07-21 07:57:33.030+0000: 1600130: debug : qemuMonitorSetCapabilities:1557 : mon:0x7fcc180430b0 vm:0x7fcbd00deae0 fd:34 2020-07-21 07:57:33.031+0000: 1600130: info : qemuMonitorSend:946 : QEMU_MONITOR_SEND_MSG: mon=0x7fcc180430b0 msg={"execute":"qmp_capabilities","id":"libvirt-1"} fd=-1 2020-07-21 07:57:33.174+0000: 1618375: error : qemuMonitorIORead:491 : Unable to read from monitor: Connection reset by peer 2020-07-21 07:57:33.174+0000: 1618375: debug : qemuDomainLogContextRead:6156 : Context read 0x7fcc20047000 manager=0x7fcc20035f20 inode=34977140 pos=221969 2020-07-21 07:57:33.175+0000: 1618375: error : qemuProcessReportLogError:2097 : internal error: qemu unexpectedly closed the monitor: 2020-07-21T07:57:33.126804Z qemu-kvm: -blockdev {"driver":"nbd","server":{"type":"inet","host":"XXXX","port":"10809"},"export":"new","tls-creds":"objlibvirt-1-storage_tls0","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}: Denied by server for option 5 (starttls) server reported: TLS not configured 2020-07-21 07:57:33.175+0000: 1618375: debug : qemuMonitorIO:618 : Error on monitor internal error: qemu unexpectedly closed the monitor: 2020-07-21T07:57:33.126804Z qemu-kvm: -blockdev {"driver":"nbd","server":{"type":"inet","host":"XXXX","port":"10809"},"export":"new","tls-creds":"objlibvirt-1-storage_tls0","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}: Denied by server for option 5 (starttls) server reported: TLS not configured 2020-07-21 07:57:33.175+0000: 1618375: debug : qemuMonitorIO:649 : Triggering error callback 2020-07-21 07:57:33.175+0000: 1618375: debug : qemuProcessHandleMonitorError:346 : Received error on 0x7fcbd00deae0 'fedora31' 2020-07-21 07:57:33.175+0000: 1600130: debug : qemuMonitorSend:958 : Send command resulted in error internal error: qemu unexpectedly closed the monitor: 2020-07-21T07:57:33.126804Z qemu-kvm: -blockdev {"driver":"nbd","server":{"type":"inet","host":"XXXX","port":"10809"},"export":"new","tls-creds":"objlibvirt-1-storage_tls0","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}: Denied by server for option 5 (starttls) server reported: TLS not configured 2020-07-21 07:57:33.175+0000: 1600130: debug : qemuDomainObjExitMonitorInternal:5449 : Exited monitor (mon=0x7fcc180430b0 vm=0x7fcbd00deae0 name=fedora31) 2020-07-21 07:57:33.175+0000: 1618375: debug : qemuMonitorIO:618 : Error on monitor internal error: qemu unexpectedly closed the monitor: 2020-07-21T07:57:33.126804Z qemu-kvm: -blockdev {"driver":"nbd","server":{"type":"inet","host":"XXXX","port":"10809"},"export":"new","tls-creds":"objlibvirt-1-storage_tls0","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}: Denied by server for option 5 (starttls) server reported: TLS not configured 2020-07-21 07:57:33.175+0000: 1600130: debug : qemuDomainObjEndJob:1130 : Stopping job: async nested (async=start vm=0x7fcbd00deae0 name=fedora31) 2020-07-21 07:57:33.175+0000: 1600130: debug : qemuDomainLogContextRead:6156 : Context read 0x7fcc20047000 manager=0x7fcc20035f20 inode=34977140 pos=221969 2020-07-21 07:57:33.175+0000: 1618375: debug : qemuMonitorIO:639 : Triggering EOF callback 2020-07-21 07:57:33.176+0000: 1600130: error : qemuProcessReportLogError:2097 : internal error: process exited while connecting to monitor: 2020-07-21T07:57:33.126804Z qemu-kvm: -blockdev {"driver":"nbd","server":{"type":"inet","host":"XXXX","port":"10809"},"export":"new","tls-creds":"objlibvirt-1-storage_tls0","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}: Denied by server for option 5 (starttls) server reported: TLS not configured 2020-07-21 07:57:33.176+0000: 1600130: debug : qemuDomainLogContextFinalize:165 : ctxt=0x7fcc20047000 2020-07-21 07:57:33.176+0000: 1600130: debug : qemuProcessStop:7313 : Shutting down vm=0x7fcbd00deae0 name=fedora31 id=3 pid=1618364, reason=failed, asyncJob=start, flags=0x0 2020-07-21 07:57:33.176+0000: 1600130: debug : qemuDomainObjBeginJobInternal:830 : Starting job: job=async nested agentJob=none asyncJob=none (vm=0x7fcbd00deae0 name=fedora31, current job=none agentJob=none async=start) 2020-07-21 07:57:33.176+0000: 1600130: debug : qemuDomainObjBeginJobInternal:877 : Started job: async nested (async=start vm=0x7fcbd00deae0 name=fedora31) 2020-07-21 07:57:33.176+0000: 1600130: debug : qemuDomainLogAppendMessage:6224 : Append log message (vm='fedora31' message='2020-07-21 07:57:33.176+0000: shutting down, reason=failed ) stdioLogD=1 2020-07-21 07:57:33.177+0000: 1600130: info : qemuMonitorClose:866 : QEMU_MONITOR_CLOSE: mon=0x7fcc180430b0 2020-07-21 07:57:33.178+0000: 1600130: debug : qemuProcessKill:7227 : vm=0x7fcbd00deae0 name=fedora31 pid=1618364 flags=0x5 2020-07-21 07:57:33.178+0000: 1600130: debug : qemuDomainCleanupRun:6796 : driver=0x7fcbd003d710, vm=fedora31 2020-07-21 07:57:33.178+0000: 1600130: debug : qemuProcessAutoDestroyRemove:7686 : vm=fedora31 2020-07-21 07:57:33.178+0000: 1600130: warning : virSecuritySELinuxRestoreFileLabel:1461 : cannot resolve symlink new: No such file or directory 2020-07-21 07:57:33.194+0000: 1600130: debug : qemuDomainObjEndJob:1130 : Stopping job: async nested (async=start vm=0x7fcbd00deae0 name=fedora31) 2020-07-21 07:57:33.197+0000: 1600130: debug : qemuDomainObjEndAsyncJob:1168 : Stopping async job: start (vm=0x7fcbd00deae0 name=fedora31) 2020-07-21 07:57:33.197+0000: 1618375: debug : qemuProcessHandleMonitorEOF:295 : Received EOF on 0x7fcbd00deae0 'fedora31' 2020-07-21 07:57:33.197+0000: 1618375: debug : qemuMonitorDispose:225 : mon=0x7fcc180430b0 2020-07-21 07:57:33.197+0000: 1600323: debug : qemuProcessEventHandler:4898 : vm=0x7fcbd00deae0, event=7 2020-07-21 07:57:33.197+0000: 1600323: debug : qemuProcessKill:7227 : vm=0x7fcbd00deae0 name=fedora31 pid=-1 flags=0x1 2020-07-21 07:57:33.197+0000: 1600323: debug : qemuProcessKill:7231 : VM 'fedora31' not active 2020-07-21 07:57:33.197+0000: 1600323: debug : qemuDomainObjBeginJobInternal:830 : Starting job: job=destroy agentJob=none asyncJob=none (vm=0x7fcbd00deae0 name=fedora31, current job=none agentJob=none async=none) 2020-07-21 07:57:33.197+0000: 1600323: debug : qemuDomainObjBeginJobInternal:877 : Started job: destroy (async=none vm=0x7fcbd00deae0 name=fedora31) 2020-07-21 07:57:33.197+0000: 1600323: debug : processMonitorEOFEvent:4790 : Domain 0x7fcbd00deae0 'fedora31' is not active, ignoring EOF 2020-07-21 07:57:33.197+0000: 1600323: debug : qemuDomainObjEndJob:1130 : Stopping job: destroy (async=none vm=0x7fcbd00deae0 name=fedora31)
The error message is reported if the 'NBD' server doesn't offer TLS transport (it was not configured). From your steps above, you invoke it as: qemu-nbd --object tls-creds-x509,id=tls0,endpoint=server,dir=/etc/pki/libvirt-nbd /tmp/test -x new -t -v Your command line is missing the '--tls-creds=tls0' argument'.
(In reply to Peter Krempa from comment #5) > The error message is reported if the 'NBD' server doesn't offer TLS > transport (it was not configured). > > From your steps above, you invoke it as: > > qemu-nbd --object > tls-creds-x509,id=tls0,endpoint=server,dir=/etc/pki/libvirt-nbd /tmp/test > -x new -t -v > > Your command line is missing the '--tls-creds=tls0' argument'. Thank you. It works after set '--tls-creds=tls0' in qemu-nbd. Check with `qemu-img info`: # qemu-img info --object secret,id=sec,data=redhat,format=raw --object tls-creds-x509,id=tls0,endpoint=client,dir=/etc/pki/libvirt-nbd/,verify-peer=yes,passwordid=sec 'json:{"file":{"driver":"nbd", "server":{"host":"XXXX", "port":10809}, "export":"new", "tls-creds":"tls0"}}' image: nbd://XXXX:10809/new file format: raw virtual size: 100 MiB (104857600 bytes) disk size: unavailable libvirt version: v6.6.0-rc1
Retest on libvirt-6.6.0-4.module+el8.3.0+7883+3d717aa8.x86_64 qemu-kvm-5.1.0-6.module+el8.3.0+8041+42ff16b8.x86_64, passed.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (virt:8.3 bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:5137