Bug 1300772

Moving to 7.4

Reassigning to Peter as it relates to his blockdev-add work and has patches ready.

The feature was added upstream with following commits (and a bunch of prequel refactors):

a8dc146a4d qemu: migration: Add support for transporting NBD over TLS
89ed82646c qemu: migration: Extract code responsible for calling drive-mirror
8bad8e7bf0 qemu: migration: Rename NBD migration functions
99223c8cca test: Add status XML test for NBD tls storage migration
93a3f9844f qemu: domain: Add private data for NBD migration storage source definition
35a8487ad9 qemu: domain: Extract parsing of NBD status XML
0d6ec712d3 qemu: domain: Extract NBD disk migration private data formatting
ceb151b24b qemu: block: Add helpers for hot-adding virStorageSource via blockdev
d30fd1fc54 qemu: monitor: Introduce support for blockdev-mirror
c7b66f2a59 qemu: monitor: Add implementation for blockdev-add and blockdev-del
a6178a6610 qemu: monitor: Factor out and document code to format QMP command
e969af4cec qemu: caps: Add capability for blockdev-add/blockdev-del
d4e57c6f04 qemu: migration: Don't access disk members without lock
082266bf27 conf: domain: Export virDomainStorageSourceParse
b8240fe704 qemu: block: Don't nest storage layer properties into format layer

Verified on libvirt-4.5.0-1.el7.x86_64 qemu-kvm-rhev-2.12.0-6.el7.x86_64:
Preparation:
I. Create tls cert on server (target host)
1. Set up a Certificate Authority (CA)
1.1 
# certtool --generate-privkey > ca-key.pem

1.2 self-sign ca-key.pem by creating a file with the signature details called ca.info containing:
cn = $host1
ca
cert_signing_key

1.3 
# certtool --generate-self-signed --load-privkey ca-key.pem --template ca.info --outfile ca-cert.pem


2. Create server certificates
2.1 
# certtool --generate-privkey > server-key.pem

2.2 sign that key with the CA's private key by first creating a template file called server.info:
organization = Red Hat
cn = $host1
tls_www_server
encryption_key
signing_key

2.3 
# certtool --generate-certificate --load-privkey server-key.pem --load-ca-certificate ca-cert.pem \
--load-ca-privkey ca-key.pem --template server.info --outfile server-cert.pem

3. Copy CA key and server key to correct directory
3.1 # mkdir -p /etc/pki/qemu
3.2 # cp ca-key.pem ca-cert.pem /etc/pki/qemu
3.3 # cp server-key.pem /etc/pki/qemu
3.4 # cp server-cert.pem /etc/pki/qemu

II. Create client certs and keys
1. Copy CA cert to client(source host)

2.  Create client certificates

2.1 
# certtool --generate-privkey > client-key.pem

2.2 
Act as CA and sign the certificate.  Create client.info containing:

country = GB
state = London
locality = London
organization = Red Hat
cn = $host2
tls_www_client
encryption_key
signing_key

2.3 # certtool --generate-certificate  --load-privkey client-key.pem --load-ca-certificate /etc/pki/qemu/ca-cert.pem \
--load-ca-privkey /etc/pki/qemu/ca-key.pem --template client.info --outfile client-cert.pem

3. Copy client key to nbd tls directory

3.1 # cp client-key.pem /etc/pki/libvirt-nbd

3.2 # cp client-cert.pem /etc/pki/libvirt-nbd



Test nbd with tls:
1. Set qemu.conf:
nbd_tls = 1
nbd_tls_x509_cert_dir = "/etc/pki/libvirt-nbd"

2. Restart libvirtd

3. On server start qemu nbd:
# qemu-nbd --object tls-creds-x509,id=tls0,endpoint=server,dir=/root/server-keys --tls-creds=tls0 -p 10000 -t /tmp/c

4. On client host, prepare an VM with nbd disk:
# virsh dumpxml nbd-tls|grep disk -A 8
    <disk type='network' device='disk'>
      <driver name='qemu' type='raw'/>
      <source protocol='nbd'>
        <host name='lab.test.me' port='10000'/>
      </source>
      <target dev='sda' bus='scsi'/>
      <alias name='ua-TLS'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>

Start VM, check qemu cmdline, there is tls object like following:
# ps aux|grep qemu|grep nbd
-object tls-creds-x509,id=objua-TLS_tls0,dir=/etc/pki/libvirt-nbd,endpoint=client,verify-peer=yes -drive file.driver=nbd,file.server.type=inet,file.server.host=lab.test.me,file.server.port=10000,file.tls-creds=objua-TLS_tls0,format=raw,if=none,id=drive-ua-TLS -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-ua-TLS,id=ua-TLS

Detech the nbd:
# virsh detach-disk nbd-tls sda
Disk detached successfully

Sorry,comment6 is for BZ1544869. Please ignore it.

Verified on libvirt-4.5.0-6.el7.x86_64 qemu-kvm-rhev-2.12.0-10.el7.x86_64:
Preparation:
1. Prepare two named hosts with libvirt and qemu-kvm-rhev installed
2. Open ports 49152-49215/tcp for the firewalls of hosts
3. Prepare a shareable storage, like glusterfs server
4. Prepare a VM.
5. Set tls env referring to https://bugzilla.redhat.com/show_bug.cgi?id=1300772#c6

Test steps
SC1: Tests migration cancel with --copy-storage-inc and --tls
1. Create backing file on src host and dst host:
# qemu-img create -f qcow2 -b 'json:{"file.driver":"gluster","file.volume":"gv0","file.path":"A.qcow2","file.server":[{"type":"inet","host":"XX.XX.XX.XX","port":"24007"}]}' /var/lib/libvirt/images/fuse.qcow2
Formatting '/var/lib/libvirt/images/fuse.qcow2', fmt=qcow2 size=10737418240 backing_file=json:{"file.driver":"gluster",,"file.volume":"gv0",,"file.path":"A.qcow2",,"file.server":[{"type":"
inet",,"host":"XX.XX.XX.XX",,"port":"24007"}]} cluster_size=65536 lazy_refcounts=off refcount_bits=16

2. Start VM with the backing file on disk and write some data in VM:
# virsh dumpxml fuse
...
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/fuse.qcow2'/>
      <backingStore type='network' index='1'>
        <format type='raw'/>
        <source protocol='gluster' name='gv0/A.qcow2'>
          <host name='XX.XX.XX.XX' port='24007'/>
        </source>
        <backingStore/>
      </backingStore>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0d' function='0x0'/>
    </disk>
...
(in VM) # dd if=/dev/urandom of=file bs=1G count=1
0+1 records in
0+1 records out
33554431 bytes (34 MB, 32 MiB) copied, 0.155817 s, 215 MB/s

3. Migrate with --tls --copy-storage-inc and cancel it before finish
# virsh migrate fuse qemu+ssh://root/system  --tls --copy-storage-inc --verbose                                                                                
Migration: [ 32 %]^Cerror: operation aborted: migration out: canceled by client


SC2: Tests migration finish with --copy-storage-inc and --tls
1. Do as the steps in step1~2 in SC1
2. Migrate with --tls --copy-storage-inc
# virsh migrate fuse qemu+ssh://root/system  --tls --copy-storage-inc --verbose
Migration: [100 %]


SC3: Tests migration cancel with --copy-storage-all and --tls
1. Prepare a local disk and an VM using it:
# qemu-img info /var/lib/libvirt/images/fuse.qcow2 -U                                    
image: /var/lib/libvirt/images/fuse.qcow2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 1.4G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

# virsh dumpxml fuse
...
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/fuse.qcow2'/>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0d' function='0x0'/>
    </disk>
...

2. Migrate with --copy-storage-all --tls then cancel it:
# virsh migrate fuse qemu+ssh://root/system  --tls --copy-storage-all --verbose                                                                                
Migration: [  9 %]^Cerror: operation aborted: migration out: canceled by client


SC4: Tests migration finish with --copy-storage-all and --tls
1. Do as step1 in SC3
2. Migrate with --copy-storage-all --tls:
# virsh migrate fuse qemu+ssh://root/system  --tls --copy-storage-all --verbose
Migration: [100 %]

3.Do some write in VM
(In VM) # dd if=/dev/urandom of=file bs=100M count=2
dd: warning: partial read (33554431 bytes); suggest iflag=fullblock
0+2 records in
0+2 records out
67108862 bytes (67 MB) copied, 10.8148 s, 6.2 MB/s

All works well. Verified.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:3113