Bug 1300769

Summary: RFE: add support for native TLS encryption on migration TCP transport
Product: Red Hat Enterprise Linux 7 Reporter: Daniel Berrangé <berrange>
Component: libvirtAssignee: John Ferlan <jferlan>
Status: CLOSED ERRATA QA Contact: zhe peng <zpeng>
Severity: unspecified Docs Contact:
Priority: medium    
Version: 7.3CC: dgilbert, dyuan, jdenemar, jsuchane, lcheng, mtessun, mzhan, rbalakri, virt-bugs, virt-maint, xuzhang, yafu, yanqzhan, yppnts, zpeng
Target Milestone: rcKeywords: FutureFeature
Target Release: 7.4   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-3.2.0-2.el7 Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: 1300768 Environment:
Last Closed: 2017-08-01 17:09:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1300768    
Bug Blocks: 1301025, 1625597, 1664790, 1665042    

Description Daniel Berrangé 2016-01-21 16:45:27 UTC
Clone of QEMU bug to track libvirt enablement tasks for native TLS encryption with migration. 

+++ This bug was initially created as a clone of Bug #1300768 +++

Description of problem:
None of the QEMU migration transports have native support for encryption. As such apps/users needing security must tunnel the QEMU migration transport over another channel such as libvirt's secure connection. This is inefficient resulting in many more data copies and lower throughput for migration which reduces chances of it completing.

Providing native TLS encryption support for migration in QEMU will allow for secure migration with a lower performance overhead

Latest upstream code review posting is:

  https://lists.gnu.org/archive/html/qemu-devel/2016-01/msg01914.html

Comment 2 John Ferlan 2016-06-27 13:36:28 UTC
Moving to 7.4

Comment 4 John Ferlan 2017-02-17 19:41:31 UTC
Posted some patches upstream:

http://www.redhat.com/archives/libvir-list/2017-February/msg00897.html

Comment 5 John Ferlan 2017-03-25 13:22:05 UTC
After a few more iterations, final series of patches:

https://www.redhat.com/archives/libvir-list/2017-March/msg01108.html

has been pushed upstream:



git show a69e266d5eef60e0d6a85cc0734887f52ccd8992
commit a69e266d5eef60e0d6a85cc0734887f52ccd8992
Author: John Ferlan <jferlan>
Date:   Thu Feb 16 15:56:10 2017 -0500

    qemu: Set up the migration TLS objects for source
    
...
    
    If the migration flags indicate this migration will be using TLS,
    then while we have connection in the Begin phase check and setup the
    TLS environment that will be used by virMigrationRun during the Perform
    phase for the source to configure TLS.
    
    Processing adds an "-object tls-creds-x509,endpoint=client,..." and
    possibly an "-object secret,..." to handle the passphrase response.
    
    Then it sets the 'tls-creds' and possibly 'tls-hostname' migration
    parameters.
    
    The qemuMigrateCancel will clean up and reset the environment as it
    was originally found.


git describe a69e266d5eef60e0d6a85cc0734887f52ccd8992
v3.1.0-266-ga69e266

Comment 9 zhe peng 2017-06-16 06:07:05 UTC
verify with build:
libvirt-3.2.0-9.el7.x86_64

step:
1 prepare migration evn. two host,source and client
2 generate key both on source and client host and copy them to right dir
3 uncomment below both on server and client host qemu config
#vi /etc/libvirt/qemu.conf
default_tls_x509_verify = 1
migrate_tls_x509_verify = 1
4 on client server, define a guest and start
5 on client server, do live migration with tls enable
# virsh migrate --live rhel7 qemu+ssh://$sourcehost/system --verbose --tls
Migration: [100 %]

check source server libvirtd log file:
4450: info : qemuMonitorSend:1034 : QEMU_MONITOR_SEND_MSG: mon=0x7f092c0068c0 msg={"execute":"object-add","arguments":{"qom-type":"tls-creds-x509","id":"objlibvirt_migrate_tls0","props":{"dir":"/etc/pki/qemu","endpoint":"server","verify-peer":true}},"id":"libvirt-34"}
.....
4448: info : qemuMonitorIOWrite:536 : QEMU_MONITOR_IO_WRITE: mon=0x7f093c00d6d0 buf={"execute":"object-add","arguments":{"qom-type":"tls-creds-x509","id":"objlibvirt_migrate_tls0","props":{"dir":"/etc/pki/qemu","endpoint":"server","verify-peer":true}},"id":"libvirt-34"}^M
 len=188 ret=188 errno=0

check client server libvirtd log file:
3187: info : qemuMonitorSend:1034 : QEMU_MONITOR_SEND_MSG: mon=0x7f66d0056730 msg={"execute":"object-add","arguments":{"qom-type":"tls-creds-x509","id":"objlibvirt_migrate_tls0","props":{"dir":"/etc/pki/qemu","endpoint":"client","verify-peer":true}},"id":"libvirt-34"}^M
 fd=-1
....
3182: info : qemuMonitorIOWrite:536 : QEMU_MONITOR_IO_WRITE: mon=0x7f66d0056730 buf={"execute":"object-add","arguments":{"qom-type":"tls-creds-x509","id":"objlibvirt_migrate_tls0","props":{"dir":"/etc/pki/qemu","endpoint":"client","verify-peer":true}},"id":"libvirt-34"}^M
 len=188 ret=188 errno=0

Comment 10 ChanFisher 2017-06-28 01:26:38 UTC
(In reply to zhe peng from comment #9)
> verify with build:
> libvirt-3.2.0-9.el7.x86_64
> 
> step:
> 1 prepare migration evn. two host,source and client
> 2 generate key both on source and client host and copy them to right dir
> 3 uncomment below both on server and client host qemu config
> #vi /etc/libvirt/qemu.conf
> default_tls_x509_verify = 1
> migrate_tls_x509_verify = 1
> 4 on client server, define a guest and start
> 5 on client server, do live migration with tls enable
> # virsh migrate --live rhel7 qemu+ssh://$sourcehost/system --verbose --tls
> Migration: [100 %]
> 
> check source server libvirtd log file:
> 4450: info : qemuMonitorSend:1034 : QEMU_MONITOR_SEND_MSG:
> mon=0x7f092c0068c0
> msg={"execute":"object-add","arguments":{"qom-type":"tls-creds-x509","id":
> "objlibvirt_migrate_tls0","props":{"dir":"/etc/pki/qemu","endpoint":"server",
> "verify-peer":true}},"id":"libvirt-34"}
> .....
> 4448: info : qemuMonitorIOWrite:536 : QEMU_MONITOR_IO_WRITE:
> mon=0x7f093c00d6d0
> buf={"execute":"object-add","arguments":{"qom-type":"tls-creds-x509","id":
> "objlibvirt_migrate_tls0","props":{"dir":"/etc/pki/qemu","endpoint":"server",
> "verify-peer":true}},"id":"libvirt-34"}^M
>  len=188 ret=188 errno=0
> 
> check client server libvirtd log file:
> 3187: info : qemuMonitorSend:1034 : QEMU_MONITOR_SEND_MSG:
> mon=0x7f66d0056730
> msg={"execute":"object-add","arguments":{"qom-type":"tls-creds-x509","id":
> "objlibvirt_migrate_tls0","props":{"dir":"/etc/pki/qemu","endpoint":"client",
> "verify-peer":true}},"id":"libvirt-34"}^M
>  fd=-1
> ....
> 3182: info : qemuMonitorIOWrite:536 : QEMU_MONITOR_IO_WRITE:
> mon=0x7f66d0056730
> buf={"execute":"object-add","arguments":{"qom-type":"tls-creds-x509","id":
> "objlibvirt_migrate_tls0","props":{"dir":"/etc/pki/qemu","endpoint":"client",
> "verify-peer":true}},"id":"libvirt-34"}^M
>  len=188 ret=188 errno=0

hi, zhe peng:
error in my migrate env:
“kvm: Not a migration stream
kvm: load of migration failed: Invalid argument”
I think my certificate failed.
“2 generate key both on source and client host and copy them to right dir”
How to generate a certificate?and copy them where?

Comment 11 zhe peng 2017-06-29 02:54:44 UTC
hi ChanFisher:
   I use certtool to generate it, something like:
1.  Set up a Certificate Authority:
# certtool --generate-privkey > ca-key.pem
2  self-sign ca-key.pem by creating a file with the signature details called ca.info containing:
# certtool --generate-self-signed --load-privkey ca-key.pem --template ca.info --outfile ca-cert.pem
3  Create server certificates
#certtool --generate-privkey > server-key.pem
#certtool --generate-certificate --load-privkey server-key.pem --load-ca-certificate ca-cert.pem --load-ca-privkey ca-key.pem --template server.info --outfile server-cert.pem

it's not all of them, you also need create client certificates.

the default dir is /etc/pki/qemu, you can create it by yourself or change it in
/etc/libvirt/qemu.conf.

Comment 12 John Ferlan 2017-06-29 10:07:35 UTC
FWIW: There's a good description starting here w/ pictures:

http://wiki.libvirt.org/page/TLSSetup

Comment 13 ChanFisher 2017-06-30 01:55:07 UTC
(In reply to zhe peng from comment #11)
> hi ChanFisher:
>    I use certtool to generate it, something like:
> 1.  Set up a Certificate Authority:
> # certtool --generate-privkey > ca-key.pem
> 2  self-sign ca-key.pem by creating a file with the signature details called
> ca.info containing:
> # certtool --generate-self-signed --load-privkey ca-key.pem --template
> ca.info --outfile ca-cert.pem
> 3  Create server certificates
> #certtool --generate-privkey > server-key.pem
> #certtool --generate-certificate --load-privkey server-key.pem
> --load-ca-certificate ca-cert.pem --load-ca-privkey ca-key.pem --template
> server.info --outfile server-cert.pem
> 
> it's not all of them, you also need create client certificates.
> 
> the default dir is /etc/pki/qemu, you can create it by yourself or change it
> in
> /etc/libvirt/qemu.conf.

Thank you very much! ^_^

Comment 14 ChanFisher 2017-06-30 01:56:27 UTC
(In reply to John Ferlan from comment #12)
> FWIW: There's a good description starting here w/ pictures:
> 
> http://wiki.libvirt.org/page/TLSSetup

very useful

Comment 15 errata-xmlrpc 2017-08-01 17:09:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846

Comment 16 errata-xmlrpc 2017-08-01 23:51:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846