RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 729874 - SSL spice session can't be kept during migration
Summary: SSL spice session can't be kept during migration
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt
Version: 6.2
Hardware: x86_64
OS: Linux
high
urgent
Target Milestone: rc
: ---
Assignee: Michal Privoznik
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks: 728234
TreeView+ depends on / blocked
 
Reported: 2011-08-11 06:38 UTC by Vivian Bian
Modified: 2011-12-06 11:26 UTC (History)
15 users (show)

Fixed In Version: libvirt-0.9.4-5.el6
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-12-06 11:26:08 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
script to generate spice cert files (1.54 KB, application/x-shellscript)
2011-08-11 06:38 UTC, Vivian Bian
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2011:1513 0 normal SHIPPED_LIVE libvirt bug fix and enhancement update 2011-12-06 01:23:30 UTC

Description Vivian Bian 2011-08-11 06:38:05 UTC
Created attachment 517741 [details]
script to generate spice cert files

Description of problem:
Tried migration testing with spice client connected on source host . But after migration to target host , the spice session get disconnected . And there is the error log message in the target /var/log/libvirt/libvirt.log


Version-Release number of selected component (if applicable):
libvirt-0.9.4-2.el6.x86_64
qemu-kvm-0.12.1.2-2.177.el6.x86_64
kernel-2.6.32-178.el6.x86_64
spice-server-0.8.2-3.el6.x86_64
spice-client-0.8.2-3.el6.x86_64

How reproducible 
Always

Steps:
1. edit spice client with following xml section
    <graphics type='spice' autoport='yes' listen='0.0.0.0' passwd='redhat'>
      <listen type='address' address='0.0.0.0'/>
      <channel name='main' mode='secure'/>
      <channel name='record' mode='insecure'/>
    </graphics>
2. edit /etc/libvirt/qemu.conf
    enable  following options :
    a.  spice_tls = 1
    b.  spice_tls_x509_cert_dir = "/etc/pki/libvirt-spice"
  
3. perform spice.sh to generate spice ssl cert files , and scp /etc/pki/libvirt-spice/* to the target machine /etc/pki/libvirt-spice 
 
4. restart libvirtd on both source machine and target machine
5. start spice guest
6. perform this command on source machine to connect spice session on source machine
   # spicec -h 10.66.4.220 -p 5900 -s 5901 --host-subject "C=IL,L=Raanana,O=Red Hat,CN=my server" --ca-file /etc/pki/libvirt-spice/ca-cert.pem --secure-channels main --enable-channels all -w redhat

7. migrate guest
   #  virsh migrate --live spice qemu+ssh://10.66.4.241/system

Actual result :
1. spice session disconnected
2. error message in target machine
    19:55:34.660: 24254: info : libvirt version: 0.9.4, package: 2.el6 (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>, 2011-08-09-04:48:09, hs20-bc2-3.build.redhat.com)
    19:55:34.660: 24254: error : qemuDomainExtractTLSSubject:164 : internal error cannot initialize cert object: ASN1 parser: Element was not found.
    19:55:34.660: 24254: warning : qemuMigrationPrepareAny:1078 : Unable to encode migration cookie
3. No extra error message generated on source machine , and no error message record in messages log

Additional info
1. both on the source machine and target machine , the spice session could be accessed successfully . After migration , the spice session could be accessed by target host and source host .

2. tested with following package, spice session WON'T be cut during migration 
libvirt-0.8.7-18.el6.x86_64.rpm
qemu-kvm-0.12.1.2-2.177.el6.x86_64
kernel-2.6.32-178.el6.x86_64
spice-server-0.8.2-3.el6.x86_64
spice-client-0.8.2-3.el6.x86_64

Comment 2 Vivian Bian 2011-08-11 07:49:22 UTC
Btw , we keep the same spice cert files when doing the testing on both the old libvirt version and the new libvirt version . And with the old libvirt version , the spice session could be kept . With the new libvirt version , spice session would be interrupted .

Comment 3 Christophe Fergeau 2011-08-11 08:30:06 UTC
19:55:34.660: 24254: error : qemuDomainExtractTLSSubject:164 : internal
error cannot initialize cert object: ASN1 parser: Element was not found.

This error comes from 

    ret = gnutls_x509_crt_init(&cert);
    if (ret < 0) {
        qemuReportError(VIR_ERR_INTERNAL_ERROR,
                        _("cannot initialize cert object: %s"),
                        gnutls_strerror(ret));
        goto error;
    }

so it happens before checking the spice certificate.
gnutls_x509_crt_init is 

int
gnutls_x509_crt_init (gnutls_x509_crt_t * cert)
{
  gnutls_x509_crt_t tmp = gnutls_calloc (1, sizeof (gnutls_x509_crt_int));
  int result;

  if (!tmp)
    return GNUTLS_E_MEMORY_ERROR;

  result = asn1_create_element (_gnutls_get_pkix (),
                                "PKIX1.Certificate", &tmp->cert);
  if (result != ASN1_SUCCESS)
    {
      gnutls_assert ();
      gnutls_free (tmp);
      return _gnutls_asn2err (result);
    }

  /* If you add anything here, be sure to check if it has to be added
     to gnutls_x509_crt_import as well. */

  *cert = tmp;

  return 0;                     /* success */
}

it it seems it's failing in the asn1_create_element call. Given the error message, I'd guess that _gnutls_get_pkix() doesn't contain the "PKIX1.Certificate" element. _gnutls_get_pkix() is initialized in gnutls_global_init(), and doing a git grep on libvirt source, I can only find 2 occurrences:
src/rpc/virnettlscontext.c:    gnutls_global_init();
tests/virnettlscontexttest.c:    gnutls_global_init();

Could it be that a call to gnutls_global_init() is missing in libvirt? Or did I get misled somewhere on the way?

Comment 4 Daniel Berrangé 2011-08-11 09:27:57 UTC
> Could it be that a call to gnutls_global_init() is missing in libvirt? Or did I
> get misled somewhere on the way?

This sounds like most likely cause. If  libvirtd itself is not configured to use TLS, they we could be missing the call to gnutls_global_init().

Comment 5 Michal Privoznik 2011-08-17 16:58:14 UTC
Patch sent upstream:

https://www.redhat.com/archives/libvir-list/2011-August/msg00785.html

Comment 6 Michal Privoznik 2011-08-19 09:37:40 UTC
Moving to POST:

http://post-office.corp.redhat.com/archives/rhvirt-patches/2011-August/msg00540.html


commit 4e9ad9a46dc8bb50e65c7b544072f66377fd54a7
Author: Michal Privoznik <mprivozn>
Date:   Thu Aug 18 10:44:08 2011 +0200

    daemon: initialize GnuTLS
    
    When spice_tls is set but listen_tls is not, we don't initialize
    GnuTLS library. So any later gnutls call (e.g. during migration,
    where we initialize a certificate) will access uninitialized GnuTLS
    internal structs and throws an error.
    
    Although, we might now initialize GnuTLS twice, it is safe according
    to the documentation:
    
        This function can be called many times,
        but will only do something the first time.
    
    This patch creates 2 functions: virNetTLSInit and virNetTLSDeinit
    with respect to written above.
    (cherry picked from commit 74c75671331d284e1f777f9692b72e9737520bf0)

Comment 8 weizhang 2011-08-23 06:20:38 UTC
verify pass on 
kernel-2.6.32-191.el6.x86_64
qemu-kvm-0.12.1.2-2.184.el6.x86_64
libvirt-0.9.4-5.el6.x86_64

After live migration, spice session is still alive and no error in libvirtd.log.

Comment 9 errata-xmlrpc 2011-12-06 11:26:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1513.html


Note You need to log in before you can comment on or make changes to this bug.