Bug 951630 - Initialization of secure memory
Initialization of secure memory
Product: Virtualization Tools
Classification: Community
Component: libvirt (Show other bugs)
x86_64 Linux
unspecified Severity high
: ---
: ---
Assigned To: Libvirt Maintainers
Depends On:
  Show dependency treegraph
Reported: 2013-04-12 12:04 EDT by Shree
Modified: 2016-04-09 18:51 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2016-04-09 18:51:40 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Shree 2013-04-12 12:04:18 EDT
Description of problem:

We are facing a libvirtd crash when we are trying to connect to qemu by default TLS transport.
# virsh -c qemu+tls://localhost/system version
error: authentication failed: TLS handshake failed A TLS packet with unexpected length was received.
error: failed to connect to the hypervisor
I used my own CA and certificates (generated using http://libvirt.org/remote.html#Remote_libvirtd_configuration on Redhat PC) on both Kontron PC and our Board (with patched libvirt). Libvirtd.conf was modified so that libvirt is listening all IPs using default IP (so that it was possible to use same certificates on all machines) 
These directories and files created and used.
TLS connection worked fine with Kontron PC
# virsh -c qemu+tls://localhost/system version
Compiled against library: libvir 0.9.5
Using library: libvir 0.9.5
Using API: QEMU 0.9.5
Running hypervisor: QEMU 0.12.1
But libvirt crashed on our Board (using libvirt 0.10.2, gnutls-2.10.5-1_WR4.3.x86_64 and libudev-161-4 rpms )
# virsh -c qemu+tls://localhost/system version
error: authentication failed: TLS handshake failed A TLS packet with unexpected length was received.
error: failed to connect to the hypervisor

Version-Release number of selected component (if applicable):

libvirt 0.10.2

How reproducible:

The issue is easily reproducable when we are trying to securely connect to qemu using default TLS transport.

virsh -c qemu+tls://localhost/system version
Actual results:

Libvirt crash, dumping some internal log buffer

Expected results:

The command should not dump any core.
Additional info:


Program received signal SIGABRT, Aborted.
0x00007f8b786e1005 in raise () from /lib64/libc.so.6
(gdb) bt
#0  0x00007f8b786e1005 in raise () from /lib64/libc.so.6
#1  0x00007f8b786e3e40 in abort () from /lib64/libc.so.6
#2  0x00007f8b79ecadc5 in _gcry_logv (level=50, fmt=0x7f8b79f11170 "operation is not possible without initialized secure memory\n",
    arg_ptr=0x7fffab7a40a0) at misc.c:136
#3  0x00007f8b79ecb3d5 in _gcry_log_bug (fmt=0x870 <Address 0x870 out of bounds>) at misc.c:220
#4  0x00007f8b79ed0697 in _gcry_secmem_malloc_internal (size=<value optimized out>) at secmem.c:497
#5  0x00007f8b79ed079c in _gcry_secmem_malloc (size=136) at secmem.c:522
#6  0x00007f8b79ecba65 in do_malloc (n=2160, flags=<value optimized out>, mem=0x7fffab7a4200) at global.c:553
#7  0x00007f8b79ecbaa9 in _gcry_malloc_secure (n=2160) at global.c:592
#8  0x00007f8b79ecbb19 in _gcry_xmalloc_secure (n=136) at global.c:746
#9  0x00007f8b79f0e5df in _gcry_mpi_alloc_limb_space (nlimbs=17, secure=2160) at mpiutil.c:92
#10 0x00007f8b79f0e65f in _gcry_mpi_alloc_secure (nlimbs=17) at mpiutil.c:75
#11 0x00007f8b79efb25a in secret (output=0xcc71e0, input=0xcc6f80, skey=0x6) at rsa.c:365
#12 0x00007f8b79efb45a in _gcry_rsa_sign (algo=<value optimized out>, resarr=0xcc7130, data=0xcc6f80, skey=<value optimized out>) at rsa.c:608
#13 0x00007f8b79ed71ef in pubkey_sign (r_sig=0x7fffab7a43f8, s_hash=<value optimized out>, s_skey=<value optimized out>) at pubkey.c:692
#14 _gcry_pk_sign (r_sig=0x7fffab7a43f8, s_hash=<value optimized out>, s_skey=<value optimized out>) at pubkey.c:1807
---Type <return> to continue, or q <return> to quit---
#15 0x00007f8b7a174f9c in _wrap_gcry_pk_sign (algo=GNUTLS_PK_RSA, signature=0x7fffab7a4480, vdata=<value optimized out>, pk_params=0x7fffab7a44a0)
    at pk-libgcrypt.c:308
#16 0x00007f8b7a16108a in _gnutls_pkcs1_rsa_encrypt (ciphertext=<value optimized out>, plaintext=<value optimized out>, params=<value optimized out>,
    params_len=6, btype=<value optimized out>) at gnutls_pk.c:150
#17 0x00007f8b7a168fe6 in _gnutls_sign (algo=<value optimized out>, params=<value optimized out>, params_size=<value optimized out>,
    data=0x7fffab7a45e0, signature=0x0) at gnutls_sig.c:251
#18 0x00007f8b7a16988f in _gnutls_handshake_sign_data (session=0xced3a0, cert=0xce9270, pkey=<value optimized out>, params=<value optimized out>,
    signature=0x7fffab7a46c0, sign_algo=<value optimized out>) at gnutls_sig.c:226
#19 0x00007f8b7a169fbf in gen_dhe_server_kx (session=0xced3a0, data=0x7fffab7a4730) at auth_dhe.c:152
#20 0x00007f8b7a156195 in _gnutls_send_server_kx_message (session=0x870, again=<value optimized out>) at gnutls_kx.c:207
#21 0x00007f8b7a151c55 in _gnutls_handshake_server (session=0xced3a0) at gnutls_handshake.c:3047
#22 0x00007f8b7a152481 in gnutls_handshake (session=0xced3a0) at gnutls_handshake.c:2709

#23 0x00007f8b7aef4744 in virNetTLSSessionHandshake (sess=0xce9d70) at rpc/virnettlscontext.c:1377
#24 0x00007f8b7aee9a2b in virNetServerClientInit (client=0xce9800) at rpc/virnetserverclient.c:730
#25 0x00007f8b7aee7821 in virNetServerAddClient (svc=0xcef150, clientsock=<value optimized out>, opaque=0xce3cc0) at rpc/virnetserver.c:277
#26 virNetServerDispatchNewClient (svc=0xcef150, clientsock=<value optimized out>, opaque=0xce3cc0) at rpc/virnetserver.c:319


# ulimit -a
core file size          (blocks, -c) 1000000
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 63706
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 1024
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited


Daniel P Berrange is aware of this issue. Please find his comments below.

Ordinarily gnutls would initialize libgcrypt disabling secmem.
Libvirt, however, needs to register thread callbacks with gcrypt. Doing
this in turn disables gnutls' setup code. So secmem is left enabled.
This is not an issue on most distros, since they allow users to mlock
sufficient memory.

Anyway we need to fix libvirt to disable secmem, since we've blocked
gnutls' own setup from running
Comment 1 Shree 2013-04-12 12:06:01 EDT
We need this fix on high priority. Please give us some more leads to us so that we could try to fix this.

Thanks a lot !
Comment 2 Giorgio Zoppi 2014-01-20 05:13:07 EST
Shree, could you put your board in the internet so I can reproduce. I am not able to reproduce I cannot help. This seems more related to your gnutls implementation.
Comment 3 Martin Kletzander 2014-01-20 07:32:13 EST
Please try reproducing this on latest upstream release.  Also please describe what do you mean by "patched libvirt".  If that's your patch applied on top of libvirt's source than we cannot find where the problem is.  Thank you.

Note You need to log in before you can comment on or make changes to this bug.