Bug 1046672

Summary: Crash in gnutls_global_deinit
Product: [Fedora] Fedora Reporter: Daniel BerrangĂ© <berrange>
Component: gnutlsAssignee: Nikos Mavrogiannopoulos <nmavrogi>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 20CC: fedora, jorton, nmavrogi, rdieter, rossetyler, tmraz
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: gnutls-3.1.18-3.fc19 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-01-05 06:05:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Fix global variable cleanup in gnutls_global_init/deinit none

Description Daniel Berrangé 2013-12-26 13:36:35 UTC
Description of problem:
The sound-juicer application has been experiencing crashes in gnutls_global_deinit

Program received signal SIGSEGV, Segmentation fault.
__gmpz_clear (m=0xb49def9668b7a12a) at ../../mpz/clear.c:28
28        (*__gmp_free_func) (PTR (m), ALLOC (m) * BYTES_PER_MP_LIMB);
(gdb) bt
#0  __gmpz_clear (m=0xb49def9668b7a12a) at ../../mpz/clear.c:28
#1  0x00007ffff7d6c6ea in mp_clear_multi (a=a@entry=0xb49def9668b7a12a) at multi.c:38
#2  0x00007ffff7d6dcfa in ecc_del_point (p=0xb49def9668b7a12a) at ecc_points.c:62
#3  0x00007ffff7d6d35a in _ecc_wmnaf_cache_entry_free (p=<optimized out>) at ecc_mulmod_cached.c:55
#4  ecc_wmnaf_cache_free () at ecc_mulmod_cached.c:70
#5  0x00007ffff7d6bfe5 in gnutls_crypto_deinit () at init.c:44
#6  0x00007ffff7cd69c1 in gnutls_global_deinit () at gnutls_global.c:303
#7  0x0000003190839868 in MusicBrainz5::CHTTPFetch::Fetch (this=this@entry=0x7fffeb3d2de0, 
    URL="/ws/2/release/50a07e8a-0caa-391e-a9e8-d2a9fc5c93bb?inc=artists%20artist-credits%20labels%20recordings%20release-groups%20url-rels%20discids%20recording-level-rels%20work-level-rels%20work-rels%20artis"..., Request="GET")
    at /usr/src/debug/libmusicbrainz-5.0.1/src/HTTPFetch.cc:232
#8  0x0000003190859215 in MusicBrainz5::CQuery::PerformQuery (this=this@entry=0x7fffe40048a0, 
    Query="/ws/2/release/50a07e8a-0caa-391e-a9e8-d2a9fc5c93bb?inc=artists%20artist-credits%20labels%20recordings%20release-groups%20url-rels%20discids%20recording-level-rels%20work-level-rels%20work-rels%20artis"...)
    at /usr/src/debug/libmusicbrainz-5.0.1/src/Query.cc:145
#9  0x0000003190859b8f in MusicBrainz5::CQuery::Query (this=this@entry=0x7fffe40048a0, Entity="release", 
Python Exception <type 'exceptions.IndexError'> list index out of range: 
    ID="50a07e8a-0caa-391e-a9e8-d2a9fc5c93bb", Resource="", Params=std::map with 1 elements)
    at /usr/src/debug/libmusicbrainz-5.0.1/src/Query.cc:253
#10 0x000000319084e3b2 in mb5_query_query (Query=0x7fffe40048a0, Entity=Entity@entry=0x42573b "release", 
    ID=ID@entry=0x7fffe4089b10 "50a07e8a-0caa-391e-a9e8-d2a9fc5c93bb", Resource=Resource@entry=0x4239dd "", 
    NumParams=NumParams@entry=1, ParamName=ParamName@entry=0x7fffeb3d3340, ParamValue=ParamValue@entry=0x7fffeb3d3350)
    at /usr/src/debug/libmusicbrainz-5.0.1/src/mb5_c.cc:733
#11 0x00000000004201e4 in mb5_list_albums (metadata=<optimized out>, url=<optimized out>, error=<optimized out>)
    at sj-metadata-musicbrainz5.c:544
#12 0x000000000041f15b in lookup_cd (mdg=0x98dd20) at sj-metadata-getter.c:196
#13 0x000000317c46ea45 in g_thread_proxy (data=0x6ae320) at gthread.c:798
#14 0x000000317a007f33 in start_thread (arg=0x7fffeb3d4700) at pthread_create.c:309
#15 0x00000031794f4ead in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
(gdb) 


After debugging this for a while, I noticed that libmusicbrainz is repeatedly invoking gnutls_global_init and gnutls_global_deinit (via calls to the neon library). This lead me to create the following demo program:

$ cat gnutlsdemo.c
#include <gnutls/gnutls.h>


int main(int argc, char **argv)
{
  int i;
  for (i = 0 ;i<  1000 ; i++) {
    gnutls_global_init();
    gnutls_global_deinit();
  }
}
$ gcc `pkg-config --cflags --libs gnutls`  -o gnutls-demo gnutls-demo.c

Running this under valgrind very quickly crashes

$ valgrind ./gnutls-demo
==19724== Memcheck, a memory error detector
==19724== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==19724== Using Valgrind-3.9.0 and LibVEX; rerun with -h for copyright info
==19724== Command: ./gnutls-demo
==19724== 
==19724== Invalid read of size 4
==19724==    at 0x34F20D5340: ecc_wmnaf_cache_free (ecc_mulmod_cached.c:69)
==19724==    by 0x34F203E9C0: gnutls_global_deinit (gnutls_global.c:303)
==19724==    by 0x400741: main (in /home/berrange/g)
==19724==  Address 0x50fe320 is 0 bytes after a block of size 4,352 alloc'd
==19724==    at 0x4A081D4: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==19724==    by 0x34F20D53DC: ecc_wmnaf_cache_init (ecc_mulmod_cached.c:202)
==19724==    by 0x34F203E777: gnutls_global_init (gnutls_global.c:212)
==19724==    by 0x40073C: main (in /home/berrange/g)
==19724== 
==19724== Invalid read of size 8
==19724==    at 0x34F20D5350: ecc_wmnaf_cache_free (ecc_mulmod_cached.c:55)
==19724==    by 0x34F203E9C0: gnutls_global_deinit (gnutls_global.c:303)
==19724==    by 0x400741: main (in /home/berrange/g)
==19724==  Address 0x50fe3b0 is not stack'd, malloc'd or (recently) free'd
==19724== 
==19724== Invalid read of size 4
==19724==    at 0x319A615F50: __gmpz_clear (clear.c:28)
==19724==    by 0x34F20D46E9: mp_clear_multi (multi.c:38)
==19724==    by 0x34F20D5CF9: ecc_del_point (ecc_points.c:62)
==19724==    by 0x34F20D5359: ecc_wmnaf_cache_free (ecc_mulmod_cached.c:55)
==19724==    by 0x34F203E9C0: gnutls_global_deinit (gnutls_global.c:303)
==19724==    by 0x400741: main (in /home/berrange/g)
==19724==  Address 0x50 is not stack'd, malloc'd or (recently) free'd
==19724== 
==19724== 
==19724== Process terminating with default action of signal 11 (SIGSEGV)
==19724==  Access not within mapped region at address 0x50
==19724==    at 0x319A615F50: __gmpz_clear (clear.c:28)
==19724==    by 0x34F20D46E9: mp_clear_multi (multi.c:38)
==19724==    by 0x34F20D5CF9: ecc_del_point (ecc_points.c:62)
==19724==    by 0x34F20D5359: ecc_wmnaf_cache_free (ecc_mulmod_cached.c:55)
==19724==    by 0x34F203E9C0: gnutls_global_deinit (gnutls_global.c:303)
==19724==    by 0x400741: main (in /home/berrange/g)
==19724==  If you believe this happened as a result of a stack
==19724==  overflow in your program's main thread (unlikely but
==19724==  possible), you can try to increase the size of the
==19724==  main thread stack using the --main-stacksize= flag.
==19724==  The main thread stack size used in this run was 8388608.
==19724== 
==19724== HEAP SUMMARY:
==19724==     in use at exit: 463,989 bytes in 1,734 blocks
==19724==   total heap usage: 84,433 allocs, 82,699 frees, 12,558,529 bytes allocated
==19724== 
==19724== LEAK SUMMARY:
==19724==    definitely lost: 0 bytes in 0 blocks
==19724==    indirectly lost: 0 bytes in 0 blocks
==19724==      possibly lost: 0 bytes in 0 blocks
==19724==    still reachable: 463,989 bytes in 1,734 blocks
==19724==         suppressed: 0 bytes in 0 blocks
==19724== Rerun with --leak-check=full to see details of leaked memory
==19724== 
==19724== For counts of detected and suppressed errors, rerun with: -v
==19724== ERROR SUMMARY: 4 errors from 3 contexts (suppressed: 2 from 2)



Version-Release number of selected component (if applicable):
gnutls-3.1.18-1.fc20.x86_64
(previous 3.1.x versions crash too)

How reproducible:
Often

Steps to Reproduce:
1. Run above demo program under valgrind

Comment 1 Daniel Berrangé 2013-12-26 13:58:01 UTC
Created attachment 841907 [details]
Fix global variable cleanup in gnutls_global_init/deinit

The flaw causing the crash is that the global variable "ecc_wmnaf_cache_last" is configured to point to an entry in the "ecc_wmnaf_cache" list by the gnutls_global_init function. When the gnutls_global_deinit function runs, it frees the "ecc_wmnaf_cache" list, but does not clear the "ecc_wmnaf_cache_last" pointer. So this variable is left pointing to free'd memory. When later calls to gnutls_global_init/deinit run they will suffer from use-after-free and eventually crash.

Comment 2 Nikos Mavrogiannopoulos 2013-12-28 16:25:08 UTC
It seems that this is Fedora specific, and the issue is in the patch that disables certain ECC algorithms. I'll include your fix in the patch.

Comment 4 Fedora Update System 2014-01-02 10:01:52 UTC
gnutls-3.1.18-3.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/gnutls-3.1.18-3.fc20

Comment 5 Christophe Fergeau 2014-01-02 10:31:45 UTC
*** Bug 1029394 has been marked as a duplicate of this bug. ***

Comment 6 Fedora Update System 2014-01-02 10:31:51 UTC
gnutls-3.1.18-3.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/gnutls-3.1.18-3.fc19

Comment 7 Fedora Update System 2014-01-03 08:32:15 UTC
Package gnutls-3.1.18-3.fc19:
* should fix your issue,
* was pushed to the Fedora 19 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing gnutls-3.1.18-3.fc19'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-0039/gnutls-3.1.18-3.fc19
then log in and leave karma (feedback).

Comment 8 Nikos Mavrogiannopoulos 2014-01-03 17:01:30 UTC
On fedora 20 please leave karma on:
https://admin.fedoraproject.org/updates/FEDORA-2014-0044/gnutls-3.1.18-3.fc20

Comment 9 Fedora Update System 2014-01-05 06:05:48 UTC
gnutls-3.1.18-3.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 10 Fedora Update System 2014-01-07 09:46:01 UTC
gnutls-3.1.18-3.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.