Bug 605502

Summary: python fails with a backtrace
Product: Red Hat Enterprise Linux 6 Reporter: Rakesh Pandit <rpandit>
Component: rpmAssignee: Panu Matilainen <pmatilai>
Status: CLOSED NOTABUG QA Contact: BaseOS QE Security Team <qe-baseos-security>
Severity: high Docs Contact:
Priority: high    
Version: 6.0CC: ffesti, notting, rpandit
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-07-13 06:36:30 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Rakesh Pandit 2010-06-18 06:35:06 UTC
Description of problem:

Attached is core dump

*** glibc detected *** python: double free or corruption (fasttop): 0x00007fb8e81ca610 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x75746)[0x7fb9036c0746]
/lib64/libc.so.6(+0x7ab90)[0x7fb9036c5b90]
/lib64/libc.so.6(realloc+0xe5)[0x7fb9036c5d85]
/usr/lib64/librpmio.so.1(rrealloc+0x18)[0x7fb8f95dd4f8]
/usr/lib64/librpm.so.1(+0x1bab3)[0x7fb8f9f8fab3]
/usr/lib64/librpm.so.1(+0x1d1f0)[0x7fb8f9f911f0]
/usr/lib64/librpm.so.1(rpmdbOpen+0x4c)[0x7fb8f9f915cc]
/usr/lib64/librpm.so.1(rpmtsOpenDB+0x4b)[0x7fb8f9fbce2b]
/usr/lib64/librpm.so.1(rpmtsInitIterator+0x253)[0x7fb8f9fbd0f3]
/usr/lib64/librpm.so.1(+0x492b1)[0x7fb8f9fbd2b1]
/usr/lib64/librpm.so.1(rpmtsGetKeyring+0x2d)[0x7fb8f9fbd48d]
/usr/lib64/librpm.so.1(rpmReadPackageFile+0x33)[0x7fb8f9fa6dc3]
/usr/lib64/python2.6/site-packages/rpm/_rpmmodule.so(+0xe24a)[0x7fb8fa1ec24a]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fb9042b4ae3]
/usr/lib64/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x43)[0x7fb904347e23]
/usr/lib64/libpython2.6.so.1.0(+0x5e03c)[0x7fb9042cf03c]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fb9042b4ae3]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x45a0)[0x7fb90434cf00]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x6391)[0x7fb90434ecf1]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x871)[0x7fb90434f5d1]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5169)[0x7fb90434dac9]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x871)[0x7fb90434f5d1]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5169)[0x7fb90434dac9]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x6391)[0x7fb90434ecf1]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x871)[0x7fb90434f5d1]
/usr/lib64/libpython2.6.so.1.0(+0x6dc4d)[0x7fb9042dec4d]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fb9042b4ae3]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3e06)[0x7fb90434c766]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x6391)[0x7fb90434ecf1]
======= Memory map: ========
00400000-00401000 r-xp 00000000 fd:00 1059327                            /usr/bin/python
00600000-00601000 rw-p 00000000 fd:00 1059327                            /usr/bin/python
02441000-02d13000 rw-p 00000000 00:00 0                                  [heap]
7fb8e0000000-7fb8e0368000 rw-p 00000000 00:00 0 
7fb8e0368000-7fb8e4000000 ---p 00000000 00:00 0 
7fb8e8000000-7fb8e8309000 rw-p 00000000 00:00 0 
7fb8e8309000-7fb8ec000000 ---p 00000000 00:00 0 
7fb8eebfe000-7fb8eebff000 ---p 00000000 00:00 0 
7fb8eebff000-7fb8ef5ff000 rw-p 00000000 00:00 0 
7fb8ef5ff000-7fb8ef600000 ---p 00000000 00:00 0 
7fb8ef600000-7fb8f0000000 rw-p 00000000 00:00 0 
7fb8f0000000-7fb8f046b000 rw-p 00000000 00:00 0 
7fb8f046b000-7fb8f4000000 ---p 00000000 00:00 0 
7fb8f4195000-7fb8f4196000 ---p 00000000 00:00 0 
7fb8f4196000-7fb8f4b96000 rw-p 00000000 00:00 0 
7fb8f4b96000-7fb8f4b9f000 r-xp 00000000 fd:00 1058405                    /usr/lib64/python2.6/lib-dynload/itertoolsmodule.so
7fb8f4b9f000-7fb8f4d9f000 ---p 00009000 fd:00 1058405                    /usr/lib64/python2.6/lib-dynload/itertoolsmodule.so
7fb8f4d9f000-7fb8f4da3000 rw-p 00009000 fd:00 1058405                    /usr/lib64/python2.6/lib-dynload/itertoolsmodule.so
7fb8f4da3000-7fb8f4de4000 rw-p 00000000 00:00 0 
7fb8f4de4000-7fb8f4dec000 r-xp 00000000 fd:00 1102295                    /usr/lib64/python2.6/lib-dynload/arraymodule.so
7fb8f4dec000-7fb8f4feb000 ---p 00008000 fd:00 1102295                    /usr/lib64/python2.6/lib-dynload/arraymodule.so
7fb8f4feb000-7fb8f4fee000 rw-p 00007000 fd:00 1102295                    /usr/lib64/python2.6/lib-dynload/arraymodule.so
7fb8f4fee000-7fb8f5004000 r-xp 00000000 fd:00 1049787                    /lib64/libnsl-2.12.so
7fb8f5004000-7fb8f5203000 ---p 00016000 fd:00 1049787                    /lib64/libnsl-2.12.so
7fb8f5203000-7fb8f5204000 r--p 00015000 fd:00 1049787                    /lib64/libnsl-2.12.so
7fb8f5204000-7fb8f5205000 rw-p 00016000 fd:00 1049787                    /lib64/libnsl-2.12.so
7fb8f5205000-7fb8f5207000 rw-p 00000000 00:00 0 
7fb8f5207000-7fb8f520e000 r-xp 00000000 fd:00 1049781                    /lib64/libcrypt-2.12.so
7fb8f520e000-7fb8f540e000 ---p 00007000 fd:00 1049781                    /lib64/libcrypt-2.12.so
7fb8f540e000-7fb8f540f000 r--p 00007000 fd:00 1049781                    /lib64/libcrypt-2.12.so
7fb8f540f000-7fb8f5410000 rw-p 00008000 fd:00 1049781                    /lib64/libcrypt-2.12.so
7fb8f5410000-7fb8f543e000 rw-p 00000000 00:00 0 
7fb8f543e000-7fb8f5576000 r-xp 00000000 fd:00 1057360                    /usr/lib64/mysql/libmysqlclient_r.so.16.0.0
7fb8f5576000-7fb8f5776000 ---p 00138000 fd:00 1057360                    /usr/lib64/mysql/libmysqlclient_r.so.16.0.0
7fb8f5776000-7fb8f57c2000 rw-p 00138000 fd:00 1057360                    /usr/lib64/mysql/libmysqlclient_r.so.16.0.0
7fb8f57c2000-7fb8f57c4000 rw-p 00000000 00:00 0 
7fb8f57c4000-7fb8f57ce000 r-xp 00000000 fd:00 1094416                    /usr/lib64/python2.6/site-packages/_mysql.so
7fb8f57ce000-7fb8f59cd000 ---p 0000a000 fd:00 1094416                    /usr/lib64/python2.6/site-packages/_mysql.so
7fb8f59cd000-7fb8f59d2000 rw-p 00009000 fd:00 1094416                    /usr/lib64/python2.6/site-packages/_mysql.so
7fb8f59d2000-7fb8f59e2000 r-xp 00000000 fd:00 1058397                    /usr/lib64/python2.6/lib-dynload/datetime.so
7fb8f59e2000-7fb8f5be2000 ---p 00010000 fd:00 1058397                    /usr/lib64/python2.6/lib-dynload/datetime.so
7fb8f5be2000-7fb8f5be6000 rw-p 00010000 fd:00 1058397                    /usr/lib64/python2.6/lib-dynload/datetime.so
7fb8f5be6000-7fb8f5c68000 rw-p 00000000 00:00 0 
7fb8f5c68000-7fb8f5c69000 r-xp 00000000 fd:00 1102294                    /usr/lib64/python2.6/lib-dynload/_weakref.so
7fb8f5c69000-7fb8f5e69000 ---p 00001000 fd:00 1102294                    /usr/lib64/python2.6/lib-dynload/_weakref.so
7fb8f5e69000-7fb8f5e6a000 rw-p 00001000 fd:00 1102294                    /usr/lib64/python2.6/lib-dynload/_weakref.so
7fb8f5e6a000-7fb8f5eab000 rw-p 00000000 00:00 0 
7fb8f5eab000-7fb8f5ead000 r-xp 00000000 fd:00 1058354                    /usr/lib64/python2.6/lib-dynload/_bisectmodule.so
7fb8f5ead000-7fb8f60ac000 ---p 00002000 fd:00 1058354                    /usr/lib64/python2.6/lib-dynload/_bisectmodule.so
7fb8f60ac000-7fb8f60ad000 rw-p 00001000 fd:00 1058354                    /usr/lib64/python2.6/lib-dynload/_bisectmodule.so
7fb8f60ad000-7fb8f60b3000 r-xp 00000000 fd:00 1058363                    /usr/lib64/python2.6/lib-dynload/_collectionsmodule.so
7fb8f60b3000-7fb8f62b2000 ---p 00006000 fd:00 1058363                    /usr/lib64/python2.6/lib-dynload/_collectionsmodule.so
Aborted (core dumped)

Comment 1 Rakesh Pandit 2010-06-18 06:39:56 UTC
$ gdb --core core.dump
(gdb) Core was generated by `python ./script.py ...'
Program terminated with signal 6, Aborted.
#0  0x00007fb90367d9c5 in raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
64	  return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
 ...
(gdb) bt
#0  0x00007fb90367d9c5 in raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1  0x00007fb90367f1a5 in abort () at abort.c:92
#2  0x00007fb9036bae2b in __libc_message (do_abort=2, fmt=0x7fb90378ea98 "*** glibc detected *** %s: %s: 0x%s ***\n")
    at ../sysdeps/unix/sysv/linux/libc_fatal.c:186
#3  0x00007fb9036c0746 in malloc_printerr (action=3, str=0x7fb90378ee20 "double free or corruption (fasttop)", ptr=<value optimized out>)
    at malloc.c:6283
#4  0x00007fb9036c5b90 in _int_realloc (av=0x7fb8e8000020, oldp=0x7fb8e81ca600, oldsize=<value optimized out>, nb=<value optimized out>)
    at malloc.c:5339
#5  0x00007fb9036c5d85 in __libc_realloc (oldmem=0x7fb8e81ca610, bytes=32) at malloc.c:3821
#6  0x00007fb8f95dd4f8 in ?? ()
#7  0x0000000000000008 in ?? ()
#8  0x00007fb8f9f8fab3 in ?? ()
#9  0x0000000000000003 in ?? ()
#10 0x0000000000000000 in ?? ()
(gdb) list
59	    if (__builtin_expect (pid <= 0, 0))
60	      pid = (pid & INT_MAX) == 0 ? selftid : -pid;
61	#endif
62	
63	#if __ASSUME_TGKILL
64	  return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
65	#else
66	# ifdef __NR_tgkill
67	  int res = INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
68	  if (res != -1 || errno != ENOSYS)

Comment 3 Rakesh Pandit 2010-06-18 06:45:10 UTC
http://people.pnq.redhat.com/~rpandit/core.12029 core dump is too big to be attached in BZ, so I have kept it on my internal people page.

Comment 4 RHEL Program Management 2010-06-18 06:53:16 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.

Comment 5 Bill Nottingham 2010-06-18 14:49:35 UTC
Looks like it's in the rpm library routines.

Comment 6 Rakesh Pandit 2010-06-18 16:54:06 UTC
Umm .. I did not looked carefully. Will try to get a reproducible script at least (Monday now :) .. this side of world it is already weekend.

Comment 7 Panu Matilainen 2010-06-29 05:50:46 UTC
Any chance of a reproducer (or is this actually reproducable for you)?

Comment 8 Rakesh Pandit 2010-06-29 05:59:30 UTC
Yes it does on one of our servers. I will try to update with a reproducer today evening.

Comment 9 Panu Matilainen 2010-07-09 05:49:44 UTC
Ping? :)

If a minimal reproducer is tough, a backtrace with all debuginfos installed and/or a pointer to the crashing script, and the exact rpm version involved would be helpful.

Comment 11 Florian Festi 2010-07-13 06:36:30 UTC
It turns out this crash is caused by using the - not thread save - rpmlib in a threaded environment. Moving all calls to the library into one thread or using any other serialization method should fix this.

CLOSING as NOTABUG