Red Hat Bugzilla – Bug 82640
Gdb unable to debug threaded apps
Last modified: 2016-11-24 07:27:25 EST
From Bugzilla Helper: User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20021130 Description of problem: AFAICT, gdb cannot debug anything linked to libpthread. I *think* the RHAS version is "Q1 Errata Beta", but I'm not sure. The exact versions of relevant packages are: gdb-5.2-2 ncurses-5.2-12 glibc-2.2.4-31.7 kernel-smp-2.4.18-e.14 /proc/version says: Linux version 2.4.18-e.14smp (bhcompile@rocky.devel.redhat.com) (gcc version 2.96 20000731 (Red Hat Linux 7.2 2.96-112.7.2)) #1 SMP Thu Jan 16 22:08:18 EST 2003 Version-Release number of selected component (if applicable): How reproducible: Always Steps to Reproduce: 1. gdb -nx pthread_create 2. break main 3. run Actual Results: Starting program: /home/johan/src/test/pthread_create Error while reading shared library symbols: Cannot find new threads: generic error Cannot find user-level thread for LWP 10411: no LWP to satisfy query Expected Results: The program should have started and then stopped at the breakpoint in main(). Additional info: I can't say for sure that the above versions are current for RHAS/ia64. I *think so*, but I'm really not sure. So this *may* just be a matter of me not being up to date enough. Also, severity high because this stops most of our development on RHAS/ia64 (which may affect other people as well).
Created attachment 89569 [details] A simple threaded test program. Build with "-g -lpthread" The test program wasn't really created for testing this, so it does some other junk as well (but works fine as a test case for this).
FWIW I get the same result with a home-brew gdb 5.3 on this system.
Downgrading glibc to glibc-2.2.4-29.2 fixes this problem. Since we don't have any dependencies on any particular glibc version we'll just downgrade glibc on our development machines, but you probably want to fix this anyway. We are not panicking anymore anyway :-).
thanks for the suggestion. I will add the dependency to the spec file from now on.
But it is a glibc problem, not a gdb one.
fhirtz: Why did you change the platform to i686? As the original reporter of this bug I can say for sure it exists on ia64, but I don't know about i686. If you see it on i686 as well, shouldn't the Platform be All rather than i686? Having this fixed on ia64 would be great!
Oversight on my part. The arch for this has been changed to 'all'.
The following glibc patch re-introduces the static thread handle initalizer and seems to fix the problem. (This patch replaces the maxthreads patch in the 2.2.4-31.7 src rpm.) --- linuxthreads/pthandles.c.dh Fri Apr 4 09:11:57 2003 +++ linuxthreads/pthandles.c Fri Apr 4 09:18:35 2003 @@ -0,0 +1,9 @@ +#include <ldsodefs.h> +#include "pthread.h" +#include "internals.h" + +/* Array of active threads. Entry 0 is reserved for the initial thread. */ +struct pthread_handle_struct __pthread_handles[PTHREAD_THREADS_MAX] = +{ { __LOCK_INITIALIZER, &__pthread_initial_thread, 0}, + { __LOCK_INITIALIZER, &__pthread_manager_thread, 0}, /* All NULLs */ }; +
Sorry, folks - the above is only a partial patch. Here's a full replacement for the maxthreads patch: 2003-04-04 Don Howard <dhoward@redhat.com> * pthandles.c: Reintroduce the static thread handle initializer. GDB can't locate threads if __pthread_handles is uninitalized. See bugzilla 82640. 2002-08-08 Jakub Jelinek <jakub@redhat.com> * sysdeps/unix/sysv/linux/bits/local_lim.h (PTHREAD_THREADS_MAX): Bump to 8192. * manager.c (__pthread_handles): Remove. * pthandles.c: New file. * pthread.c (__pthread_initialize_minimal): Initialize __pthread_handles[0] and __pthread_handles[1]. * Makefile (libpthread-routines): Add pthandles (must be last). --- libc/linuxthreads/sysdeps/unix/sysv/linux/bits/local_lim.h.jj Tue Jun 13 01:29:21 2000 +++ libc/linuxthreads/sysdeps/unix/sysv/linux/bits/local_lim.h Sat Oct 26 18:00:59 2002 8of threads per process. */ #define _POSIX_THREAD_THREADS_MAX 64 /* This is the value this implementation supports. */ -#define PTHREAD_THREADS_MAX 1024 +#define PTHREAD_THREADS_MAX 8192 /* Maximum amount by which a process can descrease its asynchronous I/O priority level. */ --- libc/linuxthreads/manager.c.jj Tue Sep 18 07:38:24 2001 +++ libc/linuxthreads/manager.c Sat Oct 26 18:02:27 2002 5rt.h" #include "semaphore.h" -/* Array of active threads. Entry 0 is reserved for the initial thread. */ -struct pthread_handle_struct __pthread_handles[PTHREAD_THREADS_MAX] = -{ { __LOCK_INITIALIZER, &__pthread_initial_thread, 0}, - { __LOCK_INITIALIZER, &__pthread_manager_thread, 0}, /* All NULLs */ }; - /* For debugging purposes put the maximum number of threads in a variable. */ const int __linuxthreads_pthread_threads_max = PTHREAD_THREADS_MAX; --- libc/linuxthreads/Makefile.jj Wed Jul 25 17:24:09 2001 +++ libc/linuxthreads/Makefile Sat Oct 26 18:03:08 2002 @@ -36,7 +36,7 @@ libpthread-routines := attr cancel condv ptlongjmp pthread signals specific errno lockfile \ semaphore spinlock wrapsyscall rwlock pt-machine \ oldsemaphore events getcpuclockid pspinlock barrier \ - ptclock_gettime ptclock_settime + ptclock_gettime ptclock_settime pthandles nodelete-yes = -Wl,--enable-new-dtags,-z,nodelete initfirst-yes = -Wl,--enable-new-dtags,-z,initfirst --- libc/linuxthreads/pthread.c.jj Tue Sep 18 07:38:24 2001 +++ libc/linuxthreads/pthread.c Sat Oct 26 18:05:29 2002 @@ -390,6 +390,14 @@ extern void *__dso_handle __attribute__ void __pthread_initialize_minimal(void) { + /* First of all init __pthread_handles[0] and [1]. */ +#if __LT_SPINLOCK_INIT != 0 + __pthread_handles[0].h_lock = __LOCK_INITIALIZER; + __pthread_handles[1].h_lock = __LOCK_INITIALIZER; +#endif + __pthread_handles[0].h_descr = &__pthread_initial_thread; + __pthread_handles[1].h_descr = &__pthread_manager_thread; + /* If we have special thread_self processing, initialize that for the main thread now. */ #ifdef INIT_THREAD_SELF --- linuxthreads/pthandles.c.dh Fri Apr 4 09:11:57 2003 +++ linuxthreads/pthandles.c Fri Apr 4 09:18:35 2003 @@ -0,0 +1,9 @@ +#include <ldsodefs.h> +#include "pthread.h" +#include "internals.h" + +/* Array of active threads. Entry 0 is reserved for the initial thread. */ +struct pthread_handle_struct __pthread_handles[PTHREAD_THREADS_MAX] = +{ { __LOCK_INITIALIZER, &__pthread_initial_thread, 0}, + { __LOCK_INITIALIZER, &__pthread_manager_thread, 0}, /* All NULLs */ }; + 2003-04-04 Don Howard <dhoward@redhat.com> * pthandles.c: Reintroduce the static thread handle initializer. GDB can't locate threads if __pthread_handles is uninitalized. See bugzilla 82640. 2002-08-08 Jakub Jelinek <jakub@redhat.com> * sysdeps/unix/sysv/linux/bits/local_lim.h (PTHREAD_THREADS_MAX): Bump to 8192. * manager.c (__pthread_handles): Remove. * pthandles.c: New file. * pthread.c (__pthread_initialize_minimal): Initialize __pthread_handles[0] and __pthread_handles[1]. * Makefile (libpthread-routines): Add pthandles (must be last). --- libc/linuxthreads/sysdeps/unix/sysv/linux/bits/local_lim.h.jj Tue Jun 13 01:29:21 2000 +++ libc/linuxthreads/sysdeps/unix/sysv/linux/bits/local_lim.h Sat Oct 26 18:00:59 2002 8of threads per process. */ #define _POSIX_THREAD_THREADS_MAX 64 /* This is the value this implementation supports. */ -#define PTHREAD_THREADS_MAX 1024 +#define PTHREAD_THREADS_MAX 8192 /* Maximum amount by which a process can descrease its asynchronous I/O priority level. */ --- libc/linuxthreads/manager.c.jj Tue Sep 18 07:38:24 2001 +++ libc/linuxthreads/manager.c Sat Oct 26 18:02:27 2002 5rt.h" #include "semaphore.h" -/* Array of active threads. Entry 0 is reserved for the initial thread. */ -struct pthread_handle_struct __pthread_handles[PTHREAD_THREADS_MAX] = -{ { __LOCK_INITIALIZER, &__pthread_initial_thread, 0}, - { __LOCK_INITIALIZER, &__pthread_manager_thread, 0}, /* All NULLs */ }; - /* For debugging purposes put the maximum number of threads in a variable. */ const int __linuxthreads_pthread_threads_max = PTHREAD_THREADS_MAX; --- libc/linuxthreads/Makefile.jj Wed Jul 25 17:24:09 2001 +++ libc/linuxthreads/Makefile Sat Oct 26 18:03:08 2002 @@ -36,7 +36,7 @@ libpthread-routines := attr cancel condv ptlongjmp pthread signals specific errno lockfile \ semaphore spinlock wrapsyscall rwlock pt-machine \ oldsemaphore events getcpuclockid pspinlock barrier \ - ptclock_gettime ptclock_settime + ptclock_gettime ptclock_settime pthandles nodelete-yes = -Wl,--enable-new-dtags,-z,nodelete initfirst-yes = -Wl,--enable-new-dtags,-z,initfirst --- libc/linuxthreads/pthread.c.jj Tue Sep 18 07:38:24 2001 +++ libc/linuxthreads/pthread.c Sat Oct 26 18:05:29 2002 @@ -390,6 +390,14 @@ extern void *__dso_handle __attribute__ void __pthread_initialize_minimal(void) { + /* First of all init __pthread_handles[0] and [1]. */ +#if __LT_SPINLOCK_INIT != 0 + __pthread_handles[0].h_lock = __LOCK_INITIALIZER; + __pthread_handles[1].h_lock = __LOCK_INITIALIZER; +#endif + __pthread_handles[0].h_descr = &__pthread_initial_thread; + __pthread_handles[1].h_descr = &__pthread_manager_thread; + /* If we have special thread_self processing, initialize that for the main thread now. */ #ifdef INIT_THREAD_SELF --- linuxthreads/pthandles.c.dh Fri Apr 4 09:11:57 2003 +++ linuxthreads/pthandles.c Fri Apr 4 09:18:35 2003 @@ -0,0 +1,9 @@ +#include <ldsodefs.h> +#include "pthread.h" +#include "internals.h" + +/* Array of active threads. Entry 0 is reserved for the initial thread. */ +struct pthread_handle_struct __pthread_handles[PTHREAD_THREADS_MAX] = +{ { __LOCK_INITIALIZER, &__pthread_initial_thread, 0}, + { __LOCK_INITIALIZER, &__pthread_manager_thread, 0}, /* All NULLs */ }; +
I have isolated the libthread_db patch that should have been backported along with the problematic libpthread patch (linuxthreads_db/td_ta_thr_iter.c). I'll work on an update with Jakub. For those playing along at home, getting that file from mainline glibc (or any rpm since glibc-2.2.90-24) should work.
Should be fixed in glibc-2.2.4-32.3.
*** Bug 88698 has been marked as a duplicate of this bug. ***
An errata has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on the solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2003-090.html
glibc-2.2.4-32.3 does not correct this.
Initial customer feedback is that this patch appears to correct the issue. Will follow up if further testing does not corroborate this.
I've just installed Linux ES 2.1 Update 2, which comes pre-packaged with glibc- 2.2.24-32.3 (as well as gdb-5.2-2) and have the exact same problem - can't debug threaded applications: Error while reading shared library symbols: Cannot get thread info: generic error Cannot find user-level thread for LWP 17127: no LWP to satisfy query
We have two machines running RHAS and saw the bug listed above. On one machine the i386 version of glibc-2.2.4-32.3.i386.rpm was installed top fix the problem on the other the i686 version was installed. The i686 rpm does correct this problem (gdb works correctly), the i386 rpm does not (gdb reports errors). Someone should look into the build differences between those two packages. - What rpm command string are people supposed to use to identify *exactly* which rpm's they have installed? rpm -qa is certainly not enough, because these machines both show the same package list. rpm -V checks them out as clean... yet a simple ls -l /lib/libpthread* shows they have very different binaries. - Could someone explain why i686 versions of -devel, and -profile are not available?
a) e.g. rpm -q --qf '%{name}-%{version}-%{release}.%{arch}\n' glibc glibc-devel # etc. b) i686 -devel/-profile packages are both undesirable and not needed glibc-devel package contains headers, which are the same between the two arches, symbolic links to shared libraries (again, the same) and .a libraries (linking statically against i686+ libraries would only cause confusion, because such binaries could not be run on any previous CPU)
glibc-2.2.4-32.3.i686 as applied by up2date on ES 2.1 does not correct the problem. My ES distribution came with 2.2.4-31.7. An earlier comment noted that downgrading to glibc-2.2.4-29.2 corrects the problem, but I am unable to find anything other than a SRPM for that version. Is there a binary RPM for that version? This is a show-stopper for me.
Also note the following: [root@NCD11081363 root]# rpm -q --qf '%{name}-%{version}-%{release}.%{arch}\n' glibc-devel glibc-devel-2.2.4-32.3.i386 [root@NCD11081363 root]# rpm -q --qf '%{name}-%{version}-%{release}.%{arch}\n' glibc-common glibc-common-2.2.4-32.3.i386 [root@NCD11081363 root]# ll /usr/lib/libpthread* -rw-r--r-- 1 root root 1682770 Apr 9 03:59 /usr/lib/libpthread.a lrwxrwxrwx 1 root root 25 Jul 1 08:26 /usr/lib/libpthread.so -> ../../lib/libpthread.so.0 [root@NCD11081363 root]# ll /lib/libpthread* -rwxr-xr-x 1 root root 71872 Apr 9 04:43 /lib/libpthread-0.9.so lrwxrwxrwx 1 root root 17 Jul 1 08:25 /lib/libpthread.so.0 -> libpthread-0.9.so [root@NCD11081363 root]# ll /usr/lib/libthread* lrwxrwxrwx 1 root root 27 Jul 1 08:26 /usr/lib/libthread_db.so -> ../../lib/libthread_db.so.1 [root@NCD11081363 root]# ll /lib/libthread* -rwxr-xr-x 1 root root 264245 Apr 9 03:59 /lib/libthread_db-1.0.so lrwxrwxrwx 1 root root 19 Jul 1 08:25 /lib/libthread_db.so.1 -> libthread_db-1.0.so
ftp://people.redhat.com/jakub/glibc/errata/2.2.4-32.7/ if you want to see thepackages earlier than when they complete the QA process.
If you download the source rpm, it looks like the initialization in linuxthreads/pthandles.c is not included. All the rest of the patch shown above is there - including the initialization performed in __pthread_initialize_minimal(). When I added those two lines into pthandles.c ( via glibc-2.2.4-maxthread.patch ) ; rebuilt the RPM; slotted the resulting /lib/i686/libpthread-0.9.so into place; and ran ldconfig; this resolved this particular problem on my machine ( i686; AS2.1; gdb5.1-1; gcc 2.96 ). ( I previously did the above without adding those two lines and still had the problem, so the recompilation itself was not a solution ).
An errata has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on the solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2003-249.html