Bug 1960867 - thread debugging broken with gdb
Summary: thread debugging broken with gdb
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: glibc
Version: 34
Hardware: Unspecified
OS: Linux
unspecified
urgent
Target Milestone: ---
Assignee: Carlos O'Donell
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-05-15 17:55 UTC by Avi Kivity
Modified: 2021-07-21 03:02 UTC (History)
19 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-07-21 03:02:01 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Avi Kivity 2021-05-15 17:55:55 UTC
Description of problem:

Trying to debug a threaded program fails, because gdb can't find libthread_db.


Version-Release number of selected component (if applicable):


gdb-10.1-14.fc34.x86_64

How reproducible:


Steps to Reproduce:
1. compile an empty program (just main) with g++ -g -pthread test.cc
2. gdb a.out
3. issue the command 'start'

Actual results:

warning: Expected absolute pathname for libpthread in the inferior, but got .gnu_debugdata for /lib64/libpthread.so.0.
warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.

Accessing thread-local variables will not work.

Expected results:

No such warning.

Additional info:


Observed with g++ and clang++, and with ld.bfd and gold.

Comment 1 Robert Tucker 2021-05-29 07:32:53 UTC
Also getting this:

(gdb) run
Starting program: /usr/bin/vlc 
warning: Expected absolute pathname for libpthread in the inferior, but got .gnu_debugdata for /lib64/libpthread.so.0.
warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
VLC media player 4.0.0-dev Otto Chriek (revision 4.0.0-dev-15406-g183c8d9971)
[New LWP 14303]
[New LWP 14304]

Comment 2 Botond Dénes 2021-06-02 12:43:34 UTC
I have a variant of this:

Trying host libthread_db library: libthread_db.so.1.
Host libthread_db.so.1 resolved to: /lib64/libthread_db.so.1.
td_ta_new failed: application not linked with libthread
thread_db_load_search returning 0
Trying host libthread_db library: libthread_db.so.1.
Host libthread_db.so.1 resolved to: /lib64/libthread_db.so.1.
td_ta_new failed: application not linked with libthread
warning: Expected absolute pathname for libpthread in the inferior, but got .gnu_debugdata for /lib64/libpthread.so.0.
Trying host libthread_db library: /lib64/libthread_db.so.1.
td_ta_new failed: application not linked with libthread
thread_db_load_search returning 0


Looking at the gdb source, the error "warning: Expected absolute pathname for libpthread in the inferior, but got .gnu_debugdata for /lib64/libpthread.so.0." comes from a function which just tries to find the path to load libthread-db from. The other error "td_ta_new failed: application not linked with libthread" comes when initializing libthread-db. So it looks like there are two problems (which might be connected): an unexpected path to libpthread.so and libthread_db.so failing the initialization after having been found and loaded.

Comment 3 Botond Dénes 2021-06-02 15:31:30 UTC
For me installing glibc debug info made the problem go away for now:

sudo dnf debuginfo-install glibc-2.32-6.fc33.x86_64

Comment 4 Kevin Buettner 2021-07-12 03:04:42 UTC
It turns out that this is a problem in both Fedora 33 and Fedora 34.  I've investigated and have found that the symbol 'nptl_version' is not present in .symtab or .dynsym in libpthread (or even libc on F34). GDB (on behalf of td_ta_new in libthread_db) needs to be able to find the address of nptl_version in order for libthread_db to be loaded / initialized.

To verify this, run the following commands on the specified Fedora releases:

Fedora 32:

[kev@f32-1 ~]$ objdump -t -T /lib64/libpthread-2.31.so | grep nptl_version
0000000000017037 l     O .rodata	0000000000000005              nptl_version

This shows that nptl_version is present in either .symtab (-t switch) or .dynsym (-T switch) in the libpthread library.

Fedora 33:

[kev@f33-1 ~]$ objdump -t -T /lib64/libpthread-2.32.so | grep nptl_version
[kev@f33-1 ~]$ 

I.e. no nptl_version symbol found.

Fedora 34:

[kev@f34-1 ~]$ objdump -t -T /lib64/libpthread-2.33.so | grep nptl_version
[kev@f34-1 ~]$ objdump -t -T /lib64/libc-2.33.so | grep nptl_version

Likewise, nptl_version wasn't found in Fedora 34.

Rawhide (with glibc-2.33.9000-37 installed):

[kev@rawhide-1 ~]$ objdump -t -T /lib64/libc.so.6 | grep nptl_version
00000000001b5688 g     O .rodata	000000000000000a              __nptl_version
00000000001b5688 g    DO .rodata	000000000000000a  GLIBC_PRIVATE __nptl_version


Note that the symbol name has changed somewhat, but it's still there. I've tested GDB and have found that it works as expected without debuginfo packages installed (and also with debuginfod disabled, plus cached debuginfod packages deleted).

So... Fedora 33 and Fedora 34 need fixing.

Comment 5 Florian Weimer 2021-07-13 17:02:03 UTC
Kevin, would you please test the latest Fedora 33 and 34 builds?

Fedora 33: glibc-2.32-10.fc33
Fedora 34: glibc-2.33-20.fc34

I expect the issue has been fixed there under bug 1975895. Thanks.

Comment 6 Kevin Buettner 2021-07-21 03:02:01 UTC
I've tested gdb.threads/tls.exp and gdb.threads/linux-dp.exp on Fedora 33 and Fedora 34. On both OSes, I removed the glibc debuginfo packages. No failures from any of the tests.

I've also checked to see whether nptl_version is present in libpthread as described in Comment 4. I found the following (which is good):

Fedora 33:

[kev@f33-1 gdb]$ objdump -t -T /lib64/libpthread-2.32.so | grep nptl_version
0000000000017037 l     O .rodata	0000000000000005              nptl_version

Fedora 34:

[kev@f34-1 gdb]$ objdump -t -T /lib64/libpthread-2.33.so | grep nptl_version
0000000000016037 l     O .rodata	0000000000000005              nptl_version

So... it looks fixed to me.  I'm closing it.


Note You need to log in before you can comment on or make changes to this bug.