(I'm not sure if this is gdb or debuginfo..) I'm debugging a crash with coredumpctl debug and I'm in gdb and working up the backtrace, and it shows me: (gdb) up Downloading 190.70 K source file /usr/src/debug/glib2-2.86.0-2.fc43.x86_64/redhat-linux-build/../glib/gmain.c #4 0x00007fdb977d49cd in g_idle_dispatch (source=0x558333c46060, callback=0x55831d084f10 <notify_show_windows>, user_data=0x558333c615b0) at ../glib/gmain.c:6466 6466 again = callback (user_data); (gdb) list 6461 once_callback (user_data); 6462 again = G_SOURCE_REMOVE; 6463 } 6464 else 6465 { 6466 again = callback (user_data); 6467 } 6468 6469 TRACE (GLIB_IDLE_DISPATCH (source, source->context, callback, user_data, again)); 6470 OK, so debuginfod found the file and is showing it great, but I want to see what's above in the same file: (gdb) list 6450 warning: 6445 ../glib/gmain.c: No such file or directory (gdb) so something in the backtrace knew how to find the debuginfod file, but 'list' doesn't. The file is actually in .cache/debuginfod_client/17844d70d31f26f7fa9495a6b49840a8dc4028bc/source-c43fc6 f2-#usr#src#debug#glib2-2.86.0-2.fc43.x86_64#redhat-linux-build#..#glib#gmain.c and above we saw: Downloading 190.70 K source file /usr/src/debug/glib2-2.86.0-2.fc43.x86_64/redhat-linux-build/../glib/gmain.c but then we saw: #4 0x00007fdb977d49cd in g_idle_dispatch (source=0x558333c46060, callback=0x55831d084f10 <notify_show_windows>, user_data=0x558333c615b0) at ../glib/gmain.c:6466 ^^^^^^^^^^^^^^^ and it's that shortened name that list is trying to use Reproducible: Always Steps to Reproduce: 1. run a program that crashes 2. cordeumpctl debug (actually I think plain gdb the program would be fine) 3. walk up the backtrace a bit with 'up' 4. list shows you a few lines 5. now try an explicit line number in the same file but a few lines earlier Actual Results: (gdb) list 6450 warning: 6445 ../glib/gmain.c: No such file or directory Expected Results: The code listing
We'll take a look...
Created attachment 2110452 [details] A failed attempt to reproduce this issue. I had a go at reproducing this bug. Attached is an (AI generated) test case (glib_idle_crash.c), which passes through the same `g_idle_dispatch` frame that you are having problems with. Here's my session on an x86-64 Fedora 43 install: ``` $ gcc -o glib_idle_crash glib_idle_crash.c $(pkg-config --cflags --libs glib-2.0) -g $ ./glib_idle_crash Starting the glib idle callback crash test program. Registering an idle callback that will cause a SIGSEGV. Running the GMainLoop. The crash should happen shortly... Idle callback entered. Preparing to crash... Segmentation fault (core dumped) ./glib_idle_crash $ rm -fr ~/.cache/ $ coredumpctl debug ... snip lots of noise ... This GDB supports auto-downloading debuginfo from the following URLs: <ima:enforcing> <https://debuginfod.fedoraproject.org/> <ima:ignore> Enable debuginfod for this session? (y or [n]) y Debuginfod has been enabled. To make this setting permanent, add 'set debuginfod enabled on' to .gdbinit. Downloading 3.55 M separate debug info for /lib64/libglib-2.0.so.0 Downloading 1.10 M separate debug info for /home/andrew/.cache/debuginfod_client/17844d70d31f26f7fa9495a6b49840a8dc4028bc/debuginfo Downloading 6.69 M separate debug info for /lib64/libc.so.6 Downloading 1.63 M separate debug info for /lib64/libpcre2-8.so.0 Downloading 35.09 K separate debug info for /home/andrew/.cache/debuginfod_client/57fa7ab6294a1aaad525f569ef5b02654d0009b6/debuginfo Downloading 57.17 K separate debug info for system-supplied DSO at 0x7feba42b4000 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `./glib_idle_crash'. Program terminated with signal SIGSEGV, Segmentation fault. #0 crashing_idle_callback (user_data=0x0) at glib_idle_crash.c:15 15 *((volatile int*)NULL) = 0; (gdb) up Downloading 190.70 K source file /usr/src/debug/glib2-2.86.0-2.fc43.x86_64/redhat-linux-build/../glib/gmain.c #1 0x00007feba41949cd in g_idle_dispatch (source=0x15402b10, callback=0x4004d6 <crashing_idle_callback>, user_data=0x0) at ../glib/gmain.c:6466 6466 again = callback (user_data); (gdb) list 6461 once_callback (user_data); 6462 again = G_SOURCE_REMOVE; 6463 } 6464 else 6465 { 6466 again = callback (user_data); 6467 } 6468 6469 TRACE (GLIB_IDLE_DISPATCH (source, source->context, callback, user_data, again)); 6470 (gdb) list 6450 6445 GSourceFunc callback, 6446 gpointer user_data) 6447 { 6448 GIdleSource *idle_source = (GIdleSource *)source; 6449 gboolean again; 6450 6451 if (!callback) 6452 { 6453 g_warning ("Idle source dispatched without callback. " 6454 "You must call g_source_set_callback()."); (gdb) q $ find ~/.cache -name \*gmain.c | xargs wc -l 6800 /home/andrew/.cache/debuginfod_client/17844d70d31f26f7fa9495a6b49840a8dc4028bc/source-c43fc6f2-#usr#src#debug#glib2-2.86.0-2.fc43.x86_64#redhat-linux-build#..#glib#gmain.c 4 /home/andrew/.cache/debuginfod_client/17844d70d31f26f7fa9495a6b49840a8dc4028bc/hdr-source-c43fc6f2-#usr#src#debug#glib2-2.86.0-2.fc43.x86_64#redhat-linux-build#..#glib#gmain.c 6804 total ``` In this experiment I deleted my ~/.cache directory before restarting GDB, this forced GDB to re-download all the content via debuginfod. Maybe you could try the above experiment and see if you still see the same problem. If the above works, then is your original test still broken? Or has clearing the cache got it working again? Within GDB I believe the same code is used for printing source lines for the backtrace and for the list command, so I don't think it is the case that GDB is using the wrong filename for the `list` command. Especially as your first `list` with no line number worked. It almost looks like GDB was suddenly unable to read the file contents from the debuginfod cache for some reason. I wonder, is there anything interesting about the location of your .cache folder? For me the default ~/.cache is used, but If you start GDB and do `info sources` you'll be able to see exactly where the debuginfod files are being cached – is this on a network share for example where the connection might be unreliable? If the bug is still reproducible for you, then I think you might need to run GDB under strace, something like: ``` $ coredumpctl dump > corefile ... snip ... $ strace -o trace -s50000 gdb -c corefile ... carry out steps to reproduce bug ... ``` Then attach the `trace` file that was created to the bug, this might give some insight into where GDB was trying to access, and maybe even, why the access failed.
Hi Andrew (long time!), I can confirm for at least one of my coredumps I can reliably reproduce it with a clean cache; The trace is too chunky for bz, so I uploaded it to my website: https://www.treblig.org/debug/trace-cleancache.bz2 I did: rm -rf .cache/debuginfod_client strace -o trace-cleancache -s50000 gdb -c coredump.6189 .. Downloading 190.70 K source file /usr/src/debug/glib2-2.86.0-2.fc43.x86_64/redhat-linux-build/../glib/gmain.c #4 0x00007fdb977d49cd in g_idle_dispatch (source=0x558333c46060, callback=0x55831d084f10 <notify_show_windows>, user_data=0x558333c615b0) at ../glib/gmain.c:6466 6466 again = callback (user_data); (gdb) list 6461 once_callback (user_data); 6462 again = G_SOURCE_REMOVE; 6463 } 6464 else 6465 { 6466 again = callback (user_data); 6467 } 6468 6469 TRACE (GLIB_IDLE_DISPATCH (source, source->context, callback, user_data, again)); 6470 (gdb) list 6450 warning: 6445 ../glib/gmain.c: No such file or directory (gdb) q some notes I found in the trace (with line numbers in the trace file): I think this is it reading the source in the first time to print the 6466 line: 217641 openat(AT_FDCWD, "/home/dg/.cache/debuginfod_client/17844d70d31f26f7fa9495a6b49840a8dc4028bc/source-c43fc6f2-#usr#src#debug#glib2-2.86.0-2.fc43.x86_64#redhat-linux-build#..#glib#gmain.c", O_RDONLY|O_CLOEXEC) = 11 217649 read(11, .... 217650 close(11) = 0 217660 write(1, "\33[2m6466\33[m\t again \33[31m=\33[m \33[01mcallback\33[m \33[31m(\33[muser_data\33[31m);\33[m\n", 80) = 80 then I type 'list' and I see it showing 6461 etc: 217798 write(1, "\33[2m6461\33[m\t \33[01monce_callback\33[m \33[31m(\33[muser_data\33[31m);\33[m\n", 69) = 69 then I type list 6450: 220551 openat(AT_FDCWD, "/home/dg/.cache/debuginfod_client/153f53f1c5f8fe56bf11f1ea17f5ba7fd683bb61/source-c43fc6f2-#usr#src#debug#glib2-2.86.0-2.fc43.x86_64#redhat-linux-build#..#glib#gmain.c", O_RDONLY) = 107 220552 fstat(107, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 220553 close(107) = 0 220554 openat(AT_FDCWD, "/home/dg/.cache/debuginfod_client/cache_miss_s", O_RDWR|O_CREAT, 0666) = 107 220555 fstat(107, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 220556 write(107, "600", 3) = 3 220557 close(107) = 0 220558 unlink("/home/dg/.cache/debuginfod_client/153f53f1c5f8fe56bf11f1ea17f5ba7fd683bb61/source-c43fc6f2-#usr#src#debug#glib2-2.86.0-2.fc43.x86_64#redhat-linux-build#..#glib#gmain.c.XXXXXX") = -1 ENOENT (No such file or directory) 220559 unlink("/home/dg/.cache/debuginfod_client/153f53f1c5f8fe56bf11f1ea17f5ba7fd683bb61/hdr-source-c43fc6f2-#usr#src#debug#glib2-2.86.0-2.fc43.x86_64#redhat-linux-build#..#glib#gmain.c.XXXXXX") = -1 ENOENT (No such file or directory) 220560 ioctl(1, TCGETS2, {c_iflag=ICRNL|IXON|IUTF8, c_oflag=NL0|CR0|TAB0|BS0|VT0|FF0|OPOST|ONLCR, c_cflag=B38400|B38400<<IBSHIFT|CS8|CREAD, c_lflag=ISIG|ICANON|ECHO|ECHOE|ECHOK|IEXTEN|ECHOCTL|ECHOKE, ...}) = 0 220561 write(2, "warning: ", 9) = 9 although I'm a bit confused since the one it's using at 220551 is 153f53... which is 0 bytes long, but the one it's using earlier is 17844d7.. which is the full file.
I also tried gdb'ing the gdb and breakpointing on the unlink's we see at 220558, and I got the following bt which may or may not be related. One thing that confuses me there is on the unlinks the name ends in .XXXXXX where I'm used to mktmp and friends replacing the XXXXXX by the name it want to use. (gdb) bt full #0 __unlink ( name=0x5595bf27d3c0 "/home/dg/.cache/debuginfod_client/153f53f1c5f8fe56bf11f1ea17f5ba7fd683bb61/source-c43fc6f2-#usr#src#debug#glib2-2.86.0-2.fc43.x86_64#redhat-linux-build#..#glib#gmain.c.XXXXXX") at ../sysdeps/unix/sysv/linux/unlink.c:26 No locals. #1 0x00007fd18a0941ce in debuginfod_query_server_by_buildid (c=<optimized out>, build_id=<optimized out>, build_id_len=<optimized out>, type=<optimized out>, type_arg=<optimized out>, path=<optimized out>) at /usr/src/debug/elfutils-0.193-3.fc43.x86_64/debuginfod/debuginfod-client.c:2541 server_urls = <optimized out> urls_envvar = <optimized out> section = <optimized out> filename = <optimized out> cache_path = <optimized out> maxage_path = 0x5595bfb2c3a0 "l\347H\234\220U" interval_path = 0x5595c54120a0 "ҞV\234\220U" cache_miss_path = 0x5595c4f32cb0 "\222\214\356\346\220U" target_cache_dir = 0x5595c6177200 "\367\021K\237\220U" target_cache_path = 0x5595bf356440 "V\363[Y\005" target_cachehdr_path = 0x5595bf3dda60 "/home/dg/.cache/debuginfod_client/153f53f1c5f8fe56bf11f1ea17f5ba7fd683bb61/hdr-source-c43fc6f2-#usr#src#debug#glib2-2.86.0-2.fc43.x86_64#redhat-linux-build#..#glib#gmain.c" target_cache_tmppath = 0x5595bf27d3c0 "/home/dg/.cache/debuginfod_client/153f53f1c5f8fe56bf11f1ea17f5ba7fd683bb61/source-c43fc6f2-#usr#src#debug#glib2-2.86.0-2.fc43.x86_64#redhat-linux-build#..#glib#gmain.c.XXXXXX" target_cachehdr_tmppath = 0x5595c59f0450 "/home/dg/.cache/debuginfod_client/153f53f1c5f8fe56bf11f1ea17f5ba7fd683bb61/hdr-source-c43fc6f2-#usr#src#debug#glib2-2.86.0-2.fc43.x86_64#redhat-linux-build#..#glib#gmain.c.XXXXXX" suffix = "c43fc6f2-#usr#src#debug#glib2-2.86.0-2.fc43.x86_64#redhat-linux-build#..#glib#gmain.c\000\000\000\016\000\000\000\000\000\000\000P\340\350\204\377\177\000\000T\017a\273\225U", '\000' <repeats 11 times>, "\266\224\371\211~D\347`\207뿕U\000\000\020\266$\300\225U\000\000\300\213\376\305\004\000\000\000\000\266\224\371\211~D\347\000\000\000\000\000\000\000\000\320\342\350\204\377\177\000\000\360\006\377ŕU\000\000 \a\377ŕU\000\000\002\000\000\000\000\000\000\000\260\371\376ŕ"... build_id_bytes = "153f53f1c5f8fe56bf11f1ea17f5ba7fd683bb61\000\177\304ŕU\000\000\240\340\370ŕU\000\000\200\337\350\204\377\177\000\000\000\367\036\300\225U\000\000\300\213\376ŕU\000\000p\337\350\204\377\177\000\000\371\003a\273\225U\000\000p\333\370ŕU\000\000\250\341\370ŕU\000\000\250\341\370ŕU\000\000\200\341\370ŕU\000", <incomplete sequence \320> vfd = <optimized out> rc = <optimized out> r = <optimized out> maxsize = <optimized out> maxsize_envvar = <optimized out> maxtime = <optimized out> --Type <RET> for more, q to quit, c to continue without paging-- maxtime_envvar = <optimized out> headers_file_envvar = <optimized out> fd = <optimized out> timeout = <optimized out> timeout_envvar = <optimized out> server_url_list = 0x7fff84e8df70 url_ima_policies = 0x5595c5fe8890 server_url = <optimized out> num_urls = -973546384 retry_limit = <optimized out> retry_limit_envvar = <optimized out> curlm = <optimized out> target_handle = 0x5595c5f8dc70 data = <optimized out> escaped_string = <optimized out> escaped_strlen = <optimized out> committed_to = -414941559 num_msg = -107694592 verified_handle = <optimized out> mtime = 94101756477312 curl_res = <optimized out> #2 0x00005595bb5c25f5 in debuginfod_source_query (build_id=<optimized out>, build_id_len=20, srcpath=0x5595c6246660 "/usr/src/debug/glib2-2.86.0-2.fc43.x86_64/redhat-linux-build/../glib/gmain.c", destname=destname@entry=0x7fff84e8f3b8) at ../../gdb/debuginfod-support.c:336 data = {desc = 0x5595bbcc688b "source file", fname = 0x5595c6246660 "/usr/src/debug/glib2-2.86.0-2.fc43.x86_64/redhat-linux-build/../glib/gmain.c", progress = {m_uiout = 0x5595bed42e10}} c = 0x5595bef7cd10 dname = 0x0 fd = {m_fd = -1} term_state = std::optional [no contained value] #3 0x00005595bb9054a5 in open_source_file (s=s@entry=0x5595c4fcd6c0) at ../../gdb/source.c:1199 query_fd = {m_fd = 0} ofp = 0x5595bfe887a0 srcpath = "/usr/src/debug/glib2-2.86.0-2.fc43.x86_64/redhat-linux-build/../glib/gmain.c" build_id = <optimized out> fullname = std::unique_ptr<char> = {get() = 0x0} --Type <RET> for more, q to quit, c to continue without paging-- fd = {m_fd = -2} #4 0x00005595bb90602d in print_source_lines_base (s=0x5595c4fcd6c0, line=6445, stopline=6455, flags=...) at ../../gdb/source.c:1339 desc = {m_fd = 0} noprint = false errcode = 2 nlines = 10 uiout = 0x5595bed42e10 loc = 0x5595c53bd850 lines = "" iter = <optimized out> new_lineno = <optimized out> #5 0x00005595bb906754 in print_source_lines (s=<optimized out>, line_range=..., flags=...) at ../../gdb/source.c:1492 No locals. #6 0x00005595bb551413 in list_command (arg=<optimized out>, from_tty=<optimized out>) at ../../gdb/cli/cli-cmds.c:1480 first_line = <optimized out> i = 0 sym = <optimized out> arg1 = 0x5595c5ab8cb9 "" no_end = <optimized out> dummy_end = <optimized out> dummy_beg = <optimized out> linenum_beg = <optimized out> p = <optimized out> __func__ = "list_command" sals = <optimized out> sal = {pspace = <optimized out>, symtab = 0x5595c4fcd6c0, symbol = <optimized out>, section = <optimized out>, msymbol = <optimized out>, line = <optimized out>, pc = <optimized out>, end = <optimized out>, explicit_pc = <optimized out>, explicit_line = <optimized out>, is_stmt = <optimized out>, prob = <optimized out>, objfile = <optimized out>} sal_end = <optimized out> beg = <optimized out> beg_len = <optimized out> #7 0x00005595bb555068 in cmd_func (cmd=<optimized out>, args=<optimized out>, from_tty=<optimized out>) at ../../gdb/cli/cli-decode.c:2748 restore_suppress = Python Exception <class 'gdb.error'>: value has been optimized out {<std::_Optional_base<scoped_restore_tmpl<bool>, false, false>> = { _M_payload = {<std::_Optional_payload<scoped_restore_tmpl<bool>, true, false, false>> = {<std::_Optional_payload_base<scoped_restore_tmpl<bool> >> = {_M_payload = {_M_empty = {<No data fields>}, _M_value = {<scoped_restore_base> = {m_saved_var = 0x0}, m_saved_value = false}}, _M_engaged = <optimized out>}, <No data fields>}, <No data fields>}}, <std::_Enable_copy_move<true, false, true, false, std::optional<scop--Type <RET> for more, q to quit, c to continue without paging--c ed_restore_tmpl<bool> > >> = {<No data fields>}, <No data fields>} #8 0x00005595bb99381f in execute_command (p=<optimized out>, p@entry=0x5595c5ab8cb0 "list 6450", from_tty=<optimized out>) at ../../gdb/top.c:570 default_args_and_arg = "" without_whitespace = "" cmd2 = 0x68fa1c10 <error: Cannot access memory at address 0x68fa1c10> cmd = 0x5595c5ab8cb5 "6450" arg = 0x5595c5ab8cb5 "6450" default_args = "" was_sync = <optimized out> save_repeat_args = <optimized out> args_pointer = <optimized out> c = 0x5595bea7c390 line = 0x5595c5ab8cb0 "list 6450" cmd_start = <optimized out> cleanup_if_error = <optimized out> cleanup = {m_value = <optimized out>, m_freed = false} cmd_copy = "list 6450" __func__ = "execute_command" #9 0x00005595bb659ac0 in command_handler (command=0x5595c5ab8cb0 "list 6450") at ../../gdb/event-top.c:613 ui = 0x5595be8a8260 c = <optimized out> stat_reporter = {m_msg_type = true, m_time_enabled = false, m_space_enabled = false, m_symtab_enabled = false, m_start_cpu_time = run_time_clock time_point = { 0us }, m_start_wall_time = std::chrono::_V2::steady_clock time_point = { 0ns }, m_start_space = 94101582490929, m_start_nr_symtabs = -2065106912, m_start_nr_compunit_symtabs = 32767, m_start_nr_blocks = -1098218880} #10 0x00005595bb65b9e9 in command_line_handler (rl=...) at ../../gdb/event-top.c:849 scope_exit_847 = <optimized out> line_buffer = "list 6450" ui = 0x5595be8a8260 cmd = <optimized out> #11 0x00005595bb65a426 in gdb_rl_callback_handler (rl=0x5595bef7d0d0 "list 6450") at ../../gdb/event-top.c:288 gdb_rl_expt = {reason = 0, error = GDB_NO_ERROR, message = std::shared_ptr<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >> (empty) = {get() = 0x0}} ui = 0x5595be8a8260 #12 0x00007fd18a2e7b15 in rl_callback_read_char () at ../callback.c:302 line = 0x5595bef7d0d0 "list 6450" eof = <optimized out> jcode = <optimized out> olevel = {{__jmpbuf = {94101635236448, -1294597897630532193, 0, 94101595098336, 4294967295, 94101592330104, -1294597897624240737, -1273190900505657953}, __mask_was_saved = 0, __saved_mask = {__val = {0 <repeats 16 times>}}}} #13 0x00005595bb65a58e in gdb_rl_callback_read_char_wrapper_sjlj () at ../../gdb/event-top.c:197 gdb_expt = {reason = 0, error = GDB_NO_ERROR, message = std::shared_ptr<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >> (empty) = {get() = 0x0}} #14 0x00005595bb65a753 in gdb_rl_callback_read_char_wrapper_noexcept () at ../../gdb/event-top.c:240 No locals. #15 gdb_rl_callback_read_char_wrapper (client_data=<optimized out>) at ../../gdb/event-top.c:252 gdb_expt = {reason = 0, error = GDB_NO_ERROR, message = std::shared_ptr<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >> (empty) = {get() = 0x0}} #16 0x00005595bb9d6930 in stdin_event_handler (error=<optimized out>, client_data=0x5595be8a8260) at ../../gdb/ui.c:154 ui = 0x5595be8a8260 #17 0x00005595bbc53f66 in gdb_wait_for_event (block=block@entry=1) at ../../gdbsupport/event-loop.cc:694 mask = <optimized out> file_ptr = <optimized out> num_found = <optimized out> __func__ = "gdb_wait_for_event" #18 0x00005595bbc54ccc in gdb_do_one_event (mstimeout=mstimeout@entry=-1) at ../../gdbsupport/event-loop.cc:263 event_source_head = 0 number_of_sources = 3 current = <optimized out> timer_id = std::optional [no contained value] scope_exit_250 = <optimized out> #19 0x00005595bb745fca in start_event_loop () at ../../gdb/main.c:402 result = 0 #20 captured_command_loop () at ../../gdb/main.c:466 ui = 0x5595be8a8260 #21 0x00005595bb7491e5 in captured_main (data=0x7fff84e8fb10) at ../../gdb/main.c:1343 context = 0x7fff84e8fb10 #22 gdb_main (args=args@entry=0x7fff84e8fb50) at ../../gdb/main.c:1362 No locals. #23 0x00005595bb417856 in main (argc=3, argv=0x7fff84e8fca8) at ../../gdb/gdb.c:38 __func__ = "main" args = {argc = 3, argv = 0x7fff84e8fca8, interpreter_p = 0x5595bbca4aa5 "console"} (gdb) c Continuing. Thread 1 "gdb" hit Breakpoint 1.5, __unlink ( name=0x5595c59f0450 "/home/dg/.cache/debuginfod_client/153f53f1c5f8fe56bf11f1ea17f5ba7fd683bb61/hdr-source-c43fc6f2-#usr#src#debug#glib2-2.86.0-2.fc43.x86_64#redhat-linux-build#..#glib#gmain.c.XXXXXX") at ../sysdeps/unix/sysv/linux/unlink.c:26 26 {
Observation: /lib64/libglib-2.0.so.0.8600.0: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=17844d70d31f26f7fa9495a6b49840a8dc4028bc, stripped that's the one that works. /lib64/libgobject-2.0.so.0.8600.0: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=153f53f1c5f8fe56bf11f1ea17f5ba7fd683bb61, stripped is the one that fails. they're both part of the same glib2-2.86.0-2.fc43.x86_64 binary package. but I think it's actually probably the same source file it's trying to read
Created attachment 2110662 [details] A modified test program that triggers the problem I've modified the AI created crasher, this cyborg triggers the crash using a gobject call in the idle function. Based on my previous comment, I think the trick is to cause debuginfo to fetch the gobject stuff in the backtrace as well as the glib stuff dg@dalek:~/bugs/gdb-bz-2403580$ gdb ./cyborg GNU gdb (Fedora Linux) 16.3-6.fc43 Copyright (C) 2024 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <https://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from ./cyborg... (gdb) r Starting program: /home/dg/bugs/gdb-bz-2403580/cyborg This GDB supports auto-downloading debuginfo from the following URLs: <ima:enforcing> <https://debuginfod.fedoraproject.org/> <ima:ignore> Enable debuginfod for this session? (y or [n]) y Debuginfod has been enabled. To make this setting permanent, add 'set debuginfod enabled on' to .gdbinit. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Starting the glib idle callback crash test program. Running the GMainLoop. The crash should happen shortly... Idle callback entered. Preparing to crash... Program received signal SIGSEGV, Segmentation fault. g_type_check_instance_is_a (type_instance=<optimized out>, iface_type=Python Exception <class 'gdb.error'>: value has been optimized out ) at ../gobject/gtype.c:3905 3905 check = node && node->is_instantiatable && iface && type_node_conforms_to_U (node, iface, TRUE, FALSE); (gdb) up #1 0x000000000040058c in crashing_idle_callback (user_data=0x0) at cyborg.c:21 21 fprintf(stderr, "Result: %d\n", G_IS_BINDING(&doom)); (gdb) #2 0x00007ffff7e1b9cd in g_idle_dispatch (source=0x40ddb0, callback=0x400506 <crashing_idle_callback>, user_data=0x0) at ../glib/gmain.c:6466 6466 again = callback (user_data); (gdb) list 6461 once_callback (user_data); 6462 again = G_SOURCE_REMOVE; 6463 } 6464 else 6465 { 6466 again = callback (user_data); 6467 } 6468 6469 TRACE (GLIB_IDLE_DISPATCH (source, source->context, callback, user_data, again)); 6470 (gdb) where #0 g_type_check_instance_is_a (type_instance=<optimized out>, iface_type=Python Exception <class 'gdb.error'>: value has been optimized out ) at ../gobject/gtype.c:3905 #1 0x000000000040058c in crashing_idle_callback (user_data=0x0) at cyborg.c:21 #2 0x00007ffff7e1b9cd in g_idle_dispatch (source=0x40ddb0, callback=0x400506 <crashing_idle_callback>, user_data=0x0) at ../glib/gmain.c:6466 #3 0x00007ffff7e15263 in g_main_dispatch (context=0x40dbb0) at ../glib/gmain.c:3565 #4 g_main_context_dispatch_unlocked (context=0x40dbb0) at ../glib/gmain.c:4425 #5 0x00007ffff7e1e1b8 in g_main_context_iterate_unlocked (context=0x40dbb0, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../glib/gmain.c:4490 #6 0x00007ffff7e1e45f in g_main_loop_run (loop=0x40db70) at ../glib/gmain.c:4695 #7 0x000000000040064e in main () at cyborg.c:50 (gdb) list 6450 warning: 6445 ../glib/gmain.c: No such file or directory (gdb)
Looks like gdb is asking debuginfod for the right source file name, but with the wrong buildid (libgobject2's vs. libglib2's.) The former does not include compilation units from the gmain.c file at all.
(In reply to Frank Ch. Eigler from comment #8) > Looks like gdb is asking debuginfod for the right source file name, but with > the wrong buildid (libgobject2's vs. libglib2's.) The former does not > include compilation units from the gmain.c file at all. Sort of. libglib2 and libgobject2 share a DWZ file. The DWZ file's line table contains an entry for gmain.c. As a consequence, GDB ends up creating a symtab data structure for gmain.c associated with both libglib2 and libgobject2. When listing gmain.c, GDB manages to find the symtab in libgobject2 rather than the one in libglib2. Commit a736ff7d886dbcc85026264c3ce11c125a8409b2 actually seems to fix this issue. After this commit GDB finds the symtab from libglib2 first. What's not clear to me at the moment is whether this change is by design, or purely by luck. The commit message for the quoted commit seems to suggest the change in that commit should have no user visible impact (though it doesn't explicitly say that), but the commit also doesn't suggest that it's setting out to fix this (or any) specific bug. And after this commit the gmain.c symtab within libgobject2 still exists; so it's not clear to me if we could still end up finding that symtab in some other situation or not. I'm still actively looking into this issue, and hopefully will understand it better soon.
Hi! Work on this issue continues. The TLDR is there's no immediate fix ready to backport. If you are debugging a core file generated on the local machine then the best work around for this will be to install the debug information for glib onto the local machine (the command 'sudo dnf debuginfo-install glib' should do the job). Of course, this only helps if the debug information installed matches the version needed for the core file. If not, then this isn't going to help. In terms of a fix, upstream commit a736ff7d886dbcc85026264c3ce11c125a8409b2 will avoid triggering this problem, but this commit isn't really a fix, it just avoids the problematic code path. This commit is going to be part of GDB 18. The next release of GDB is 17, so this commit isn't going to appear any time soon. Further, this commit is part of a bigger series of DWARF reworking, I feel trying to backport this commit (plus dependencies) would be pretty risky. As part of looking at this issue I have also pushed upstream commit c1859f4b6863655c84646816f0885d316d35871b. This commit also isn't a fix for this issue, but it yet another commit which will avoid the problematic code path. This commit has no real dependencies, so could be backported. We are currently expecting GDB 17 to be released at some point before the end of 2026, and work to rebase Fedora will start in the new year. My current thinking is that we'll consider backporting c1859f4b6863655c84646816f0885d316d35871b once the rebase is complete. Meanwhile, I am still working on a real fix for the underlying issue. I believe I have something that might work, but it needs more testing, and is unlikely to be posted upstream before the new year now. If/when this is accepted, then we could certainly consider backporting this fix. Sorry not to have better news at this time.
Thanks for keeping digging at it - no rush from my side.
FEDORA-2026-0630a7be2f (gdb-17.1-3.fc43) has been submitted as an update to Fedora 43. https://bodhi.fedoraproject.org/updates/FEDORA-2026-0630a7be2f
FEDORA-2026-0630a7be2f has been pushed to the Fedora 43 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2026-0630a7be2f` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2026-0630a7be2f See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
(In reply to Fedora Update System from comment #13) > FEDORA-2026-0630a7be2f has been pushed to the Fedora 43 testing repository. > Soon you'll be able to install the update with the following command: > `sudo dnf upgrade --enablerepo=updates-testing --refresh > --advisory=FEDORA-2026-0630a7be2f` > You can provide feedback for this update here: > https://bodhi.fedoraproject.org/updates/FEDORA-2026-0630a7be2f > > See also https://fedoraproject.org/wiki/QA:Updates_Testing for more > information on how to test updates. Yep that works great; thanks Andrew! (Bodhi feedback given)
FEDORA-2026-0630a7be2f (gdb-17.1-3.fc43) has been pushed to the Fedora 43 stable repository. If problem still persists, please make note of it in this bug report.