python3-graph-tool-2.29-2.fc32.aarch64 contains a very large file: -rw------- 1 root root 4308811776 Nov 13 15:38 /usr/lib64/python3.8/site-packages/graph_tool/inference/libgraph_tool_inference.so.gdb-index-rbpeap Koji build: https://koji.fedoraproject.org/koji/buildinfo?buildID=1411278 The build log contains this: ../../gdb/dwarf-index-write.c:1340: internal-error: void assert_file_size(FILE*, size_t): Assertion `file_size == expected_size' failed. A problem internal to GDB has been detected, further debugging may prove unreliable. Quit this debugging session? (y or n) [answered Y; input not from terminal] This is a bug, please report it. For instructions, see: <http://www.gnu.org/software/gdb/bugs/>. /usr/bin/gdb-add-index: line 106: 22220 Aborted (core dumped) $GDB --batch -nx -iex 'set auto-load no' -ex "file $file" -ex "save gdb-index $dwarf5 $dir" gdb-add-index: gdb error generating index for /builddir/build/BUILDROOT/python-graph-tool-2.29-2.fc32.aarch64/usr/lib64/python3.8/site-packages/graph_tool/inference/libgraph_tool_inference.so It should really have failed the build instead of leaving around such a large file.
Bug 1745020 discusses a previous aarch64 corruption. However, the builder here is different (buildvm-aarch64-05.arm.fedoraproject.org).
(In reply to Florian Weimer from comment #1) > Bug 1745020 discusses a previous aarch64 corruption. However, the builder > here is different (buildvm-aarch64-05.arm.fedoraproject.org). Nope, the assert failure reproduces, even on x86_64.
This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle. Changing version to 32.
This message is a reminder that Fedora 32 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora 32 on 2021-05-25. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '32'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 32 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Fedora 32 changed to end-of-life (EOL) status on 2021-05-25. Fedora 32 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed.
*** Bug 2184150 has been marked as a duplicate of this bug. ***
I reproduced this problem by doing: 1. fedpkg co python-graph-tool 2. cd python-graph-tool 3. fedpkg mockbuild (which I found in bug 2184150) The build output shows: + /usr/bin/find-debuginfo -j6 --strict-build-id -m -i --build-id-seed 2.57-1.fc39 --unique-debug-suffix -2.57-1.fc39.x86_64 --unique-debug-src-base python-graph-tool-2.57-1.fc39.x86_64 --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 110000000 -S debugsourcefiles.list /builddir/build/BUILD/graph-tool-2.57 find-debuginfo: starting Extracting debug info from 16 files ../../gdb/dwarf2/index-write.c:1069: internal-error: assert_file_size: Assertion `file_size == expected_size' failed. A problem internal to GDB has been detected, further debugging may prove unreliable. (followed by an unusable backtrace) FWIW, I watched htop while it was running and saw that the machine used 33G of RAM at one point. It may have used more - I didn't watch that closely. In any case, the machine in question was not resource constrained.
Note that you can add %define _find_debuginfo_opts --verbose in the spec file to get a bit more verbose output: + /usr/bin/find-debuginfo -j2 --strict-build-id -m -i --build-id-seed 2.57-2.fc39 --unique-debug-suffix -2.57-2.fc39.x86_64 --unique-debug-src-base python-graph-tool-2.57-2.fc39.x86_64 --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 110000000 --verbose -S debugsourcefiles.list /builddir/build/BUILD/graph-tool-2.57 find-debuginfo: starting Extracting debug info from 16 files extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/clustering/libgraph_tool_clustering.so extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/centrality/libgraph_tool_centrality.so extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/correlations/libgraph_tool_correlations.so extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/draw/libgraph_tool_draw.so extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/draw/libgraph_tool_layout.so extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/draw/libgt_pycairo_aux.so extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/dynamics/libgraph_tool_dynamics.so extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/flow/libgraph_tool_flow.so extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/generation/libgraph_tool_generation.so extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/inference/libgraph_tool_inference.so extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/libgraph_tool_core.so extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/search/libgraph_tool_search.so extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/spectral/libgraph_tool_spectral.so extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/stats/libgraph_tool_stats.so extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/topology/libgraph_tool_topology.so extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/util/libgraph_tool_util.so ../../gdb/dwarf2/index-write.c:1069: internal-error: assert_file_size: Assertion `file_size == expected_size' failed. A problem internal to GDB has been detected, further debugging may prove unreliable. ----- Backtrace ----- 0x562a2e75cc20 ??? 0x562a2eb62104 ??? 0x562a2eb6240e ??? 0x562a2ed41304 ??? 0x562a2e81d473 ??? 0x562a2e81e430 ??? 0x562a2e82170e ??? 0x562a2e822b74 ??? 0x562a2e795ce4 ??? 0x562a2eb3ea94 ??? 0x562a2e96f599 ??? 0x562a2e96f681 ??? 0x562a2e971938 ??? 0x562a2e97251f ??? 0x562a2e67356e ??? ../../gdb/dwarf2/index-write.c:1069: internal-error: assert_file_size: Assertion `file_size == expected_size' failed. A problem internal to GDB has been detected, further debugging may prove unreliable. Quit this debugging session? (y or n) [answered Y; input not from terminal] 0x7fba58828149 ??? 0x7fba5882820a ??? 0x562a2e67c564 ??? 0xffffffffffffffff ??? --------------------- This is a bug, please report it. For instructions, see: <https://www.gnu.org/software/gdb/bugs/>. /usr/bin/gdb-add-index: line 159: 24580 Aborted (core dumped) $GDB --batch -nx -iex 'set auto-load no' -iex 'set debuginfod enabled off' -ex "file $file" -ex "save gdb-index $dwarf5 $dir" gdb-add-index: gdb error generating index for /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/inference/libgraph_tool_inference.so DWARF-compressing 16 files original debug info size: 5831472kB, size after compression: 5655276kB sepdebugcrcfix: Updated 16 CRC32s, 0 CRC32s did match. Creating .debug symlinks for symlinks to ELF files Copying sources found by 'debugedit -l' to /usr/src/debug/python-graph-tool-2.57-2.fc39.x86_64 6054 blocks find-debuginfo: done Unfortunately the backtrace is still unusable, but at least we now know which file gdb-add-index crashes on. libgraph_tool_inference.so
Within my mock chroot, after hiding gdb.minimal, installing plain old gdb (to load any additional dependencies), and then replacing it with one built with -O0 -g3, I can finally see: (gdb) up 8 #8 0x000000000074ea68 in assert_file_size (file=0x3fa52b0, expected_size=280673559) at ../../gdb/dwarf2/index-write.c:1069 1069 gdb_assert (file_size == expected_size); (gdb) p file_size $1 = 4575640855 (gdb) p expected_size $2 = 280673559 Wow - not even close! (So... not an off-by-one error somewhere.) Here's the full backtrace: (gdb) bt #0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44 #1 0x00007f9a35c8fb43 in __pthread_kill_internal (signo=6, threadid=<optimized out>) at pthread_kill.c:78 #2 0x00007f9a35c3e8ee in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 #3 0x00007f9a35c268ff in __GI_abort () at abort.c:79 #4 0x0000000000dd97fb in dump_core () at ../../gdb/utils.c:202 #5 0x0000000000dd9e1c in internal_vproblem(internal_problem *, const char *, int, const char *, typedef __va_list_tag __va_list_tag *) ( problem=0x1ae55a0 <internal_error_problem>, file=0x12081f8 "../../gdb/dwarf2/index-write.c", line=1069, fmt=0x12080f8 "%s: Assertion `%s' failed.", ap=0x7ffe853267b8) at ../../gdb/utils.c:450 #6 0x0000000000dd9ef2 in internal_verror ( file=0x12081f8 "../../gdb/dwarf2/index-write.c", line=1069, fmt=0x12080f8 "%s: Assertion `%s' failed.", ap=0x7ffe853267b8) at ../../gdb/utils.c:476 #7 0x00000000010a104a in internal_error_loc ( file=0x12081f8 "../../gdb/dwarf2/index-write.c", line=1069, fmt=0x12080f8 "%s: Assertion `%s' failed.") at ../../gdbsupport/errors.cc:58 #8 0x000000000074ea68 in assert_file_size (file=0x3fa52b0, expected_size=280673559) at ../../gdb/dwarf2/index-write.c:1069 #9 0x000000000074ec19 in write_gdbindex_1 (out_file=0x3fa52b0, cu_list=..., types_cu_list=..., addr_vec=..., symtab_vec=..., constant_pool=...) at ../../gdb/dwarf2/index-write.c:1119 #10 0x000000000074f4de in write_gdbindex (per_objfile=0x3f979a0, table=0x57e7fe0, out_file=0x3fa52b0, dwz_out_file=0x0) at ../../gdb/dwarf2/index-write.c:1273 #11 0x00000000007501d8 in write_dwarf_index (per_objfile=0x3f979a0, dir=0x7ffe85328f7d "/builddir/build/BUILDROOT/python-graph-tool-2.57-1.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/inference", basename=0x32c9827 "libgraph_tool_inference.so", dwz_basename=0x0, index_kind=dw_index_kind::GDB_INDEX) at ../../gdb/dwarf2/index-write.c:1493 #12 0x000000000075046b in save_gdb_index_command ( arg=0x7ffe85328f7d "/builddir/build/BUILDROOT/python-graph-tool-2.57-1.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/inference", from_tty=0) at ../../gdb/dwarf2/index-write.c:1548 #13 0x000000000065385a in do_simple_func ( args=0x7ffe85328f7d "/builddir/build/BUILDROOT/python-graph-tool-2.57-1.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/inference", from_tty=0, c=0x311c930) at ../../gdb/cli/cli-decode.c:95 #14 0x0000000000658545 in cmd_func (cmd=0x311c930, args=0x7ffe85328f7d "/builddir/build/BUILDROOT/python-graph-tool-2.57-1.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/inference", from_tty=0) at ../../gdb/cli/cli-decode.c:2543 #15 0x0000000000d5adfc in execute_command (p=0x7ffe85328ff2 "e", from_tty=0) at ../../gdb/top.c:690 #16 0x00000000009d5dba in catch_command_errors ( command=0xd5a7e3 <execute_command(char const*, int)>, arg=0x7ffe85328f6d "save gdb-index /builddir/build/BUILDROOT/python-graph-tool-2.57-1.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/inference", from_tty=0, do_bp_actions=true) at ../../gdb/main.c:513 #17 0x00000000009d5f9b in execute_cmdargs (cmdarg_vec=0x7ffe853276d0, file_type=CMDARG_FILE, cmd_type=CMDARG_COMMAND, ret=0x7ffe8532768c) at ../../gdb/main.c:608 #18 0x00000000009d73f1 in captured_main_1 (context=0x7ffe85327960) at ../../gdb/main.c:1299 #19 0x00000000009d7652 in captured_main (data=0x7ffe85327960) at ../../gdb/main.c:1320 #20 0x00000000009d76bd in gdb_main (args=0x7ffe85327960) at ../../gdb/main.c:1345 #21 0x000000000041971d in main (argc=11, argv=0x7ffe85327aa8) at ../../gdb/gdb.c:40
Nice! At a glance, it looks like the expected size is exactly 4 GiB, or 2^32 bytes, less than the actual size, as if it was accumulated in an unsigned 32-bit value that wrapped around.
(In reply to Ben Beasley from comment #10) > Nice! At a glance, it looks like the expected size is exactly 4 GiB, or 2^32 > bytes, less than the actual size, as if it was accumulated in an unsigned > 32-bit value that wrapped around. That's exactly right - see my next comment below...
Upstream GDB (14.0.50.20230714-git) shows the same behavior. The values for file_size and expected_size are also the same. But look at this... (gdb) up 8 #8 0x000000000064a7c9 in assert_file_size (file=0x2fef340, expected_size=280673559) at /ironwood1/sourceware-git/rawhide-master/bld/../../worktree-master/gdb/dwarf2/index-write.c:1071 1071 gdb_assert (file_size == expected_size); (gdb) ptype file_size type = const long (gdb) ptype expected_size type = unsigned long (gdb) p sizeof(file_size) $1 = 8 (gdb) p sizeof(expected_size) $2 = 8 It'd be nice if they were the same type, but an 8 byte long is big enough to hold the values, whether signed or not. Now for the interesting bit... (gdb) up #9 0x000000000064a97a in write_gdbindex_1 (out_file=0x2fef340, cu_list=..., types_cu_list=..., addr_vec=..., symtab_vec=..., constant_pool=...) at /ironwood1/sourceware-git/rawhide-master/bld/../../worktree-master/gdb/dwarf2/index-write.c:1121 1121 assert_file_size (out_file, total_len); (gdb) p total_len $3 = 280673559 (gdb) ptype total_len type = unsigned int (gdb) p sizeof(total_len) $4 = 4 (gdb) p total_len + ((long)1<<32) $6 = 4575640855 So, what's happened is that the length of the file in question exceeds that of an unsigned 32-bit offset. That $6 value matches file_size in assert_file_size(). In write_gdbindex_1, total_len is declared as follows: offset_type total_len = size_of_header; And, in gdb/dwarf2/index-common.h, I find: /* All offsets in the index are of this type. It must be architecture-independent. */ typedef uint32_t offset_type; The type of total_len could be changed to an 64-bit type, but at a number of points along the way, a 32-bit offset is saved in a "contents" buffer. If it overflows 32 bits, at some point it'll be wrong. However, if we were to change the type of total_len to a 64-bit type, we could at least check for 32-bit overflow and throw an error if that happens. Surely that's preferable to failing an assert, which causes an internal error to occur. I'll ponder it some more and try to come up with a patch...
This bug appears to have been reported against 'rawhide' during the Fedora Linux 39 development cycle. Changing version to 39.
This bug has been fixed upstream via the following commit: 98f6baad7c3 Throw error when creating an overly large gdb-index file It'll be fixed in Fedora GDB 14.1.