Bug 1773651 - gdb: Large .gdb-index file created when building python-graph-tool
Summary: gdb: Large .gdb-index file created when building python-graph-tool
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: Fedora
Classification: Fedora
Component: gdb
Version: 39
Hardware: All
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Kevin Buettner
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 2184150 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-11-18 16:16 UTC by Florian Weimer
Modified: 2023-09-18 20:50 UTC (History)
7 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2023-09-18 20:50:21 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Florian Weimer 2019-11-18 16:16:15 UTC
python3-graph-tool-2.29-2.fc32.aarch64 contains a very large file:

-rw-------    1 root    root               4308811776 Nov 13 15:38 /usr/lib64/python3.8/site-packages/graph_tool/inference/libgraph_tool_inference.so.gdb-index-rbpeap

Koji build: https://koji.fedoraproject.org/koji/buildinfo?buildID=1411278

The build log contains this:

../../gdb/dwarf-index-write.c:1340: internal-error: void assert_file_size(FILE*, size_t): Assertion `file_size == expected_size' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Quit this debugging session? (y or n) [answered Y; input not from terminal]
This is a bug, please report it.  For instructions, see:
<http://www.gnu.org/software/gdb/bugs/>.
/usr/bin/gdb-add-index: line 106: 22220 Aborted                 (core dumped) $GDB --batch -nx -iex 'set auto-load no' -ex "file $file" -ex "save gdb-index $dwarf5 $dir"
gdb-add-index: gdb error generating index for /builddir/build/BUILDROOT/python-graph-tool-2.29-2.fc32.aarch64/usr/lib64/python3.8/site-packages/graph_tool/inference/libgraph_tool_inference.so

It should really have failed the build instead of leaving around such a large file.

Comment 1 Florian Weimer 2019-11-18 16:25:46 UTC
Bug 1745020 discusses a previous aarch64 corruption. However, the builder here is different (buildvm-aarch64-05.arm.fedoraproject.org).

Comment 2 Florian Weimer 2019-11-18 19:44:11 UTC
(In reply to Florian Weimer from comment #1)
> Bug 1745020 discusses a previous aarch64 corruption. However, the builder
> here is different (buildvm-aarch64-05.arm.fedoraproject.org).

Nope, the assert failure reproduces, even on x86_64.

Comment 3 Ben Cotton 2020-02-11 17:45:45 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle.
Changing version to 32.

Comment 4 Fedora Program Management 2021-04-29 15:59:58 UTC
This message is a reminder that Fedora 32 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 32 on 2021-05-25.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '32'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 32 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 5 Ben Cotton 2021-05-25 15:11:07 UTC
Fedora 32 changed to end-of-life (EOL) status on 2021-05-25. Fedora 32 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 6 Florian Weimer 2023-04-04 17:54:23 UTC
*** Bug 2184150 has been marked as a duplicate of this bug. ***

Comment 7 Kevin Buettner 2023-07-11 03:19:21 UTC
I reproduced this problem by doing:

1. fedpkg co python-graph-tool
2. cd python-graph-tool
3. fedpkg mockbuild

(which I found in bug 2184150)

The build output shows:

+ /usr/bin/find-debuginfo -j6 --strict-build-id -m -i --build-id-seed 2.57-1.fc39 --unique-debug-suffix -2.57-1.fc39.x86_64 --unique-debug-src-base python-graph-tool-2.57-1.fc39.x86_64 --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 110000000 -S debugsourcefiles.list /builddir/build/BUILD/graph-tool-2.57
find-debuginfo: starting
Extracting debug info from 16 files
../../gdb/dwarf2/index-write.c:1069: internal-error: assert_file_size: Assertion `file_size == expected_size' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.

(followed by an unusable backtrace)

FWIW, I watched htop while it was running and saw that the machine used 33G of RAM at one point.  It may have used more - I didn't watch that closely. In any case, the machine in question was not resource constrained.

Comment 8 Mark Wielaard 2023-07-11 13:35:56 UTC
Note that you can add
%define _find_debuginfo_opts --verbose
in the spec file to get a bit more verbose output:

+ /usr/bin/find-debuginfo -j2 --strict-build-id -m -i --build-id-seed 2.57-2.fc39 --unique-debug-suffix -2.57-2.fc39.x86_64 --unique-debug-src-base python-graph-tool-2.57-2.fc39.x86_64 --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 110000000 --verbose -S debugsourcefiles.list /builddir/build/BUILD/graph-tool-2.57
find-debuginfo: starting
Extracting debug info from 16 files
extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/clustering/libgraph_tool_clustering.so
extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/centrality/libgraph_tool_centrality.so
extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/correlations/libgraph_tool_correlations.so
extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/draw/libgraph_tool_draw.so
extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/draw/libgraph_tool_layout.so
extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/draw/libgt_pycairo_aux.so
extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/dynamics/libgraph_tool_dynamics.so
extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/flow/libgraph_tool_flow.so
extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/generation/libgraph_tool_generation.so
extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/inference/libgraph_tool_inference.so
extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/libgraph_tool_core.so
extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/search/libgraph_tool_search.so
extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/spectral/libgraph_tool_spectral.so
extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/stats/libgraph_tool_stats.so
extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/topology/libgraph_tool_topology.so
extracting debug info from /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/util/libgraph_tool_util.so
../../gdb/dwarf2/index-write.c:1069: internal-error: assert_file_size: Assertion `file_size == expected_size' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
----- Backtrace -----
0x562a2e75cc20 ???
0x562a2eb62104 ???
0x562a2eb6240e ???
0x562a2ed41304 ???
0x562a2e81d473 ???
0x562a2e81e430 ???
0x562a2e82170e ???
0x562a2e822b74 ???
0x562a2e795ce4 ???
0x562a2eb3ea94 ???
0x562a2e96f599 ???
0x562a2e96f681 ???
0x562a2e971938 ???
0x562a2e97251f ???
0x562a2e67356e ???
../../gdb/dwarf2/index-write.c:1069: internal-error: assert_file_size: Assertion `file_size == expected_size' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Quit this debugging session? (y or n) [answered Y; input not from terminal]
0x7fba58828149 ???
0x7fba5882820a ???
0x562a2e67c564 ???
0xffffffffffffffff ???
---------------------

This is a bug, please report it.  For instructions, see:
<https://www.gnu.org/software/gdb/bugs/>.

/usr/bin/gdb-add-index: line 159: 24580 Aborted                 (core dumped) $GDB --batch -nx -iex 'set auto-load no' -iex 'set debuginfod enabled off' -ex "file $file" -ex "save gdb-index $dwarf5 $dir"
gdb-add-index: gdb error generating index for /builddir/build/BUILDROOT/python-graph-tool-2.57-2.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/inference/libgraph_tool_inference.so
DWARF-compressing 16 files
original debug info size: 5831472kB, size after compression: 5655276kB
sepdebugcrcfix: Updated 16 CRC32s, 0 CRC32s did match.
Creating .debug symlinks for symlinks to ELF files
Copying sources found by 'debugedit -l' to /usr/src/debug/python-graph-tool-2.57-2.fc39.x86_64
6054 blocks
find-debuginfo: done

Unfortunately the backtrace is still unusable, but at least we now know which file gdb-add-index crashes on.
libgraph_tool_inference.so

Comment 9 Kevin Buettner 2023-07-14 01:45:37 UTC
Within my mock chroot, after hiding gdb.minimal, installing plain old gdb (to load any additional dependencies), and then replacing it with one built with -O0 -g3, I can finally see:

(gdb) up 8
#8  0x000000000074ea68 in assert_file_size (file=0x3fa52b0, 
    expected_size=280673559) at ../../gdb/dwarf2/index-write.c:1069
1069	  gdb_assert (file_size == expected_size);
(gdb) p file_size
$1 = 4575640855
(gdb) p expected_size
$2 = 280673559

Wow - not even close!  (So... not an off-by-one error somewhere.)

Here's the full backtrace:

(gdb) bt
#0  __pthread_kill_implementation (threadid=<optimized out>, 
    signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x00007f9a35c8fb43 in __pthread_kill_internal (signo=6, 
    threadid=<optimized out>) at pthread_kill.c:78
#2  0x00007f9a35c3e8ee in __GI_raise (sig=sig@entry=6)
    at ../sysdeps/posix/raise.c:26
#3  0x00007f9a35c268ff in __GI_abort () at abort.c:79
#4  0x0000000000dd97fb in dump_core () at ../../gdb/utils.c:202
#5  0x0000000000dd9e1c in internal_vproblem(internal_problem *, const char *, int, const char *, typedef __va_list_tag __va_list_tag *) (
    problem=0x1ae55a0 <internal_error_problem>, 
    file=0x12081f8 "../../gdb/dwarf2/index-write.c", line=1069, 
    fmt=0x12080f8 "%s: Assertion `%s' failed.", ap=0x7ffe853267b8)
    at ../../gdb/utils.c:450
#6  0x0000000000dd9ef2 in internal_verror (
    file=0x12081f8 "../../gdb/dwarf2/index-write.c", line=1069, 
    fmt=0x12080f8 "%s: Assertion `%s' failed.", ap=0x7ffe853267b8)
    at ../../gdb/utils.c:476
#7  0x00000000010a104a in internal_error_loc (
    file=0x12081f8 "../../gdb/dwarf2/index-write.c", line=1069, 
    fmt=0x12080f8 "%s: Assertion `%s' failed.")
    at ../../gdbsupport/errors.cc:58
#8  0x000000000074ea68 in assert_file_size (file=0x3fa52b0, 
    expected_size=280673559) at ../../gdb/dwarf2/index-write.c:1069
#9  0x000000000074ec19 in write_gdbindex_1 (out_file=0x3fa52b0, cu_list=..., 
    types_cu_list=..., addr_vec=..., symtab_vec=..., constant_pool=...)
    at ../../gdb/dwarf2/index-write.c:1119
#10 0x000000000074f4de in write_gdbindex (per_objfile=0x3f979a0, 
    table=0x57e7fe0, out_file=0x3fa52b0, dwz_out_file=0x0)
    at ../../gdb/dwarf2/index-write.c:1273
#11 0x00000000007501d8 in write_dwarf_index (per_objfile=0x3f979a0, 
    dir=0x7ffe85328f7d "/builddir/build/BUILDROOT/python-graph-tool-2.57-1.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/inference", 
    basename=0x32c9827 "libgraph_tool_inference.so", dwz_basename=0x0, 
    index_kind=dw_index_kind::GDB_INDEX) at ../../gdb/dwarf2/index-write.c:1493
#12 0x000000000075046b in save_gdb_index_command (
    arg=0x7ffe85328f7d "/builddir/build/BUILDROOT/python-graph-tool-2.57-1.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/inference", from_tty=0)
    at ../../gdb/dwarf2/index-write.c:1548
#13 0x000000000065385a in do_simple_func (
    args=0x7ffe85328f7d "/builddir/build/BUILDROOT/python-graph-tool-2.57-1.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/inference", from_tty=0, 
    c=0x311c930) at ../../gdb/cli/cli-decode.c:95
#14 0x0000000000658545 in cmd_func (cmd=0x311c930, 
    args=0x7ffe85328f7d "/builddir/build/BUILDROOT/python-graph-tool-2.57-1.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/inference", from_tty=0)
    at ../../gdb/cli/cli-decode.c:2543
#15 0x0000000000d5adfc in execute_command (p=0x7ffe85328ff2 "e", from_tty=0)
    at ../../gdb/top.c:690
#16 0x00000000009d5dba in catch_command_errors (
    command=0xd5a7e3 <execute_command(char const*, int)>, 
    arg=0x7ffe85328f6d "save gdb-index  /builddir/build/BUILDROOT/python-graph-tool-2.57-1.fc39.x86_64/usr/lib64/python3.12/site-packages/graph_tool/inference", from_tty=0, do_bp_actions=true) at ../../gdb/main.c:513
#17 0x00000000009d5f9b in execute_cmdargs (cmdarg_vec=0x7ffe853276d0, 
    file_type=CMDARG_FILE, cmd_type=CMDARG_COMMAND, ret=0x7ffe8532768c)
    at ../../gdb/main.c:608
#18 0x00000000009d73f1 in captured_main_1 (context=0x7ffe85327960)
    at ../../gdb/main.c:1299
#19 0x00000000009d7652 in captured_main (data=0x7ffe85327960)
    at ../../gdb/main.c:1320
#20 0x00000000009d76bd in gdb_main (args=0x7ffe85327960)
    at ../../gdb/main.c:1345
#21 0x000000000041971d in main (argc=11, argv=0x7ffe85327aa8)
    at ../../gdb/gdb.c:40

Comment 10 Ben Beasley 2023-07-14 02:03:22 UTC
Nice! At a glance, it looks like the expected size is exactly 4 GiB, or 2^32 bytes, less than the actual size, as if it was accumulated in an unsigned 32-bit value that wrapped around.

Comment 11 Kevin Buettner 2023-07-14 06:02:37 UTC
(In reply to Ben Beasley from comment #10)
> Nice! At a glance, it looks like the expected size is exactly 4 GiB, or 2^32
> bytes, less than the actual size, as if it was accumulated in an unsigned
> 32-bit value that wrapped around.

That's exactly right - see my next comment below...

Comment 12 Kevin Buettner 2023-07-14 06:03:07 UTC
Upstream GDB (14.0.50.20230714-git) shows the same behavior.

The values for file_size and expected_size are also the same.

But look at this...

(gdb) up 8
#8  0x000000000064a7c9 in assert_file_size (file=0x2fef340, 
    expected_size=280673559)
    at /ironwood1/sourceware-git/rawhide-master/bld/../../worktree-master/gdb/dwarf2/index-write.c:1071
1071	  gdb_assert (file_size == expected_size);
(gdb) ptype file_size
type = const long
(gdb) ptype expected_size
type = unsigned long
(gdb) p sizeof(file_size)
$1 = 8
(gdb) p sizeof(expected_size)
$2 = 8

It'd be nice if they were the same type, but an 8 byte long is big enough to hold the values, whether signed or not.

Now for the interesting bit...

(gdb) up
#9  0x000000000064a97a in write_gdbindex_1 (out_file=0x2fef340, cu_list=..., 
    types_cu_list=..., addr_vec=..., symtab_vec=..., constant_pool=...)
    at /ironwood1/sourceware-git/rawhide-master/bld/../../worktree-master/gdb/dwarf2/index-write.c:1121
1121	  assert_file_size (out_file, total_len);
(gdb) p total_len
$3 = 280673559
(gdb) ptype total_len
type = unsigned int
(gdb) p sizeof(total_len)
$4 = 4
(gdb) p total_len + ((long)1<<32)
$6 = 4575640855

So, what's happened is that the length of the file in question exceeds that of an unsigned 32-bit offset.

That $6 value matches file_size in assert_file_size().

In write_gdbindex_1, total_len is declared as follows:

  offset_type total_len = size_of_header;

And, in gdb/dwarf2/index-common.h, I find:

/* All offsets in the index are of this type.  It must be
   architecture-independent.  */
typedef uint32_t offset_type;

The type of total_len could be changed to an 64-bit type, but at a number of points along the way, a 32-bit offset is saved in a "contents" buffer. If it overflows 32 bits, at some point it'll be wrong.

However, if we were to change the type of total_len to a 64-bit type, we could at least check for 32-bit overflow and throw an error if that happens.  Surely that's preferable to failing an assert, which causes an internal error to occur.

I'll ponder it some more and try to come up with a patch...

Comment 13 Fedora Release Engineering 2023-08-16 07:04:25 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 39 development cycle.
Changing version to 39.

Comment 14 Kevin Buettner 2023-09-18 20:50:21 UTC
This bug has been fixed upstream via the following commit:

98f6baad7c3 Throw error when creating an overly large gdb-index file

It'll be fixed in Fedora GDB 14.1.


Note You need to log in before you can comment on or make changes to this bug.