I'm trying to reproducibly rebuild rpms (see [1] for the background), and repeat builds are identical except for the .gnu_debuglink and .gdb_index sections [2]. .gnu_debuglink has the same filename, but the CRC hash differs. IIUC, this is because the .gdb_index section in the .debug file is different. If the latter is fixed, both issues will go away. .gdb_index seems to have the same content, but just in different order (see [2]). The invocations are the same, except for cpu count: + /usr/bin/find-debuginfo -j6 --strict-build-id -i --build-id-seed 3.21.0-8.fc39 --unique-debug-suffix -3.21.0-8.fc39.x86_64 --unique-debug-src-base valgrind-3.21.0-8.fc39.x86_64 --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 110000000 -S debugsourcefiles.list /builddir/build/BUILD/valgrind-3.21.0 + /usr/bin/find-debuginfo -j8 --strict-build-id -i --build-id-seed 3.21.0-8.fc39 --unique-debug-suffix -3.21.0-8.fc39.x86_64 --unique-debug-src-base valgrind-3.21.0-8.fc39.x86_64 --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 110000000 -S debugsourcefiles.list /builddir/build/BUILD/valgrind-3.21.0 Please make the output stable so that repeat builds are identical. I don't know anything about this process, so please redirect me to the right place if it's not debuginfo that is responsible for the contents of that section. [1] https://discussion.fedoraproject.org/t/report-from-the-reproducible-builds-hackfest-during-flock-2023/87469 [2] https://pagure.io/fedora-reproducible-builds/project/issue/5
The issue only occurs in some cases. E.g. for valgrind, it's only the files under /usr/libexec/valgrind/, and for some other packages I rebuilt, the issue does not occur at all. Setting RPM_BUILD_NCPUS= doesn't seem to have an effect.
After looking at this some more, find-debuginfo calls gdb-add-index, which calls gdb.minimal: $GDB --batch -nx -iex 'set auto-load no' \ -iex 'set debuginfod enabled off' \ -ex "file $file" -ex "save gdb-index $dwarf5 $dir" $OBJCOPY --add-section $section="$index" \ --set-section-flags $section=readonly \ $(if $debugstradd; then \ echo --add-section .debug_str="$debugstrmerge"; \ echo --set-section-flags .debug_str=readonly; \ fi; \ if $debugstrupdate; then \ echo --update-section .debug_str="$debugstrmerge"; \ fi) \ "$fpath" "$fpath" + /usr/bin/gdb.minimal --batch -nx -iex 'set auto-load no' -iex 'set debuginfod enabled off' -ex 'file /builddir/build/BUILDROOT/valgrind-3.21.0-8.fc40.x86_64_5/usr/libexec/valgrind/massif-amd64-linux' -ex 'save gdb-index /builddir/build/BUILDROOT/valgrind-3.21.0-8.fc40.x86_64_5/usr/libexec/valgrind' + objcopy --add-section .gdb_index=/builddir/build/BUILDROOT/valgrind-3.21.0-8.fc40.x86_64_5/usr/libexec/valgrind/massif-amd64-linux.gdb-index --set-section-flags .gdb_index=readonly /builddir/build/BUILDROOT/valgrind-3.21.0-8.fc40.x86_64_5/usr/libexec/valgrind/massif-amd64-linux /builddir/build/BUILDROOT/valgrind-3.21.0-8.fc40.x86_64_5/usr/libexec/valgrind/massif-amd64-linux So it seems the contents of that section are generated by gdb.minimal and then just copied over. So maybe this needs to be reassigned to gdb?
Yes, gdb-add-index is part of gdb. I don't immediately see anything that could cause non-deterministic output in the gdb-add-index script itself. So it must be something in gdb itself that generates or saves the gdb-index data non-deterministically. Do you happen to have an example invocation of gdb-add-index that produces different results on different runs? Is it the contents of .gdb_index section or something else in the ELF file that is different?