Bug 2408418
| Summary: | error: Empty %files file /builddir/build/BUILD/webkitgtk-2.51.1-build/webkitgtk-2.51.1/debugsourcefiles.list | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Michael Catanzaro <mcatanza> | ||||
| Component: | debugedit | Assignee: | Mark Wielaard <mjw> | ||||
| Status: | CLOSED UPSTREAM | QA Contact: | |||||
| Severity: | medium | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 44 | CC: | mcermak, mjw, ngompa13 | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | --- | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2025-11-06 15:10:02 UTC | Type: | --- | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
|
Description
Michael Catanzaro
2025-10-30 21:34:13 UTC
Created attachment 2111534 [details]
dist-git patch
This is odd. You are right that the dwz warnings are harmless. It looks like the package is using clang with -gdwarf5 which generates these .debug_addr sections dwz doesn't handle. But that just means the debuginfo isn't dwz compressed. So the issue is that debugedit -l is supposed to find the debugsources in the debuginfo. And as you can see from the cpio copying some of those source files cannot be found (in the successful build). Those cpio warnings are also not really an issue (as you can see they are gperf or yacc files, there should be plenty of "real" source files. It would be nice to figure out why those generator source files cannot be found, but that is for another time). The same file $BUILDDIR/debugsources.list is used for the %files directive. So that we don't see "Copying sources found by 'debugedit -l' ..." (which is guarded by test -s) and do see "Empty %files file [...]/debugsourcefiles.list" has the same cause. The debugsourcefiles.list is empty. Which means debugedit -l couldn't find/extract any source files from the debug files. I don't know why and couldn't quickly build this package (it is huge!). Would you happen to have a full build.log for both? Ideally generated with _find_debuginfo_opts --verbose One cause could be that none of the object files are generated with -g (so nothing contains any debuginfo). But that is unlikely given dwz is complaining about unrecognized .debug_addr section. Another could be that the whole build is not actually done under the builddir (but in /tmp/ for example) so none of the source files would look like they come from the actual build (any source file referenced outside the builddir is assumed to to be a system/devel file, like /usr/include/...). Also not very likely, but has been seen with some builds. (In reply to Mark Wielaard from comment #2) > Would you happen to have a full build.log for both? > Ideally generated with _find_debuginfo_opts --verbose Without --verbose, both logs are in my first comment. I think I've probably already pasted the relevant bits, though. I will do new builds later today to add --verbose. (In reply to Mark Wielaard from comment #3) > One cause could be that none of the object files are generated with -g (so > nothing contains any debuginfo). > But that is unlikely given dwz is complaining about unrecognized .debug_addr > section. > > Another could be that the whole build is not actually done under the > builddir (but in /tmp/ for example) so none of the source files would look > like they come from the actual build (any source file referenced outside the > builddir is assumed to to be a system/devel file, like /usr/include/...). > Also not very likely, but has been seen with some builds. I doubt either of these is happening. Here are new build logs with the additional debugging you requested (%global _find_debuginfo_opts --verbose): Good (WebKitGTK 2.50.1): https://kojipkgs.fedoraproject.org/work/tasks/307/138740307/build.log Bad (WebKitGTK 2.51.1): https://kojipkgs.fedoraproject.org/work/tasks/306/138740306/build.log This time I immediately spot the problem: the bad build uses `--file-prefix-map=.` (setting it to the current working directory), introduced by https://commits.webkit.org/301772@main. This was actually already evident in the build logs that I provided last week, and I simply did not notice it. Oh well. I've never heard of this option before, but gcc(1) says: -ffile-prefix-map=old=new When compiling files residing in directory old, record any references to them in the result of the compilation as if the files resided in directory new instead. Specifying this option is equivalent to specifying all the individual -f*-prefix-map options. This can be used to make reproducible builds that are location independent. Directories referenced by directives are not affected by these options. See also -fmacro-prefix-map, -fdebug-prefix-map, -fprofile-prefix-map and -fcanon-prefix-map. The extra debugging shows us a little bit more of what is happening, all the source files present in the 2.50.1 build: + /usr/bin/find-debuginfo -j8 --strict-build-id -m -i --build-id-seed 2.50.1-2.fc44 --unique-debug-suffix -2.50.1-2.fc44.x86_64 --unique-debug-src-base webkitgtk-2.50.1-2.fc44.x86_64 --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 250000000 --verbose -j1 -S debugsourcefiles.list /builddir/build/BUILD/webkitgtk-2.50.1-build/webkitgtk-2.50.1 find-debuginfo: starting Extracting debug info from 17 files extracting debug info from /builddir/build/BUILD/webkitgtk-2.50.1-build/BUILDROOT/usr/bin/WebKitWebDriver found 17810 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.50.1-build/BUILDROOT/usr/lib64/libjavascriptcoregtk-4.1.so.0.9.6 found 132867 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.50.1-build/BUILDROOT/usr/lib64/libjavascriptcoregtk-6.0.so.1.6.6 found 247917 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.50.1-build/BUILDROOT/usr/lib64/libwebkit2gtk-4.1.so.0.19.4 found 839502 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.50.1-build/BUILDROOT/usr/lib64/libwebkitgtk-6.0.so.4.13.4 found 1424300 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.50.1-build/BUILDROOT/usr/lib64/webkit2gtk-4.1/injected-bundle/libwebkit2gtkinjectedbundle.so found 1424408 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.50.1-build/BUILDROOT/usr/lib64/webkitgtk-6.0/injected-bundle/libwebkitgtkinjectedbundle.so found 1424516 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.50.1-build/BUILDROOT/usr/libexec/webkit2gtk-4.1/MiniBrowser found 1424635 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.50.1-build/BUILDROOT/usr/libexec/webkit2gtk-4.1/WebKitGPUProcess found 1424646 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.50.1-build/BUILDROOT/usr/libexec/webkit2gtk-4.1/WebKitNetworkProcess found 1424657 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.50.1-build/BUILDROOT/usr/libexec/webkit2gtk-4.1/WebKitWebProcess found 1424668 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.50.1-build/BUILDROOT/usr/libexec/webkit2gtk-4.1/jsc found 1425081 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.50.1-build/BUILDROOT/usr/libexec/webkitgtk-6.0/MiniBrowser found 1425187 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.50.1-build/BUILDROOT/usr/libexec/webkitgtk-6.0/WebKitGPUProcess found 1425198 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.50.1-build/BUILDROOT/usr/libexec/webkitgtk-6.0/WebKitNetworkProcess found 1425209 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.50.1-build/BUILDROOT/usr/libexec/webkitgtk-6.0/WebKitWebProcess found 1425220 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.50.1-build/BUILDROOT/usr/libexec/webkitgtk-6.0/jsc found 1425633 source files And all gone in the 2.51.1 build: + /usr/bin/find-debuginfo -j8 --strict-build-id -m -i --build-id-seed 2.51.1-2.fc44 --unique-debug-suffix -2.51.1-2.fc44.x86_64 --unique-debug-src-base webkitgtk-2.51.1-2.fc44.x86_64 --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 250000000 --verbose -j1 -S debugsourcefiles.list /builddir/build/BUILD/webkitgtk-2.51.1-build/webkitgtk-2.51.1 find-debuginfo: starting Extracting debug info from 17 files extracting debug info from /builddir/build/BUILD/webkitgtk-2.51.1-build/BUILDROOT/usr/bin/WebKitWebDriver found 0 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.51.1-build/BUILDROOT/usr/lib64/libjavascriptcoregtk-4.1.so.0.10.0 found 0 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.51.1-build/BUILDROOT/usr/lib64/libjavascriptcoregtk-6.0.so.1.7.0 found 0 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.51.1-build/BUILDROOT/usr/lib64/libwebkit2gtk-4.1.so.0.20.0 found 0 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.51.1-build/BUILDROOT/usr/lib64/libwebkitgtk-6.0.so.4.14.0 found 0 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.51.1-build/BUILDROOT/usr/lib64/webkit2gtk-4.1/injected-bundle/libwebkit2gtkinjectedbundle.so found 0 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.51.1-build/BUILDROOT/usr/lib64/webkitgtk-6.0/injected-bundle/libwebkitgtkinjectedbundle.so found 0 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.51.1-build/BUILDROOT/usr/libexec/webkit2gtk-4.1/MiniBrowser found 0 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.51.1-build/BUILDROOT/usr/libexec/webkit2gtk-4.1/WebKitGPUProcess found 0 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.51.1-build/BUILDROOT/usr/libexec/webkit2gtk-4.1/WebKitNetworkProcess found 0 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.51.1-build/BUILDROOT/usr/libexec/webkit2gtk-4.1/WebKitWebProcess found 0 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.51.1-build/BUILDROOT/usr/libexec/webkit2gtk-4.1/jsc found 0 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.51.1-build/BUILDROOT/usr/libexec/webkitgtk-6.0/MiniBrowser found 0 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.51.1-build/BUILDROOT/usr/libexec/webkitgtk-6.0/WebKitGPUProcess found 0 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.51.1-build/BUILDROOT/usr/libexec/webkitgtk-6.0/WebKitNetworkProcess found 0 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.51.1-build/BUILDROOT/usr/libexec/webkitgtk-6.0/WebKitWebProcess found 0 source files extracting debug info from /builddir/build/BUILD/webkitgtk-2.51.1-build/BUILDROOT/usr/libexec/webkitgtk-6.0/jsc found 0 source files I have not yet confirmed for certain that this is really the problem, but it seems close enough to a smoking gun. I'm not sure what to do about this other than stop using the flag. (I should also change it to be *prepended* to CXXFLAGS, rather than *appended*, so that we can override it without patching the CMake build.) Justification for adding the flag is to make remote ccache builds work, https://bugs.webkit.org/show_bug.cgi?id=301003, which is something we plainly do not need in distro builds. Maybe we could limit the flag to developer builds only. Apparently -ffile-prefix-map=. is also required for reproducibility in Debian. That's not needed in Fedora since we build in chroots with predictable paths. For now, I will remove the flag in a downstream patch. I wonder whether `debugedit -l` can support this flag, or whether we need to just stop using it upstream (limit it to developer builds, ask Debian to add it manually in the Debian packaging). Using -ffile-prefix-map=. will definitely confuse debugedit.
(Or actually -ffile-prefix-map=${CMAKE_SOURCE_DIR}=. in this case)
It will also confuse DWARF consumers (debuggers, profilers, tracers, etc.) which don't know what the "." is relative to.
In general it will produce broken/ambiguous paths because
a) we don't have any record of what the original (absolute) path was.
b) it can result in inconsistent paths between object files linked together, but build from different source/build directories.
(so "." might refer to different absolute paths for different compile units)
I filed a bug against upstream debugedit to see if this can be supported, but it will need some "clever" heuristics to reconstruct the "." paths.
https://sourceware.org/bugzilla/show_bug.cgi?id=33602
Not sure how long that will take, or how much work it will be. So for now getting rid of -ffile-prefix-map is your best option.
BTW. debugedit does help with reproducibility in this case since it rewrites the paths to a fixed absolute dir (where we'll also put all sources).
Thanks Mark! I will forward your feedback to WebKit Bugzilla. And I'll close this, since I don't think we need the downstream bug report anymore now that you've forwarded it to upstream. Oops, yes, the log indeed uses `-ffile-prefix-map=/builddir/build/BUILD/webkitgtk-2.51.1-build/webkitgtk-2.51.1=.` not `-ffile-prefix-map=.` -- not sure how I managed to report that incorrectly. For what it's worth, it's probably better to have the flag dropped from upstream WebKit. Either Debian adds it themselves or they consider using debugedit to do smarter things for debug symbol extraction. |