Fedora Account System
Red Hat Associate
Red Hat Customer
With Fedora 35 serving debuginfo to gdb etc. via the debuginfod service by default, we see regular queries looking for debuginfo of the userspace-relevant vdso* libraries. This happens even if the user is running gdb on some ordinary program like /bin/ls, because vdso is mapped into userspace, has a buildid, thus gdb tries to fetch its dwarf data. With debuginfod, this request can now be serviced. However, this particular request is a challenge to the server, because it involves decompressing the appropriate kernel-debuginfo file on the fly, and saving the vdso*.so.debug file, then relaying it to to the remote client. The problem is that these files are practically at the END of the -debuginfo compressed-cpio archive, after all the module .ko.debug's (many hundreds), just before the final vmlinux (just the one! :-). That ends up taking something like 60 cpu seconds (!). If it were possible to sort the kernel's main debuginfo*.list file, to put the vdso* files at the front of the file list, these requests could be serviced much earlier, without decompressing the bulk of the file. I believe this code in the rawhide/master branch kernel.spec file is responsible for the sequencing: 1065 %description %{?1:%{1}-}debuginfo\ 1066 This package provides debug information for package %{name}%{?1:-%{1}}.\ 1067 This is required to use SystemTap with %{name}%{?1:-%{1}}-%{KVERREL}.\ 1068 %{expand:%%global _find_debuginfo_opts %{?_find_debuginfo_opts} -p '.*\/usr\/src\/kernels/.*|XXX' -o ignored-debuginfo.lis 1068 t -p '/.*/%%{KVERREL_RE}%{?1:[+]%{1}}/.*|/.*%%{KVERREL_RE}%{?1:\+%{1}}(\.debug)?' -o debuginfo%{?1}.list}\ 1069 %{nil} Would you consider adding a brief postprocessor (two greps?) to sort the vdso* file names earlier? (Another alternative may be to put these into a whole separate subrpm, and other options may exist too.)
Never mind. I spent some time experimenting with overriding %global __os_install_post \ # sort debugfiles*list differently \ %__os_install_post and this hooking process worked. However, rpm's build/files.c genCpioListAndHeader() sorts rpm manifest lists ANYWAY, so there appears to be nothing a .spec file can do.
(The kernel spec file could still create a separate subrpm for vdso files, and that would completely fix this particular problem.)
Our friends at suse have switched to a separate vdso subpackage, to solve precisely this problem. It makes fedora debugging much faster to start up (60s in the worst case down to fractions of a second). Please consider doing the same. Search for separate_vdso here: https://build.opensuse.org/package/view_file/Kernel:stable/kernel-default/kernel-default.spec?expand=1
(In reply to Frank Ch. Eigler from comment #3) > (60s in the worst case down to fractions of a second). Currently about 80s, and due to a server-side misconfiguration it wasn't being cached on the server, so it was quite painful if you're impatient like me. A separate debuginfo package would be nice.
Caching helps with this issue but each time a vdso debuginfo is added to the server's cache, there will have to be a client that needs to wait for the decompression to occur. It can be annoying to wait over a minute for this, especially if this occurs during an interactive session (ex. GDB). A separate debuginfo package for the vdso will eliminate all of this extra download time.
Yeah, if you are unlucky this takes ~80 seconds just for fetching the vdso debug file. Having a separate user space only debuginfo package would be helpful also in case someone just wants to install the debuginfo and not all the kernel space debuginfo they might not use.