RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1986018 - fedora 34+ debugsource files lack assembler .S files
Summary: fedora 34+ debugsource files lack assembler .S files
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: binutils
Version: CentOS Stream
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: beta
: ---
Assignee: Nick Clifton
QA Contact: Miloš Prchlík
URL:
Whiteboard:
Depends On: 1966987
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-07-26 14:13 UTC by Frank Ch. Eigler
Modified: 2021-12-07 21:38 UTC (History)
21 users (show)

Fixed In Version: binutils-2.35.2-8.el9
Doc Type: No Doc Update
Doc Text:
Clone Of: 1966987
Environment:
Last Closed: 2021-12-07 21:35:16 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Frank Ch. Eigler 2021-07-26 14:13:51 UTC
+++ This bug was initially created as a clone of Bug #1966987 +++

The build process for glibc includes a mixture of C and assembly code.  The DWARF debuginfo (post-debugedit) includes file name references such as

/usr/src/debug/glibc-2.33.9000-73-ge9e7f24543/sysdeps/x86_64/multiarch/strstr-sse2-unaligned.S

However, the -debugsource subrpm does not include any .s / .S files at all.  This makes debugging a little big harder, and makes the fedora debuginfod a little bit sad to report 404s to users.

--- Additional comment from Florian Weimer on 2021-06-02 10:04:48 UTC ---

Thanks for the report. I think this needs to be fixed in debugedit. We no longer do our own thing in glibc.

--- Additional comment from Frank Ch. Eigler on 2021-06-02 10:50:03 UTC ---

It looks as though something changed in the f34 build process.

Looking at other RPMs such as the recently built openssl-1.1.1k in koji, builds up to and including fc33 include .s files in the -debugsource.  f34 and above do not.

And ditto for GLIBC: 2.32-6.fc33 includes .S files in -debugsource.

--- Additional comment from Mark Wielaard on 2021-06-02 10:54:14 UTC ---

(In reply to Frank Ch. Eigler from comment #2)
> It looks as though something changed in the f34 build process.

That is surprising, is the same true for f35?
In f35 rpm switched to using external debugedit find-debuginfo[.sh].

But f34 should use the same rpm find-debuginfo.sh as earlier versions.

--- Additional comment from Frank Ch. Eigler on 2021-06-02 12:18:02 UTC ---

From my quick sampling tests, debugsource files
contain .S files <= fc33
lack .S files >= fc34

--- Additional comment from Frank Ch. Eigler on 2021-06-02 13:46:30 UTC ---

hand-rebuilding rawhide glibc on f33 (fedpkg local) results in a glibc-debugsource-2.33.9000-11.fc35.x86_64.rpm file that does contain .S files.

--- Additional comment from Frank Ch. Eigler on 2021-06-02 21:16:42 UTC ---

More data: gcc -gdwarf5 defaulting appears to be the driving difference.  Even for a noop assembler file

foo.s:

   main:
       nop

gcc -gdwarf-5 -c foo.s   # results in an .o with dwarf5 line records
gcc -gdwarf-5 -c foo.s   # results in an .o with old school records

and "debugedit -l FILE foo.o" only works with the latter.  Continuing to investigate.

--- Additional comment from Frank Ch. Eigler on 2021-06-02 21:17:04 UTC ---

gcc -gdwarf-5 -c foo.s   # results in an .o with dwarf5 line records
gcc -gdwarf-4 -c foo.s   # results in an .o with old school records

--- Additional comment from Frank Ch. Eigler on 2021-06-02 22:49:01 UTC ---

Yeah it appears as though debugedit just can't extract source file names from -gdwarf-5 files, not just .s but also .c:

% echo 'int main () {}' > foo.c
% gcc -gdwarf-5 foo.c
% debugedit -l foo.list a.out
% cat foo.list
[empty]

--- Additional comment from Mark Wielaard on 2021-06-22 17:29:14 UTC ---

This should at least solve the issue of comment 8. Without giving -d /foo/build/bar debugedit wouldn't actually output any file from the line number table for DWARF5.

diff --git a/tools/debugedit.c b/tools/debugedit.c
index c6975b2..75803a2 100644
--- a/tools/debugedit.c
+++ b/tools/debugedit.c
@@ -1826,8 +1826,8 @@ read_dwarf5_line_entries (DSO *dso, unsigned char **ptrp,
   /* file_names_count */
   unsigned entry_count = read_uleb128 (*ptrp);
 
-  bool collecting_dirs = dest_dir && phase == 0 && *dirs == NULL;
-  bool writing_files = dest_dir && phase == 0 && *dirs != NULL;
+  bool collecting_dirs = phase == 0 && *dirs == NULL;
+  bool writing_files = phase == 0 && *dirs != NULL;
   if (collecting_dirs)
     {
       *ndir = entry_count;
@@ -1857,15 +1857,19 @@ read_dwarf5_line_entries (DSO *dso, unsigned char **ptrp,
                {
                case DW_FORM_strp:
                case DW_FORM_line_strp:
-                 if (dest_dir && phase == 0)
+                 if (phase == 0)
                    {
                      size_t idx = do_read_32_relocated (*ptrp);
-                     if (record_file_string_entry_idx (line_strp, dso, idx))
+                     if (dest_dir)
                        {
-                         if (line_strp)
-                           need_line_strp_update = true;
-                         else
-                           need_strp_update = true;
+                         if (record_file_string_entry_idx (line_strp,
+                                                           dso, idx))
+                           {
+                             if (line_strp)
+                               need_line_strp_update = true;
+                             else
+                               need_strp_update = true;
+                           }
                        }
                      handled_strp = true;
                      if (collecting_dirs || writing_files)
@@ -1951,7 +1955,7 @@ read_dwarf5_line_entries (DSO *dso, unsigned char **ptrp,
       if (writing_files)
        {
          char *comp_dir = (*dirs)[0];
-         size_t comp_dir_len = strlen(comp_dir);
+         size_t comp_dir_len = !comp_dir ? 0 : strlen(comp_dir);
          size_t file_len = strlen (file);
          size_t dir_len = strlen (dir);
 

I am not sure this also solves the original issue because during packaging debugedit should always be invoked with --base ... and --dest ...

--- Additional comment from Mark Wielaard on 2021-07-26 12:22:36 UTC ---

As far as I can see there simply is no debuginfo for these .S files:

# eu-readelf -S ./glibc-2.33.9000-984-gddcc612ce9/build-x86_64-redhat-linux/csu/sysdep.o
There are 11 section headers, starting at offset 0x190:

Section Headers:
[Nr] Name                 Type         Addr             Off      Size     ES Flags Lk Inf Al
[ 0]                      NULL         0000000000000000 00000000 00000000  0        0   0  0
[ 1] .text                PROGBITS     0000000000000000 00000040 00000000  0 AX     0   0  1
[ 2] .data                PROGBITS     0000000000000000 00000040 00000000  0 WA     0   0  1
[ 3] .bss                 NOBITS       0000000000000000 00000040 00000000  0 WA     0   0  1
[ 4] .note.gnu.property   NOTE         0000000000000000 00000040 00000050  0 A      0   0  8
[ 5] .note.GNU-stack      PROGBITS     0000000000000000 00000090 00000000  0        0   0  1
[ 6] .gnu.build.attributes NOTE         0000000000000000 00000090 00000024  0        0   0  4
[ 7] .rela.gnu.build.attributes RELA         0000000000000000 000000f0 00000030 24 I      8   6  8
[ 8] .symtab              SYMTAB       0000000000000000 000000b8 00000030 24        9   2  8
[ 9] .strtab              STRTAB       0000000000000000 000000e8 00000001  0        0   0  1
[10] .shstrtab            STRTAB       0000000000000000 00000120 0000006a  0        0   0  1

So nothing for debugedit to extract.

This might be a gcc (or binutils gas?) bug because it does look like -g is being passed when building the file:

gcc ../sysdeps/unix/sysv/linux/x86_64/sysdep.S -c     -I../include -I/root/glibc/glibc-2.33.9000-984-gddcc612ce9/build-x86_64-redhat-linux/csu  -I/root/glibc/glibc-2.33.9000-984-gddcc612ce9/build-x86_64-redhat-linux  -I../sysdeps/unix/sysv/linux/x86_64/64  -I../sysdeps/unix/sysv/linux/x86_64  -I../sysdeps/unix/sysv/linux/x86/include -I../sysdeps/unix/sysv/linux/x86  -I../sysdeps/x86/nptl  -I../sysdeps/unix/sysv/linux/wordsize-64  -I../sysdeps/x86_64/nptl  -I../sysdeps/unix/sysv/linux/include -I../sysdeps/unix/sysv/linux  -I../sysdeps/nptl  -I../sysdeps/pthread  -I../sysdeps/gnu  -I../sysdeps/unix/inet  -I../sysdeps/unix/sysv  -I../sysdeps/unix/x86_64  -I../sysdeps/unix  -I../sysdeps/posix  -I../sysdeps/x86_64/64  -I../sysdeps/x86_64/fpu/multiarch  -I../sysdeps/x86_64/fpu  -I../sysdeps/x86/fpu  -I../sysdeps/x86_64/multiarch  -I../sysdeps/x86_64  -I../sysdeps/x86/include -I../sysdeps/x86  -I../sysdeps/ieee754/float128  -I../sysdeps/ieee754/ldbl-96/include -I../sysdeps/ieee754/ldbl-96  -I../sysdeps/ieee754/dbl-64  -I../sysdeps/ieee754/flt-32  -I../sysdeps/wordsize-64  -I../sysdeps/ieee754  -I../sysdeps/generic  -I.. -I../libio -I. -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/11/include -isystem /usr/include -D_LIBC_REENTRANT -include /root/glibc/glibc-2.33.9000-984-gddcc612ce9/build-x86_64-redhat-linux/libc-modules.h -DMODULE_NAME=libc -include ../include/libc-symbols.h  -DPIC -DSHARED     -DTOP_NAMESPACE=glibc -DASSEMBLER -I/root/glibc/glibc-2.33.9000-984-gddcc612ce9/build-x86_64-redhat-linux/csu/. -fcf-protection -include cet.h -g -Wa,--generate-missing-build-notes=yes -Werror=undef -Wa,--noexecstack   -o /root/glibc/glibc-2.33.9000-984-gddcc612ce9/build-x86_64-redhat-linux/csu/sysdep.os -MD -MP -MF /root/glibc/glibc-2.33.9000-984-gddcc612ce9/build-x86_64-redhat-linux/csu/sysdep.os.dt -MT /root/glibc/glibc-2.33.9000-984-gddcc612ce9/build-x86_64-redhat-linux/csu/sysdep.os

--- Additional comment from Mark Wielaard on 2021-07-26 12:46:21 UTC ---

Strangely when removing -DPIC from the command line makes the .debug_* sections appear again... I don't know why.

--- Additional comment from Siddhesh Poyarekar on 2021-07-26 12:53:11 UTC ---

(In reply to Mark Wielaard from comment #11)
> Strangely when removing -DPIC from the command line makes the .debug_*
> sections appear again... I don't know why.

The TU is empty if PIC is defined:

~~~~
#ifndef PIC                                                                     
                                                                                
/* The syscall stubs jump here when they detect an error.                       
   The code for Linux is almost identical to the canonical Unix                 
   code, except that the error number in %rax is negated.  */                   
                                                                                
#undef CALL_MCOUNT                                                              
#define CALL_MCOUNT /* Don't insert the profiling call, it clobbers %rax.  */   
                                                                                
        .text                                                                   
ENTRY (__syscall_error)                                                         
        neg %RAX_LP                                                             
                                                                                
#define __syscall_error __syscall_error_1                                       
#include <sysdeps/unix/x86_64/sysdep.S>                                         
                                                                                
#endif  /* !PIC */               
~~~~

--- Additional comment from Mark Wielaard on 2021-07-26 13:50:40 UTC ---

So that explains the sysdep.o not having debuginfo when -DPIC is defined.

Found another example (thanks Siddhesh).
memcpy-ssse3-back.S

Which generates:

DWARF section [13] '.debug_info' at offset 0x3964:
 [Offset]
 Compilation unit at offset 0:
 Version: 5, Abbreviation section offset: 0, Address size: 8, Offset size: 4
 Unit type: compile (1)
 [     c]  compile_unit         abbrev: 1
           stmt_list            (sec_offset) 0
           low_pc               (addr) .text.ssse3+000000000000000000 <__mempcpy_chk_ssse3_back>
           high_pc              (udata) 11014 (.text.ssse3+0x0000000000002b06)
           name                 (strp) "memcpy-ssse3-back.S"
           comp_dir             (strp) "/root/glibc/glibc-2.33.9000-984-gddcc612ce9/string"
           producer             (strp) "GNU AS 2.36.1"
           language             (data2) Mips_Assembler (32769)

[...]
Directory table:
      [path(line_strp)]
 0     ../sysdeps/x86_64/multiarch (0)
 1     ../sysdeps/x86_64/multiarch (28)

File name table:
      [path(line_strp), directory_index(udata)]
 0     memcpy-ssse3-back.S (56),  0
 1     memcpy-ssse3-back.S (76),  1

Note how the directory path zero is a relative path.
That isn't allowed, and it is unclear what it is relative to.
It is relative to the comp_dir entry.
But in DWARF5 the zero dir path entry is supposed to be equal to the comp_dir entry.

This is a bug in binutils GAS.

But we might be able to workaround it in debugedit. We do need to track the CU comp_dir and match it to the line table entry. Then if the zero dir path entry is relative (doesn't start with /) we need to prepend the comp_dir. We also need to do that for the first entry. Which is unfortunate, since normally relative paths are relative to the zero entry...

--- Additional comment from Frank Ch. Eigler on 2021-07-26 14:10:36 UTC ---

Reassigning to binutils.

Comment 5 Nick Clifton 2021-08-09 16:25:58 UTC
Fixed upstream with commit 3417bfca676.
Now waiting for ITM (and the mass rebuild to finish).

Comment 6 Nick Clifton 2021-08-16 13:46:30 UTC
Fixed in binutils-2.35.2-8.el9

Comment 9 Miloš Prchlík 2021-08-26 14:12:20 UTC
Verified with 	binutils-2.35.2-9.el9.


Note You need to log in before you can comment on or make changes to this bug.