[hjl@gnu-6 src]$ cat x.c int foo (void) { return 1; } [hjl@gnu-6 src]$ gcc -S -g x.c [hjl@gnu-6 src]$ as --compress-debug-sections -o x.o x.s [hjl@gnu-6 src]$ ./readelf -w x.o ./readelf: cannot get debug context descriptor: No DWARF information found DWARF section [ 4] '.zdebug_info' at offset 0x4c: [Offset] DWARF section [ 6] '.zdebug_abbrev' at offset 0x88: [ Code] Segmentation fault [hjl@gnu-6 src]$
Confirmed against $ eu-readelf --version readelf (Red Hat elfutils) 0.153 $ rpm -q elfutils elfutils-0.153-1.fc16.x86_64 (gdb) run Starting program: /usr/bin/eu-readelf -w x.o [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". /usr/bin/eu-readelf: cannot get debug context descriptor: No DWARF information found DWARF section [ 4] '.zdebug_info' at offset 0x4c: [Offset] DWARF section [ 6] '.zdebug_abbrev' at offset 0x86: [ Code] Program received signal SIGSEGV, Segmentation fault. 0x000000000040af60 in print_debug_abbrev_section (dwflmod=<optimized out>, ebl=<optimized out>, ehdr=<optimized out>, scn=<optimized out>, shdr=<optimized out>, dbg=0x7fffffffd9c0) at readelf.c:4612 4612 while (offset < dbg->sectiondata[IDX_debug_abbrev]->d_size)
This is weird. A hand build libdw seems fine. But at least with the libdw from elfutils-0.153-1.fc16.x86_64 zrc = inflate (&z, Z_FINISH); libdw/dwarf_begin_elf.c:199 returns Z_DATA_ERROR (-3) for the .zdebug_info section (the others seem to decode fine).
My current guess is that the following in libdw/dwarf_begin_elf.c (check_section) isn't enough: z_stream z = { .next_in = data->d_buf + 4 + 8, .avail_in = data->d_size - 4 - 8, .next_out = zdata->d_buf, .avail_out = zdata->d_size }; And that we also have to initialize (to zero) the following fields: alloc_func zalloc; /* used to allocate the internal state */ free_func zfree; /* used to free the internal state */ voidpf opaque; /* private data object passed to zalloc and zfree */ Which might explain why the results seemed a bit random (since z is stack allocated and might be initialized with random bits).
On irc Frank made the observation that the data in the section seemed to be "corrupted", and suggested that might be because libdwfl relocates the section (it is from an ET_REL file) before decompression. That seems indeed to be the issue. Disabling relocate_section in libdwfl/relocate.c makes the decompression go fine.
The comment#3 theory doesn't make sense. That initializer syntax implicitly sets all unnamed fields to zero. The real reason in comment#4 does make sense. The way we've implemented the compressed-section support just doesn't jibe with the "pure" libdwfl/libdw layering of the original relocation approach. We'd need to violate that layering and make libdwfl do the decompression before it does relocation, so libdw sees only the uncompressed, relocated data. OTOH, this should be a non-issue in the new roland/relocate branch, so perhaps this is motivation to finally tie up the loose ends there and merge in that work. Of course, this all assumes that on the producer side the relocatable case is being handled as we expect. i.e., that e.g. a .rela.zdebug_info section uses offsets consistent with the uncompressed .debug_info section image that got turned into .zdebug_info. That's the only sensical thing to do, but it's also rather a strange format. Really it seems quite questionable that anybody emitted compressed-sections format in an ET_REL file to begin with. Who does that?
I don't think we can easily support .z compressed sections for ET_REL files till we finish the roland/relocate branch for lazy relocations. Maybe we could special case .z compressed sections in __libdwfl_relocate(). But then we would need some new interface to do the relocations on the uncompressed data section later.
This message is a notice that Fedora 15 is now at end of life. Fedora has stopped maintaining and issuing updates for Fedora 15. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At this time, all open bugs with a Fedora 'version' of '15' have been closed as WONTFIX. (Please note: Our normal process is to give advanced warning of this occurring, but we forgot to do that. A thousand apologies.) Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, feel free to reopen this bug and simply change the 'version' to a later Fedora version. Bug Reporter: Thank you for reporting this issue and we are sorry that we were unable to fix it before Fedora 15 reached end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged to click on "Clone This Bug" (top right of this page) and open it against that version of Fedora. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
We still don't handle this.
This bug appears to have been reported against 'rawhide' during the Fedora 19 development cycle. Changing version to '19'. (As we did not run this process for some time, it could affect also pre-Fedora 19 development cycle bugs. We are very sorry. It will help us with cleanup during Fedora 19 End Of Life. Thank you.) More information and reason for this action is here: https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora19
It was fixed in elfutils git: commit f7cdc74ed34adc539186385342d99b357c3dbb6b Author: Mark Wielaard <mjw> Date: Wed Apr 9 22:38:07 2014 +0200 readelf: Read the decompressed data when processing .zdebug DWARF sections. readelf uses libdw to open the Dwarf and read some of the DWARF data. But it also uses its own parsers to display some of the low-level unprocessed data. If the DWARF debug section was zlib compressed it should actually use the decompressed section data from libdw instead of the raw section data. Includes a testcase for those sections that couldn't be properly displayed when compressed before. Signed-off-by: Mark Wielaard <mjw> Git tag elfutils-0.161 works correctly. But elfutils-0.161-2.fc20.x86_64 doesn't.
The problem is /usr/lib64/elfutils: [root@gnu-6 src]# eu-readelf -w -a readelf.o > /dev/null eu-readelf: cannot get debug context descriptor: No DWARF information found eu-readelf: cannot get .debug_loc content: no error eu-readelf: cannot get .debug_aranges content: no error eu-readelf: cannot get .debug_ranges content: no error eu-readelf: cannot get line data section data: no error [root@gnu-6 src]# mv /usr/lib64/elfutils /usr/lib64/elfutils.saved [root@gnu-6 src]# eu-readelf -w -a readelf.o > /dev/null [root@gnu-6 src]#
(In reply to H.J. Lu from comment #10) > It was fixed in elfutils git: > > commit f7cdc74ed34adc539186385342d99b357c3dbb6b > Author: Mark Wielaard <mjw> > Date: Wed Apr 9 22:38:07 2014 +0200 > > readelf: Read the decompressed data when processing .zdebug DWARF > sections. > > readelf uses libdw to open the Dwarf and read some of the DWARF data. > But it also uses its own parsers to display some of the low-level > unprocessed data. If the DWARF debug section was zlib compressed it > should actually use the decompressed section data from libdw instead > of the raw section data. > > Includes a testcase for those sections that couldn't be properly > displayed when compressed before. > > Signed-off-by: Mark Wielaard <mjw> That would "fix" things for non-ET_REL files. Given there are no relocations against the .[z]debug sections. The real problem is that .zdebug sections are ill-defined when there are relocations against the compressed sections. The current setup of the code (relocate.c in libdwfl) will apply the relocations against the section data, then the section will be decompressed when used as DWARF debug data (dwarf_begin_elf.c in libdw). > Git tag elfutils-0.161 works correctly. But elfutils-0.161-2.fc20.x86_64 > doesn't. How are you testing?
(In reply to H.J. Lu from comment #11) > The problem is /usr/lib64/elfutils: > > [root@gnu-6 src]# eu-readelf -w -a readelf.o > /dev/null > eu-readelf: cannot get debug context descriptor: No DWARF information found > eu-readelf: cannot get .debug_loc content: no error > eu-readelf: cannot get .debug_aranges content: no error > eu-readelf: cannot get .debug_ranges content: no error > eu-readelf: cannot get line data section data: no error > [root@gnu-6 src]# mv /usr/lib64/elfutils /usr/lib64/elfutils.saved > [root@gnu-6 src]# eu-readelf -w -a readelf.o > /dev/null I am surprised eu-readelf still works when removing the ebl backend code. It probably should explictly fail instead of limping along. When you do that there will be errors about anything that is machine specific in the ELF file or DWARF data since ebl openbackend will just fail. You'll see lots of INVALID data in the eu-readelf output.
Created attachment 1000196 [details] A hack This a hack to make it work for readelf. But it doesn't work with strip --reloc-debug-sections Ideally elf_getdata should automatically decompress the debug sections. Maybe we can add a callback to convert_data to decompress the debug sections.
Created attachment 1000681 [details] A patch to add elf_begin_convert This still doesn't work with strip --reloc-debug-sections. I will try to fix it.
Created attachment 1000712 [details] A patch This patch passed "make check" on x86-64 and x86-32 with assembler from binutils 2.26, which generates compressed debug section by default.
Created attachment 1000728 [details] A cleaned up patch
It would be best if you could first discuss your design on the upstream mailinglist: elfutils-devel.org. https://fedorahosted.org/mailman/listinfo/elfutils-devel https://git.fedorahosted.org/cgit/elfutils.git/plain/CONTRIBUTING Which make check tests fail before your patch? Adding some explicit test cases would be good, so they can be tested also on platforms that don't use .zdebug sections in ET_REL files. Having to change all programs that depend on libelf now having to link against libdw and use a special case new entry point (dwarf_elf_begin) isn't very attracive. And what about programs that already use libdw dwarf_begin_elf? Those would need to be rewritten? Exporting a new libelf function should be discussed first to see how other libelf implementations handle this.
Created attachment 1001009 [details] An updated patch to use INTUSE
(In reply to Mark Wielaard from comment #18) > It would be best if you could first discuss your design on the upstream > mailinglist: > elfutils-devel.org. > https://fedorahosted.org/mailman/listinfo/elfutils-devel > https://git.fedorahosted.org/cgit/elfutils.git/plain/CONTRIBUTING I will send emails to elfutils-devel. > Which make check tests fail before your patch? Adding some explicit test > cases would be good, so they can be tested also on platforms that don't use > .zdebug sections in ET_REL files. I will add some tests. > Having to change all programs that depend on libelf now having to link > against libdw and use a special case new entry point (dwarf_elf_begin) isn't I am open to all suggestions. I thought about updating elf_begin. But it seems to defeat the purpose of libdw. If a program wants to deal with DWARF, it should use dwarf_elf_begin. > very attracive. And what about programs that already use libdw > dwarf_begin_elf? Those would need to be rewritten? Did you mean dwarf_begin? My change updated dwarf_begin to call dwarf_elf_begin instead of elf_begin. > Exporting a new libelf function should be discussed first to see how other > libelf implementations handle this. I doubt other libelfs support compressed debug sections.
Created attachment 1001037 [details] A patch to add a testcase
(In reply to Mark Wielaard from comment #18) > It would be best if you could first discuss your design on the upstream > mailinglist: > elfutils-devel.org. I have sent an email, which is waiting for moderator approval.
(In reply to H.J. Lu from comment #20) > (In reply to Mark Wielaard from comment #18) > > And what about programs that already use libdw > > dwarf_begin_elf? Those would need to be rewritten? > > Did you mean dwarf_begin? My change updated dwarf_begin to call > dwarf_elf_begin instead of elf_begin. There are two ways to get a Dwarf handle from libdw. dwarf_begin, which takes a file descriptor. And dwarf_begin_elf, which takes an Elf handle. > > Exporting a new libelf function should be discussed first to see how other > > libelf implementations handle this. > > I doubt other libelfs support compressed debug sections. Probably not. But SHF_COMPRESSED did get standardized and at least Solaris seems to support it. They might have also altered their libelf implementation to deal with it: https://groups.google.com/forum/#!topic/generic-abi/dBOS1H47Q64
(In reply to Mark Wielaard from comment #23) > (In reply to H.J. Lu from comment #20) > > (In reply to Mark Wielaard from comment #18) > > > And what about programs that already use libdw > > > dwarf_begin_elf? Those would need to be rewritten? > > > > Did you mean dwarf_begin? My change updated dwarf_begin to call > > dwarf_elf_begin instead of elf_begin. > > There are two ways to get a Dwarf handle from libdw. dwarf_begin, which > takes a file descriptor. And dwarf_begin_elf, which takes an Elf handle. The Elf handle should be created with dwarf_elf_begin if the user wants to support DWARF debug sections. > > > Exporting a new libelf function should be discussed first to see how other > > > libelf implementations handle this. > > > > I doubt other libelfs support compressed debug sections. > > Probably not. But SHF_COMPRESSED did get standardized and at least Solaris > seems to support it. They might have also altered their libelf > implementation to deal with it: > https://groups.google.com/forum/#!topic/generic-abi/dBOS1H47Q64 SHF_COMPRESSED doesn't help the existing relocatable files with compressed debug sections. But infrastructure update in libelf for SHF_COMPRESSED can be used to support compressed debug sections.
NB. strip --reloc-debug-sections will decompress the compressed debug sections and resolve all trivial relocations. It will generate the decompressed debug sections with the compressed debug section names. Both readelf and eu-readelf can handle them correctly.
Moved discussion upstream: https://lists.fedorahosted.org/pipermail/elfutils-devel/2015-March/004623.html
(In reply to Mark Wielaard from comment #23) > > Probably not. But SHF_COMPRESSED did get standardized and at least Solaris > seems to support it. They might have also altered their libelf > implementation to deal with it: > https://groups.google.com/forum/#!topic/generic-abi/dBOS1H47Q64 SHF_COMPRESSED support has been added to gas/binutils/ld on binutils master branch. elfutils should handle compressed debug sections properly.
(In reply to H.J. Lu from comment #27) > (In reply to Mark Wielaard from comment #23) > > > > Probably not. But SHF_COMPRESSED did get standardized and at least Solaris > > seems to support it. They might have also altered their libelf > > implementation to deal with it: > > https://groups.google.com/forum/#!topic/generic-abi/dBOS1H47Q64 > > SHF_COMPRESSED support has been added to gas/binutils/ld on binutils > master branch. elfutils should handle compressed debug sections > properly. So how does the solaris libelf implementation handle SHF_COMPRESSED sections? We should probably match any interface changes they made. In particular how do elf_getdata () and elf_newdata () work for SHF_COMPRESSED sections? Does the compression happen transparently (and does elf_rawdata provide the compressed data) or is this completely up to the user (which would be a pain and wouldn't really help with better compressed section support since it would mean to audit all elf section data accesses in all tools and other libraries based on libelf).
(In reply to Mark Wielaard from comment #28) > So how does the solaris libelf implementation handle SHF_COMPRESSED sections? > We should probably match any interface changes they made. In particular how > do elf_getdata () and elf_newdata () work for SHF_COMPRESSED sections? Does > the compression happen transparently (and does elf_rawdata provide the > compressed data) or is this completely up to the user (which would be a pain > and wouldn't really help with better compressed section support since it > would mean to audit all elf section data accesses in all tools and other > libraries based on libelf). I think the existing functions should decompress compressed section transparently. A new function should be added to return compressed content. We should check with Oracle.
(In reply to Mark Wielaard from comment #28) > So how does the solaris libelf implementation handle SHF_COMPRESSED sections? > We should probably match any interface changes they made. In particular how > do elf_getdata () and elf_newdata () work for SHF_COMPRESSED sections? Does > the compression happen transparently (and does elf_rawdata provide the > compressed data) or is this completely up to the user (which would be a pain > and wouldn't really help with better compressed section support since it > would mean to audit all elf section data accesses in all tools and other > libraries based on libelf). See https://groups.google.com/forum/#!topic/generic-abi/9ZgNqIC2X-4
This really should not have been reopened and retitiled. But should have been a separate bug from the original one (ET_REL files with compressed debug sections not handled). Lets retitle again to make more clear what we are discussing now.
This has been fixed as part of the compressed ELF section support patch series to be included in elfutils 0.165 which will be released in a couple of days. https://lists.fedorahosted.org/archives/list/elfutils-devel%40lists.fedorahosted.org/thread/CJBIABJUMXBAFBLB6MM5C3SXRYDXZ2MP/
elfutils-0.165-2.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2016-a6cf456e31
elfutils-0.165-2.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-a6cf456e31
elfutils-0.165-2.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.
elfutils-0.165-2.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2016-35e3877a89
elfutils-0.165-2.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-35e3877a89
elfutils-0.165-2.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.