Bug 457189 - Unable to Open vmcore from Kdump
Unable to Open vmcore from Kdump
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: binutils (Show other bugs)
5.2
i386 Linux
medium Severity medium
: rc
: ---
Assigned To: Denys Vlasenko
Petr Muller
:
Depends On: 215514 457187
Blocks:
  Show dependency treegraph
 
Reported: 2008-07-30 03:21 EDT by Jan Kratochvil
Modified: 2016-09-19 22:04 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-05-06 11:35:24 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Backported patch to enable elf64-i386 support (4.92 KB, patch)
2009-02-27 11:59 EST, Denys Vlasenko
no flags Details | Diff
RHEL specfile patch (1.56 KB, patch)
2009-02-27 12:00 EST, Denys Vlasenko
no flags Details | Diff
elf64-i386 vmcore for testing (109.42 KB, application/octet-stream)
2009-04-09 08:51 EDT, Denys Vlasenko
no flags Details

  None (edit)
Description Jan Kratochvil 2008-07-30 03:21:23 EDT
+++ This bug was initially created as a clone of Bug #457187 +++

Description of problem:
Looks like `objdump -x' is unable to open ELF64 vmcore containing 80386 machine
code from Kdump in i686 architecture on >4GB of memory.
readelf works fine though.

$ objdump -x /sdb1/vmcore
objdump: /sdb1/vmcore: File format not recognized

$ readelf -a /sdb1/vmcore
ELF Header:
...
  Class:                             ELF64
...
  Machine:                           Intel 80386

Version-Release number of selected component (if applicable):
gdb-6.5-37.el5
kernel-2.6.18-92.el5

How reproducible:
always


-- Additional comment from jan.kratochvil@redhat.com on 2008-07-30 03:09 EST --
Created an attachment (id=312964)
BFD fix for 32-vs-64bits of generic ELF formats.


-- Additional comment from jan.kratochvil@redhat.com on 2008-07-30 03:13 EST --
Created an attachment (id=312965)
BFD AC_SYS_LARGEFILE.
Comment 1 Jan Kratochvil 2008-07-30 03:23:08 EDT
The BFD build also needs: --enable-64-bit-bfd
--enable-targets=x86_64-unknown-linux-gnu
Comment 2 RHEL Product and Program Management 2008-07-30 03:30:14 EDT
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.
Comment 3 Jan Kratochvil 2008-08-01 02:15:39 EDT
In Rawhide 2.18.50.0.8-2.fc10.
Upstream posts at:
http://sourceware.org/ml/binutils/2008-07/msg00399.html
http://sourceware.org/ml/binutils/2008-08/msg00003.html
Comment 4 Eric Bachalo 2008-10-20 18:30:28 EDT
The patches for this issue are considered too invasive to consider including in RHEL 5.3 at this time.  Setting flag for suggested inclusion in RHEL 5.4
Comment 5 Denys Vlasenko 2009-02-27 06:11:36 EST
Porting upstream patches so far didn't produce utilities which recognize such dump on i386 arch (although it works on x86_64).

Tried building F10's binutils-2.18.50.0.9-8.fc10 from srpm. It works on i386:

# objdump -x ~/vmcore

/root/vmcore:     file format elf64-little
/root/vmcore
architecture: UNKNOWN!, flags 0x00000000:

start address 0x0000000000000000

Program Header:
    NOTE off    0x0000000000000158 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**0
         filesz 0x00000000000004b4 memsz 0x00000000000004b4 flags ---

# size ~/vmcore
   text    data     bss     dec     hex filename
3891658752            0       0 3891658752      e7f60000        /root/vmcore (core file)

[root@hp-xw8400-01 redhat]# objdump --version
GNU objdump version 2.18.50.0.9-8 20080822
...
[root@hp-xw8400-01 redhat]# size --version
GNU size version 2.18.50.0.9-8 20080822
...
Comment 6 Denys Vlasenko 2009-02-27 11:59:29 EST
Created attachment 333504 [details]
Backported patch to enable elf64-i386 support
Comment 7 Denys Vlasenko 2009-02-27 12:00:03 EST
Created attachment 333505 [details]
RHEL specfile patch
Comment 8 Denys Vlasenko 2009-02-27 12:02:52 EST
Ok, after some additional hacking I managed to backport the fix to RHEL. See attached files.

Needs additional testing, but so far with these patches i386 RHEL build produces objdump and size binaries which recognize "mutant" elf64-i386 vmcore's :)
Comment 9 RHEL Product and Program Management 2009-03-26 13:22:37 EDT
This request was evaluated by Red Hat Product Management for
inclusion, but this component is not scheduled to be updated in
the current Red Hat Enterprise Linux release. If you would like
this request to be reviewed for the next minor release, ask your
support representative to set the next rhel-x.y flag to "?".
Comment 10 Denys Vlasenko 2009-03-26 14:49:07 EDT
Unfortunately, testing revealed a problem with fixing it in RHEL.
Attached patch works, yes. But it results in binary-incompatible change to libbfd on i386 architecture.

The culprit is here, where in specfile we add i386:

%ifarch sparc ppc s390 i386
CARGS=--enable-64-bit-bfd
%endif

This causes a build problem. In one of build stages, system ar/as is (mistakenly?) tries to use new, freshly-built libbfd. And this does not work, because --enable-64-bit-bfd makes new libbfd binary incompatible with older binaries.

This is arguably a bug in the build system and it can be fixed, but in RHEL binary incompatible changes are not allowed anyway. Thus we can't add --enable-64-bit-bfd like shown above.

Without this change, the rest of the patch works, and it even gives some positive results (for one, on x86-64 "size vmcore" now works), but on i386, it still doesn't recognize elf64-i386 vmcores.

I guess this means we can't fix it in RHEL-5.4
Comment 11 Jan Kratochvil 2009-03-26 15:35:13 EDT
Sorry as while I was talking with Denys I could confuse him.

There is no ABI problem for RHEL as the application-linkable library is being shipped only statically.

libbfd.a (and libiberty.a) - being used only by 3rd party applications from binutils-devel - is already being built separately from libbfd-%{version}.so (and libiberty-%{version}.so) - being used only by the binutils /usr/bin/* executables themselves.  It is being rebuilt from the .spec file:
# Rebuild libiberty.a with -fPIC.
[...]
# Rebuild libbfd.a with -fPIC.
[...]

The ABI problem may only get hit the build process.

Provided libbfd.so and libiberty.so files are only ld script redirections to the .a libraries.
Comment 12 Denys Vlasenko 2009-03-26 16:57:13 EDT
Well, Jan, I guess if library is in /usr/lib[64], then it is public.

And libbfd dynamic library is definitely there, and it is used (by ar, for example):

# ldd `which ar`
        linux-vdso.so.1 =>  (0x00007fff5d9ff000)
        libbfd-2.18.50.0.6-7.fc9.so => /usr/lib64/libbfd-2.18.50.0.6-7.fc9.so (0x000000000060d000)
        libc.so.6 => /lib64/libc.so.6 (0x0000003824800000)
        /lib64/ld-linux-x86-64.so.2 (0x0000003824400000)

Correct me if I am wrong.
Comment 13 Jan Kratochvil 2009-03-26 17:32:43 EDT
To speak specifically about RHEL-5 (but it is the same in Fedora):

You have to link using:
gcc [...] -lbfd-2.17.50.0.6-6.el5
Linking just with common
gcc [...] -lbfd
will use only its static version (due to the special libbfd.so file there).

As its DYNAMIC SONAME contains even the package %{release}:
(SONAME)   Library soname: [libbfd-2.17.50.0.6-6.el5.so]

such linked program will break with the next RHEL-5 binutils update anyway.

So I do not see how anyone could depend on the.so file ABI across new binutils.rpm %{release}.
Comment 14 Denys Vlasenko 2009-03-30 10:58:02 EDT
> So I do not see how anyone could depend on the.so file ABI across new
binutils.rpm %{release}.

Well, "clever" developers (me) had a "bright idea" of testing builds without bumping up %{release}.

Thanks Jan, with bumped-up release it works!
Comment 17 Denys Vlasenko 2009-04-08 14:17:04 EDT
Test RPMs are uploaded to

http://people.redhat.com/dvlasenk/binutils-test-RHEL-5.4/

If you are interested in testing them, please give them a try.

Note that these packages are not official Red Hat packages (yet).
Comment 19 Denys Vlasenko 2009-04-09 08:51:47 EDT
Created attachment 338896 [details]
elf64-i386 vmcore for testing

For me, this works on i386:

# LD_LIBRARY_PATH=. ./size vmcore
   text    data     bss     dec     hex filename
3891658752            0       0 3891658752      e7f60000        vmcore (core file)

# LD_LIBRARY_PATH=. ./objdump -h vmcore

vmcore:     file format elf64-little

Sections:
Idx Name          Size      VMA               LMA               File off  Algn
  0 note0         000004b4  0000000000000000  0000000000000000  00000158  2**0
                  CONTENTS, READONLY
  1 .reg/2535     00000044  0000000000000000  0000000000000000  000001b4  2**2
                  CONTENTS
  2 .reg          00000044  0000000000000000  0000000000000000  000001b4  2**2
                  CONTENTS
  3 .reg/0        00000044  0000000000000000  0000000000000000  00000258  2**2
                  CONTENTS
  4 load1         000a0000  00000000c0000000  0000000000000000  0000060c  2**0
                  CONTENTS, ALLOC, LOAD, CODE
  5 load2         00f00000  00000000c0100000  0000000000100000  000a060c  2**0
                  CONTENTS, ALLOC, LOAD, CODE
  6 load3         2f000000  00000000c9000000  0000000009000000  00fa060c  2**0
                  CONTENTS, ALLOC, LOAD, CODE
  7 load4         b7fc0000  ffffffffffffffff  0000000038000000  2ffa060c  2**0
                  CONTENTS, ALLOC, LOAD, CODE


(note: I zeroed out load2, load3 and load4 sections with dd in order to make vmcore more compressible)
Comment 24 Denys Vlasenko 2009-04-16 11:30:38 EDT
These three additional checks in Alan's patch do not look as they materially change anything, but adopting them will make us closer to upstream.

I am willing to add them.
Comment 27 errata-xmlrpc 2009-05-06 11:35:24 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2009-0465.html

Note You need to log in before you can comment on or make changes to this bug.