Bug 1395758 - glibc: incomplete rollback of dynamic linker state on linking failure
glibc: incomplete rollback of dynamic linker state on linking failure
Status: CLOSED UPSTREAM
Product: Fedora
Classification: Fedora
Component: glibc (Show other bugs)
24
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: glibc team
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-11-16 10:36 EST by Stephan Bergmann
Modified: 2017-07-27 07:03 EDT (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-07-27 07:03:07 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Sourceware 20839 None None None 2017-07-27 07:03 EDT

  None (edit)
Description Stephan Bergmann 2016-11-16 10:36:56 EST
Description of problem:

On both F24 and F25 x86-64, running Apache OpenOffice causes crash within glibc dlopen.


Version-Release number of selected component (if applicable):

glibc-2.23.1-11.fc24.x86_64
glibc-2.24-3.fc25.x86_64


How reproducible:

always


Steps to Reproduce:

> $ wget http://downloads.sourceforge.net/project/openofficeorg.mirror/4.1.3/binaries/en-US/Apache_OpenOffice_4.1.3_Linux_x86-64_install-rpm_en-US.tar.gz
...
> $ tar xf Apache_OpenOffice_4.1.3_Linux_x86-64_install-rpm_en-US.tar.gz
> $ sudo dnf install en-US/RPMS/openoffice-*.rpm
...
> $ gdb /opt/openoffice4/program/soffice.bin
...
> (gdb) run
...
> Thread 1 "soffice.bin" received signal SIGSEGV, Segmentation fault.
> do_lookup_x (undef_name=undef_name@entry=0x7fffcef68b13 "ffi_prep_closure_loc", new_hash=new_hash@entry=371325385, old_hash=old_hash@entry=0x7fffffffbb70, ref=0x7fffcef68730, result=result@entry=0x7fffffffbb80, scope=<optimized out>, i=0, version=0x0, flags=1, skip=<optimized out>, type_class=4, undef_map=0x661770) at dl-lookup.c:366
> 366	      const struct link_map *map = list[i]->l_real;
> Missing separate debuginfos, use: dnf debuginfo-install openoffice-4.1.3-9783.x86_64
> (gdb) backtrace
> #0  0x00007ffff7de2738 in do_lookup_x (undef_name=undef_name@entry=0x7fffcef68b13 "ffi_prep_closure_loc", new_hash=new_hash@entry=371325385, old_hash=old_hash@entry=0x7fffffffbb70, ref=0x7fffcef68730, result=result@entry=0x7fffffffbb80, scope=<optimized out>, i=0, version=0x0, flags=1, skip=<optimized out>, type_class=4, undef_map=0x661770) at dl-lookup.c:366
> #1  0x00007ffff7de3322 in _dl_lookup_symbol_x (undef_name=0x7fffcef68b13 "ffi_prep_closure_loc", undef_map=undef_map@entry=0x661770, ref=ref@entry=0x7fffffffbcd0, symbol_scope=symbol_scope@entry=0x661ac8, version=0x0, type_class=4, flags=1, skip_map=0x0) at dl-lookup.c:829
> #2  0x00007ffff7de4f29 in _dl_relocate_object (skip_ifunc=0, reloc_addr_arg=0x7fffcf16ee88, version=<optimized out>, sym=<optimized out>, reloc=0x7fffcef69130, map=0x661770) at ../sysdeps/x86_64/dl-machine.h:301
> #3  0x00007ffff7de4f29 in _dl_relocate_object (skip_ifunc=0, lazy=<optimized out>, nrelative=<optimized out>, relsize=<optimized out>, reladdr=<optimized out>, map=0x661770) at do-rel.h:137
> #4  0x00007ffff7de4f29 in _dl_relocate_object (scope=<optimized out>, reloc_mode=reloc_mode@entry=1, consider_profiling=<optimized out>, consider_profiling@entry=0) at dl-reloc.c:258
> #5  0x00007ffff7dedc71 in dl_open_worker (a=a@entry=0x7fffffffc060) at dl-open.c:424
> #6  0x00007ffff7de8c54 in _dl_catch_error (objname=objname@entry=0x7fffffffc050, errstring=errstring@entry=0x7fffffffc058, mallocedp=mallocedp@entry=0x7fffffffc04f, operate=operate@entry=0x7ffff7ded660 <dl_open_worker>, args=args@entry=0x7fffffffc060) at dl-error.c:187
> #7  0x00007ffff7ded009 in _dl_open (file=0x7fffffffc2c0 "/opt/openoffice4/program/../program/ucpgvfs1.uno.so", mode=-2147483391, caller_dlopen=0x7ffff7a0ea2b <osl_loadModule+171>, nsid=-2, argc=<optimized out>, argv=<optimized out>, env=0x61b560) at dl-open.c:649
> #8  0x00007ffff615af09 in dlopen_doit (a=a@entry=0x7fffffffc290) at dlopen.c:66
> #9  0x00007ffff7de8c54 in _dl_catch_error (objname=0x61b540, errstring=0x61b548, mallocedp=0x61b538, operate=0x7ffff615aeb0 <dlopen_doit>, args=0x7fffffffc290) at dl-error.c:187
> #10 0x00007ffff615b591 in _dlerror_run (operate=operate@entry=0x7ffff615aeb0 <dlopen_doit>, args=args@entry=0x7fffffffc290) at dlerror.c:163
> #11 0x00007ffff615afa2 in __dlopen (file=<optimized out>, mode=<optimized out>) at dlopen.c:87
> #12 0x00007ffff7a0ea2b in osl_loadModule () at /opt/openoffice4/program/libuno_sal.so.3
> #13 0x00007ffff599b7e5 in cppu::loadSharedLibComponentFactory(rtl::OUString const&, rtl::OUString const&, rtl::OUString const&, com::sun::star::uno::Reference<com::sun::star::lang::XMultiServiceFactory> const&, com::sun::star::uno::Reference<com::sun::star::registry::XRegistryKey> const&) () at /opt/openoffice4/program/libuno_cppuhelpergcc3.so.3
> #14 0x00007fffe17ad25e in  () at /opt/openoffice4/program/bootstrap.uno.so
> #15 0x00007ffff598a193 in  () at /opt/openoffice4/program/libuno_cppuhelpergcc3.so.3
> #16 0x00007ffff598acea in  () at /opt/openoffice4/program/libuno_cppuhelpergcc3.so.3
> #17 0x00007ffff598770a in  () at /opt/openoffice4/program/libuno_cppuhelpergcc3.so.3
> #18 0x00007ffff598792f in  () at /opt/openoffice4/program/libuno_cppuhelpergcc3.so.3
> #19 0x00007fffe177a956 in  () at /opt/openoffice4/program/bootstrap.uno.so
> #20 0x00007fffe1770d4c in  () at /opt/openoffice4/program/bootstrap.uno.so
> #21 0x00007ffff778a393 in  () at /opt/openoffice4/program/libsofficeapp.so
> #22 0x00007ffff778abb9 in  () at /opt/openoffice4/program/libsofficeapp.so
> #23 0x00007ffff777c3d0 in  () at /opt/openoffice4/program/libsofficeapp.so
> #24 0x00007ffff2c217eb in  () at /opt/openoffice4/program/libvcl.so
> #25 0x00007ffff2c218b6 in SVMain() () at /opt/openoffice4/program/libvcl.so
> #26 0x00007ffff77a4f8c in soffice_main () at /opt/openoffice4/program/libsofficeapp.so
> #27 0x0000000000400f7b in main ()
> (gdb) info registers
> rax            0x9	9
> rbx            0x661ad0	6691536
> rcx            0x7fffcef68730	140736665650992
> rdx            0x7fffffffbb70	140737488337776
> rsi            0x1621f9c9	371325385
> rdi            0x7fffcef68b13	140736665651987
> rbp            0x0	0x0
> rsp            0x7fffffffba20	0x7fffffffba20
> r8             0x7fffffffbb80	140737488337792
> r9             0x61daf8	6413048
> r10            0x0	0
> r11            0x7fffebc22bd8	140737148759000
> r12            0x0	0
> r13            0x0	0
> r14            0x0	0
> r15            0x0	0
> rip            0x7ffff7de2738	0x7ffff7de2738 <do_lookup_x+136>
> eflags         0x10206	[ PF IF RF ]
> cs             0x33	51
> ss             0x2b	43
> ds             0x0	0
> es             0x0	0
> fs             0x0	0
> gs             0x0	0
Comment 1 Florian Weimer 2016-11-16 16:06:07 EST
The crash happens while relocating a reference to the __ctype_toupper_loc symbol in /lib64/libglib-2.0.so, after a dlopen of /opt/openoffice4/program/../program/libfwk.so.  The crash does not happen if this DSO is opened by itself.

We end up here in elf/do-rel.h:

    114 #ifdef RTLD_BOOTSTRAP
    115       /* The dynamic linker always uses versioning.  */
    116       assert (map->l_info[VERSYMIDX (DT_VERSYM)] != NULL);
    117 #else
    118       if (map->l_info[VERSYMIDX (DT_VERSYM)])
    119 #endif
    120         {
    121           const ElfW(Half) *const version =
    122             (const void *) D_PTR (map, l_info[VERSYMIDX (DT_VERSYM)]);
    123 
    124           for (; r < end; ++r)
    125             {
    126 #if defined ELF_MACHINE_IRELATIVE && !defined RTLD_BOOTSTRAP
    127               if (ELFW(R_TYPE) (r->r_info) == ELF_MACHINE_IRELATIVE)
    128                 {
    129                   if (r2 == NULL)
    130                     r2 = r;
    131                   end2 = r;
    132                   continue;
    133                 }
    134 #endif
    135 
    136               ElfW(Half) ndx = version[ELFW(R_SYM) (r->r_info)] & 0x7fff    136 ;
    137               elf_machine_rel (map, r, &symtab[ELFW(R_SYM) (r->r_info)],
    138                                &map->l_versions[ndx],
    139                                (void *) (l_addr + r->r_offset), skip_ifu    139 nc);
    140             }

At line 137, map->l_versions is NULL:

(gdb) print map->l_versions 
$7 = (struct r_found_version *) 0x0

But this is clearly wrong because glib has symbol versioning information (both on disk and as indicated by map->l_info[VERSYMIDX (DT_VERSYM)], otherwise we would not have entered that conditional.  I'm not sure if _dl_check_map_versions has run at this point.
Comment 2 Florian Weimer 2016-11-17 06:24:07 EST
It's not a recent glibc change which causes this.  I went back to glibc 2.18, and it exhibits the same problem (still with the Fedora 24 system DSOs).
Comment 3 Carlos O'Donell 2016-11-17 11:24:18 EST
(In reply to Florian Weimer from comment #1)
> But this is clearly wrong because glib has symbol versioning information
> (both on disk and as indicated by map->l_info[VERSYMIDX (DT_VERSYM)],
> otherwise we would not have entered that conditional.  I'm not sure if
> _dl_check_map_versions has run at this point.

Debugging this code is exceedingly difficult because of the relocation restrictions in the early startup and in general -O2. My expectation is that gdb is simply wrong and that map->l_versions is not NULL.

The interesting part is the SIGSEGV. Any chance you can get a trimmed down reproducer? I'm looking at extending our tracer into dlopen to be able to take a bunch of DSOs and simulate a startup load sequence, just to debug these kinds of issues (cycles, constructor orders etc.).
Comment 4 Stephan Bergmann 2016-11-17 11:49:37 EST
(In reply to Carlos O'Donell from comment #3)
> The interesting part is the SIGSEGV. Any chance you can get a trimmed down
> reproducer?

Unfortunately not.  And I have no idea what toolchain was used to produce it and what the platforms are where it is actually supposed to work.  I just assumed that it was build in a way generic enough to let it work on any contemporary distro.  Seeing it crash I just wanted to inform you about it, in case it was something obvious.
Comment 5 Florian Weimer 2016-11-17 12:04:51 EST
(In reply to Carlos O'Donell from comment #3)
> (In reply to Florian Weimer from comment #1)
> > But this is clearly wrong because glib has symbol versioning information
> > (both on disk and as indicated by map->l_info[VERSYMIDX (DT_VERSYM)],
> > otherwise we would not have entered that conditional.  I'm not sure if
> > _dl_check_map_versions has run at this point.
> 
> Debugging this code is exceedingly difficult because of the relocation
> restrictions in the early startup and in general -O2. My expectation is that
> gdb is simply wrong and that map->l_versions is not NULL.

No, I set a conditional breakpoint well before the SIGSEGV, and a _dl_printf reports:

elf_dynamic_do_Rela: NULL l_versions for: /lib64/libglib-2.0.so.0

Note that the crash happens after dlopen, not during the initial link, which means that GDB should be fairly reliable (if it works at all, but due to aggressive inlining, this is often not the case).
Comment 6 Florian Weimer 2016-11-17 12:41:20 EST
LD_DEBUG=files with some additional _dl_printf calls gives this output:

__dlopen: /opt/openoffice4/program/libvclplug_gtk.so
_dl_map_object: /opt/openoffice4/program/libvclplug_gtk.so
      8466:
      8466:     file=/opt/openoffice4/program/libvclplug_gtk.so [0];  dynamically loaded by /opt/openoffice4/program/libuno_sal.so.3 [0]
      8466:     file=/opt/openoffice4/program/libvclplug_gtk.so [0];  generating link map
      8466:       dynamic: 0x00007fffeb5bb488  base: 0x00007fffeb367000   size: 0x0000000000256870
      8466:         entry: 0x00007fffeb37c7a0  phdr: 0x00007fffeb367040  phnum:                  5
      8466:
dl_open_worker: loading dependencies for: /opt/openoffice4/program/libvclplug_gtk.so
_dl_map_object: libvclplug_gen.so
      8466:
      8466:     file=libvclplug_gen.so [0];  needed by /opt/openoffice4/program/libvclplug_gtk.so [0]
      8466:     file=libvclplug_gen.so [0];  generating link map
      8466:       dynamic: 0x00007fffeb3617a8  base: 0x00007fffeb091000   size: 0x00000000002d5830
      8466:         entry: 0x00007fffeb0c3a80  phdr: 0x00007fffeb091040  phnum:                  5
      8466:
_dl_map_object: libSM.so.6
      8466:
      8466:     file=libSM.so.6 [0];  needed by /opt/openoffice4/program/libvclplug_gtk.so [0]
      8466:     file=libSM.so.6 [0];  generating link map
      8466:       dynamic: 0x00007fffeb08fc68  base: 0x00007fffeae89000   size: 0x0000000000207030
      8466:         entry: 0x00007fffeae8aad0  phdr: 0x00007fffeae89040  phnum:                  7
      8466:
_dl_map_object: libICE.so.6
      8466:
      8466:     file=libICE.so.6 [0];  needed by /opt/openoffice4/program/libvclplug_gtk.so [0]
      8466:     file=libICE.so.6 [0];  generating link map
      8466:       dynamic: 0x00007fffeae84910  base: 0x00007fffeac6d000   size: 0x000000000021bca0
      8466:         entry: 0x00007fffeac71b10  phdr: 0x00007fffeac6d040  phnum:                  7
      8466:
_dl_map_object: libdbus-glib-1.so.2
      8466:
      8466:     file=libdbus-glib-1.so.2 [0];  needed by /opt/openoffice4/program/libvclplug_gtk.so [0]
      8466:     file=libdbus-glib-1.so.2 [0];  generating link map
      8466:       dynamic: 0x00007fffeac6b0f0  base: 0x00007fffeaa40000   size: 0x000000000022c178
      8466:         entry: 0x00007fffeaa4a1a0  phdr: 0x00007fffeaa40040  phnum:                  7
      8466:
_dl_map_object: libdbus-1.so.3
      8466:
      8466:     file=libdbus-1.so.3 [0];  needed by /opt/openoffice4/program/libvclplug_gtk.so [0]
      8466:     file=libdbus-1.so.3 [0];  generating link map
      8466:       dynamic: 0x00007fffeaa3df08  base: 0x00007fffea7f0000   size: 0x000000000024f2d0
      8466:         entry: 0x00007fffea7fd740  phdr: 0x00007fffea7f0040  phnum:                  7
      8466:
_dl_map_object: libgobject-2.0.so.0
      8466:
      8466:     file=libgobject-2.0.so.0 [0];  needed by /opt/openoffice4/program/libvclplug_gtk.so [0]
      8466:     file=libgobject-2.0.so.0 [0];  generating link map
      8466:       dynamic: 0x00007fffea7ee800  base: 0x00007fffea59e000   size: 0x0000000000251b88
      8466:         entry: 0x00007fffea5a8340  phdr: 0x00007fffea59e040  phnum:                  7
      8466:
_dl_map_object: libglib-2.0.so.0
      8466:
      8466:     file=libglib-2.0.so.0 [0];  needed by /opt/openoffice4/program/libvclplug_gtk.so [0]
      8466:     file=libglib-2.0.so.0 [0];  generating link map
      8466:       dynamic: 0x00007fffea59c700  base: 0x00007fffea290000   size: 0x000000000030df18
      8466:         entry: 0x00007fffea2aa2b0  phdr: 0x00007fffea290040  phnum:                  7
      8466:
_dl_map_object: libgtk-x11-2.0.so.0
      8466:
      8466:     file=libgtk-x11-2.0.so.0 [0];  needed by /opt/openoffice4/program/libvclplug_gtk.so [0]
      8466:     file=libgtk-x11-2.0.so.0 [0];  generating link map
      8466:       dynamic: 0x00007fffea289598  base: 0x00007fffe9c0a000   size: 0x0000000000685d38
      8466:         entry: 0x00007fffe9c70030  phdr: 0x00007fffe9c0a040  phnum:                  7
      8466:
_dl_map_object: libgdk-x11-2.0.so.0
      8466:
      8466:     file=libgdk-x11-2.0.so.0 [0];  needed by /opt/openoffice4/program/libvclplug_gtk.so [0]
      8466:     file=libgdk-x11-2.0.so.0 [0];  generating link map
      8466:       dynamic: 0x00007fffe9c07900  base: 0x00007fffe9949000   size: 0x00000000002c0758
      8466:         entry: 0x00007fffe99662c0  phdr: 0x00007fffe9949040  phnum:                  7
      8466:
_dl_map_object: libatk-1.0.so.0
      8466:
      8466:     file=libatk-1.0.so.0 [0];  needed by /opt/openoffice4/program/libvclplug_gtk.so [0]
      8466:     file=libatk-1.0.so.0 [0];  generating link map
      8466:       dynamic: 0x00007fffe9947710  base: 0x00007fffe9723000   size: 0x0000000000225498
      8466:         entry: 0x00007fffe972db30  phdr: 0x00007fffe9723040  phnum:                  7
      8466:
_dl_map_object: libpangocairo-1.0.so.0
      8466:
      8466:     file=libpangocairo-1.0.so.0 [0];  needed by /opt/openoffice4/program/libvclplug_gtk.so [0]
      8466:     file=libpangocairo-1.0.so.0 [0];  generating link map
      8466:       dynamic: 0x00007fffe97217e8  base: 0x00007fffe9516000   size: 0x000000000020c098
      8466:         entry: 0x00007fffe951a3c0  phdr: 0x00007fffe9516040  phnum:                  7
      8466:
_dl_map_object: libpango-1.0.so.0
      8466:
      8466:     file=libpango-1.0.so.0 [0];  needed by /opt/openoffice4/program/libvclplug_gtk.so [0]
      8466:     file=libpango-1.0.so.0 [0];  generating link map
      8466:       dynamic: 0x00007fffe95148a8  base: 0x00007fffe92cb000   size: 0x000000000024a288
      8466:         entry: 0x00007fffe92d78f0  phdr: 0x00007fffe92cb040  phnum:                  7
      8466:
_dl_map_object: libcairo.so.2
      8466:
      8466:     file=libcairo.so.2 [0];  needed by /opt/openoffice4/program/libvclplug_gtk.so [0]
      8466:     file=libcairo.so.2 [0];  generating link map
      8466:       dynamic: 0x00007fffe92c7030  base: 0x00007fffe8fa2000   size: 0x00000000003281d0
      8466:         entry: 0x00007fffe8fb4d20  phdr: 0x00007fffe8fa2040  phnum:                  7
      8466:
_dl_map_object: libgdk_pixbuf_xlib-2.0.so.0
      8466:
      8466:     file=libgdk_pixbuf_xlib-2.0.so.0 [0];  needed by /opt/openoffice4/program/libvclplug_gtk.so [0]
_dl_signal_error: reporting link error in libgdk_pixbuf_xlib-2.0.so.0: cannot open shared object file
_dl_signal_error: reporting link error in libgdk_pixbuf_xlib-2.0.so.0: cannot open shared object file
      8466:
      8466:     file=/opt/openoffice4/program/libvclplug_gtk.so [0];  destroying link map
      8466:
      8466:     file=/opt/openoffice4/program/libvclplug_gen.so [0];  destroying link map
      8466:
      8466:     file=/lib64/libSM.so.6 [0];  destroying link map
      8466:
      8466:     file=/lib64/libICE.so.6 [0];  destroying link map
      8466:
      8466:     file=/lib64/libdbus-glib-1.so.2 [0];  destroying link map
      8466:
      8466:     file=/lib64/libdbus-1.so.3 [0];  destroying link map
      8466:
      8466:     file=/lib64/libgtk-x11-2.0.so.0 [0];  destroying link map
      8466:
      8466:     file=/lib64/libgdk-x11-2.0.so.0 [0];  destroying link map
      8466:
      8466:     file=/lib64/libatk-1.0.so.0 [0];  destroying link map
      8466:
      8466:     file=/lib64/libpangocairo-1.0.so.0 [0];  destroying link map
      8466:
      8466:     file=/lib64/libpango-1.0.so.0 [0];  destroying link map
      8466:
      8466:     file=/lib64/libcairo.so.2 [0];  destroying link map
_dl_signal_error: reporting link error in libgdk_pixbuf_xlib-2.0.so.0: cannot open shared object file
__dlopen: /opt/openoffice4/program/libvclplug_gtk.so RETURNED WITH ERROR

Note how a link map for libglib-2.0.so.0 is set up, but never destroyed.  This is clearly a dynamic linker bug.

Stephan, the crash is triggered by a missing dependency in the OpenOffice RPMs.  You can install the gdk-pixbuf2-xlib package manually, and the crash will be gone.  I didn't test things further, but I could get to the “Welcome to OpenOffice 4.1.3” screen.
Comment 7 Florian Weimer 2016-11-18 08:08:52 EST
Note that /usr/lib64/libglib-2.0.so.0 has NODELETE set, which could be the reason why it is not subject to cleanup.
Comment 8 Fedora End Of Life 2017-07-25 19:54:27 EDT
This message is a reminder that Fedora 24 is nearing its end of life.
Approximately 2 (two) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 24. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '24'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 24 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.
Comment 9 Florian Weimer 2017-07-27 07:03:07 EDT
We are tracking this upstream.

Note You need to log in before you can comment on or make changes to this bug.