This service will be undergoing maintenance at 00:00 UTC, 2016-09-28. It is expected to last about 1 hours

Bug 960048

Summary: Can't load '/usr/lib64/perl5/auto/Fcntl/Fcntl.so' - undefined symbols
Product: [Fedora] Fedora Reporter: Jan Synacek <jsynacek>
Component: openldapAssignee: Jan Synacek <jsynacek>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 18CC: cweyl, ddick, gregory.lee.bartholomew, iarnell, jplesnik, jsynacek, jv+fedora, kasal, lkundrak, mmaslano, perl-devel, phracek, ppisar, psabata, rc040203, rmeggins, tcallawa
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openldap-2.4.35-4.fc18.1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-05-14 00:49:41 EDT Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Attachments:
Description Flags
Reproducer slapd.conf
none
Work-around for openldap none

Description Jan Synacek 2013-05-06 09:39:21 EDT
Created attachment 744147 [details]
Reproducer slapd.conf

Description of problem:
Start slapd and run the following:
# slaptest -f slapd-conf -F . (the config is attached)

This command results in
5187b088 slapd-conf: line 14: error Can't load '/usr/lib64/perl5/auto/Fcntl/Fcntl.so' for module Fcntl: /usr/lib64/perl5/auto/Fcntl/Fcntl.so: undefined symbol: PL_thr_key at /usr/share/perl5/XSLoader.pm line 68.
 at /usr/lib64/perl5/Fcntl.pm line 66.
Compilation f

Adding /usr/lib64/perl5/CORE/libperl.so to LD_PRELOAD works around the problem.


Version-Release number of selected component (if applicable):
openldap-servers-2.4.34-1.fc18.x86_64
perl-5.16.2-240.fc18.x86_64


Steps to Reproduce:
1. See description.

  
Actual results:
Errors about undefined perl symbols are spit.


Expected results:
No errors.
Comment 1 Jan Synacek 2013-05-07 03:21:59 EDT
Happens with perl-5.16.3-242.fc18.x86_64 as well.

Also:
# ldd /usr/lib64/perl5/auto/Fcntl/Fcntl.so
	linux-vdso.so.1 =>  (0x00007fff0ad6c000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f1c72ebc000)
	/lib64/ld-linux-x86-64.so.2 (0x000000370f200000)

No libperl.so here. Shouldn't Fcntl.so be linked with libperl as well? I'm not sure what the correct solution here is.
Comment 2 Petr Pisar 2013-05-07 05:17:13 EDT
Fcntl.so is plug-in into libperl.so, so there is no DT_NEED on libperl.so. That's Ok.

I checked the openldap's back_perl.so and it's built properly:

$ scanelf -nr /usr/lib64/openldap/back_perl-2.4.so.2
 TYPE   RPATH NEEDED FILE 
ET_DYN /usr/lib64/perl5/CORE libperl.so,libnsl.so.1,libm.so.6,libcrypt.so.1,libutil.so.1,libldap_r-2.4.so.2,libsasl2.so.2,libssl3.so,libsmime3.so,libnss3.so,libnssutil3.so,libplds4.so,libplc4.so,libnspr4.so,libpthread.so.0,libdl.so.2,liblber-2.4.so.2,libresolv.so.2,libc.so.6 /usr/lib64/openldap/back_perl-2.4.so.2 

I suspect slapd dlopens the back_perl.so with strange linking flags that makes some libperl symbols unavailable.

openldap relies to lt_dlopenext(), and libtool library documentation states some funny things about threads which is the symbol reported by the error message:

Note that libltdl is not well tested in a multithreaded environment, though the intention is that it should work (see Using libltdl in a multi threaded environment). It was reported that GNU/Linux's glibc 2.0's dlopen with ‘RTLD_LAZY’ (which libltdl uses by default) is not thread-safe, but this problem is supposed to be fixed in glibc 2.1. On the other hand, ‘RTLD_NOW’ was reported to introduce problems in multi-threaded applications on FreeBSD. Working around these problems is left as an exercise for the reader; contributions are certainly welcome.
Comment 3 Petr Pisar 2013-05-07 08:08:18 EDT
There is excellent analysis of the problem in Debian bug tracking system <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=327585#50>. It's as I suspected:

The libtool dynamic loader lt_dlopen() used by openldap does not allow loading modules from modules and that is exactly what openldap does. slapd loads back_perl module that pulls in libperl symbols, but libperl loads Fork.so module that is not linked against libperl (for a good reason, to prevent from double-linking). And because ld_dlopen prohibits exporting resolved symbols globally, i.e. the libperl symbols, Fork.so, or any XS Perl module, will not see libperl symbols.

There is no easy fix:

(1) One can change openldap to use dlopen() instead of lt_dlopen().

(2) One can change libtool to export all symbols globally.

(3) One can start linking all Perl XS modules against libperl.
Comment 4 Petr Pisar 2013-05-07 08:44:32 EDT
Created attachment 744718 [details]
Work-around for openldap

This is copy of patch used by Debian <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=327585> to make slapd exporting all module symbols globally.
Comment 5 Petr Pisar 2013-05-07 08:48:47 EDT
Current libtool allows to change export strategy to RTLD_GLOBAL. That merged (1) and (2) proposals into one solution on openldap side.

The (3) proposal is not doable in released Fedora because complete fix would require rebuilding all Perl packages (more the 2000).

So I propose to implement the work-around in openldap.
Comment 6 Jan Synacek 2013-05-09 03:16:19 EDT
Thanks, Petr!

As stated in the debian bugreport:

"... the *right* fix is to fix perl ... so that everything is linked against libperl.  All other solutions are workarounds, not fixes."

And also:

"The patch proposed ... carries side effects, because it will cause openldap to open all modules with RTLD_GLOBAL. This increases the risk of a symbol collision causing openldap to crash (the precise issue that libltdl was switched to RTLD_LOCAL to avoid), and even if that doesn't result in a bug now, it might do so in the future.

But it seems to be the best we can do short of the perl fix ... "

... which in our case is to rebuild more than 2000 packages.

So yes, I'll apply this workaround.
Comment 8 Fedora Update System 2013-05-09 05:13:02 EDT
openldap-2.4.35-4.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/openldap-2.4.35-4.fc19
Comment 9 Fedora Update System 2013-05-09 09:38:45 EDT
openldap-2.4.35-4.fc18.1 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/openldap-2.4.35-4.fc18.1
Comment 10 Fedora Update System 2013-05-09 13:56:27 EDT
Package openldap-2.4.35-4.fc19:
* should fix your issue,
* was pushed to the Fedora 19 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing openldap-2.4.35-4.fc19'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-7759/openldap-2.4.35-4.fc19
then log in and leave karma (feedback).
Comment 11 Fedora Update System 2013-05-14 00:49:41 EDT
openldap-2.4.35-4.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 12 Fedora Update System 2013-05-21 04:46:21 EDT
openldap-2.4.35-4.fc18.1 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.