Description of problem: FreeIPA's ipa-kdb plugin from git master fails to compile on rawhide. /usr/lib64/libndr-standard.so and /usr/lib64/libsamba-util.so export duplicate symbols "_end" Version-Release number of selected component (if applicable): samba-devel-4.8.3-2.fc29.1.x86_64 How reproducible: always Steps to Reproduce: 1. https://github.com/freeipa/freeipa.git 2. cd freeipa 3. dnf builddep -b -D "with_python2 0" -D "with_wheels 1" -D "with_lint 1" --spec freeipa.spec.in --best --allowerasing 4. ./autogen.sh 5. make Actual results: make[3]: Entering directory '/root/freeipa/daemons/ipa-kdb' /bin/sh ../../libtool --tag=CC --mode=link gcc -Wall -Wshadow -Wstrict-prototypes -Wpointer-arith -Wcast-align -Werror-implicit-function-declaration -g -O2 -Werror=implicit-function-declaration -avoid-version -module -Wl,--version-script,./ipa_kdb.exports -o ipadb.la -rpath /usr/local/lib/krb5/plugins/kdb ipa_kdb.lo ipa_kdb_common.lo ipa_kdb_mkey.lo ipa_kdb_passwords.lo ipa_kdb_principals.lo ipa_kdb_pwdpolicy.lo ipa_kdb_mspac.lo ipa_kdb_delegation.lo ipa_kdb_audit_as.lo ipa_kdb_certauth.lo -lkrb5 -lk5crypto -lcom_err -lldap_r -llber -lndr-krb5pac -lndr-standard -lndr -lsamba-util -ltevent -ltalloc -lunistring -lssl3 -lsmime3 -lnss3 -lnssutil3 -lplds4 -lplc4 -lnspr4 -lpthread -ldl -lsss_certmap ../../util/libutil.la libtool: link: gcc -shared -fPIC -DPIC .libs/ipa_kdb.o .libs/ipa_kdb_common.o .libs/ipa_kdb_mkey.o .libs/ipa_kdb_passwords.o .libs/ipa_kdb_principals.o .libs/ipa_kdb_pwdpolicy.o .libs/ipa_kdb_mspac.o .libs/ipa_kdb_delegation.o .libs/ipa_kdb_audit_as.o .libs/ipa_kdb_certauth.o -Wl,--whole-archive ../../util/.libs/libutil.a -Wl,--no-whole-archive -lndr-krb5pac -lndr-standard -lndr -lsamba-util -ltevent -ltalloc -lunistring -lsss_certmap -lcrypto -lkrb5 -lk5crypto -lcom_err -lldap_r -llber -lssl3 -lsmime3 -lnss3 -lnssutil3 -lplds4 -lplc4 -lnspr4 -lpthread -ldl -g -O2 -Wl,--version-script -Wl,./ipa_kdb.exports -Wl,-soname -Wl,ipadb.so -o .libs/ipadb.so /usr/bin/ld: /usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/libndr-standard.so:(*IND*+0x0): multiple definition of `_end' /usr/bin/ld: /usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/libsamba-util.so:(*IND*+0x0): multiple definition of `_end' Expected results: no error Additional info: # objdump -TC /usr/lib64/libndr-standard.so | grep " _end" 0000000000339001 g D .data.rel.ro.local 0000000000000000 NDR_STANDARD_0.0.1 _end 0000000000339001 g D .data.rel.ro.local 0000000000000000 (NDR_STANDARD_0.0.1) _end # objdump -TC /usr/lib64/libsamba-util.so | grep " _end" 000000000007a2f0 g D .data.rel.ro.local 0000000000000000 SAMBA_UTIL_0.0.1 _end 000000000007a2f0 g D .data.rel.ro.local 0000000000000000 (SAMBA_UTIL_0.0.1) _end
This may be a duplicate of BZ #1599441. # rpm -qa binutils samba gcc samba-4.8.3-2.fc29.1.x86_64 gcc-8.1.1-4.fc29.1.x86_64 binutils-2.30.90-1.fc29.x86_64
FreeIPA's ipa-sam plugin also fails to compile: make[3]: Entering directory '/root/freeipa/daemons/ipa-sam' /bin/sh ../../libtool --tag=CC --mode=link gcc -Wall -Wshadow -Wstrict-prototypes -Wpointer-arith -Wcast-align -Werror-implicit-function-declaration -g -O2 -Werror=implicit-function-declaration -avoid-version -module -o ipasam.la -rpath /usr/local/lib/samba/pdb ipa_sam.lo -lcrypto -lldap_r -llber -lkrb5 -lk5crypto -lcom_err -ltalloc -lsamba-util -ltevent -ltalloc -lndr -lsamba-util -ltevent -ltalloc -L/usr/lib64/samba -Wl,-rpath=/usr/lib64/samba -lsmbldap -lpdb -lsmbconf -lsss_idmap ../../asn1/libipaasn1.la ../../util/libutil.la libtool: link: gcc -shared -fPIC -DPIC .libs/ipa_sam.o -Wl,--whole-archive ../../asn1/.libs/libipaasn1.a ../../util/.libs/libutil.a -Wl,--no-whole-archive -lndr -lsamba-util -ltevent -ltalloc -L/usr/lib64/samba -lsmbldap -lpdb -lsmbconf -lsss_idmap -lcrypto -lkrb5 -lk5crypto -lcom_err -lldap_r -llber -lssl3 -lsmime3 -lnss3 -lnssutil3 -lplds4 -lplc4 -lnspr4 -lpthread -ldl -g -O2 -Wl,-rpath=/usr/lib64/samba -Wl,-soname -Wl,ipasam.so -o .libs/ipasam.so /usr/bin/ld: cannot find -lpdb /usr/bin/ld: /usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/libsmbconf.so:(*IND*+0x0): multiple definition of `_edata' /usr/bin/ld: /usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/libsmbconf.so:(*IND*+0x0): multiple definition of `__bss_start' /usr/bin/ld: /usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/libsmbconf.so:(*IND*+0x0): multiple definition of `_end'
Nick, could you please explain is this change in binutils behavior related to https://sourceware.org/ml/binutils/2018-05/msg00127.html and how we should fix it?
I guess Samba packages have be rebuild after the binutil bug has been fixed.
I pushed a rebuild https://koji.fedoraproject.org/koji/taskinfo?taskID=28126741
Alexander's rebuild didn't the issue: # koji download-task 28126841 ... # rm samba-4.8.3-3.fc29.1.src.rpm # dnf install -y ./*.rpm ... # rpm -qa samba-devel samba-devel-4.8.3-3.fc29.1.x86_64 # make ... libtool: link: gcc -shared -fPIC -DPIC .libs/ipa_kdb.o .libs/ipa_kdb_common.o .libs/ipa_kdb_mkey.o .libs/ipa_kdb_passwords.o .libs/ipa_kdb_principals.o .libs/ipa_kdb_pwdpolicy.o .libs/ipa_kdb_mspac.o .libs/ipa_kdb_delegation.o .libs/ipa_kdb_audit_as.o .libs/ipa_kdb_certauth.o -Wl,--whole-archive ../../util/.libs/libutil.a -Wl,--no-whole-archive -lndr-krb5pac -lndr-standard -lndr -lsamba-util -ltevent -ltalloc -lunistring -lsss_certmap -lcrypto -lkrb5 -lk5crypto -lcom_err -lldap_r -llber -lssl3 -lsmime3 -lnss3 -lnssutil3 -lplds4 -lplc4 -lnspr4 -lpthread -ldl -g -O2 -Wl,--version-script -Wl,./ipa_kdb.exports -Wl,-soname -Wl,ipadb.so -o .libs/ipadb.so /usr/bin/ld: /usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/libndr-standard.so:(*IND*+0x0): multiple definition of `_end' /usr/bin/ld: /usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/libsamba-util.so:(*IND*+0x0): multiple definition of `_end'
Yes, rebuilding against binutils 2.30.90-2.fc29 does not fix the problem. We need a guidance from binutils guys.
(In reply to Alexander Bokovoy from comment #3) > Nick, could you please explain is this change in binutils behavior related > to https://sourceware.org/ml/binutils/2018-05/msg00127.html and how we > should fix it? Right - H.J's patch, and several others that followed it, are all in the about-to-be-released 2.31 FSF binutils. They change how shared libraries are linked, such that they always provide their own local definitions of the _end, _edata and _bss_start symbols. This would all be fine, except for shared libraries that export all symbols be default. (Rather than just exporting those symbols that form part of their API). Obviously there are libraries in the Fedora ecosystem that now trigger this problem, and the samba libraries are some of them. When the bug/mis-feature was reported on Monday I added a patch to the linker to revert part of the fix for PR 23162. I had hoped that this would be enough. It appears however that I may need to revert some more. The best way to fix the problem, in my opinion, would be to fix the build machinery for the samba libraries so that a version script is used when they are linked, and this script explicitly defines which symbols are exported by the library. (And also explicitly ensures that all other symbols are kept local). This would be a lot of hassle for the samba developers however, and I can appreciate that it is not something that they might want to do. So instead I need to investigate why these multiple definitions are occurring and see if I can find another way to prevent them. The other alternative is to just abandon the binutils rebase to 2.31 and go back to the good old 2.30 release. I would prefer not to do that if possible however. Cheers Nick
A couple of quick questions: > /usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/libndr-standard.so: > (*IND*+0x0): multiple definition of `_end' Is this error x86_64 specific, or does it happen with other architectures as well ? Is libndr-standard part of the samba libraries, and was it rebuilt with Alexander's rebuild ? Thanks. Nick
(In reply to Christian Heimes from comment #6) Third question: > /usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/libndr-standard.so: > (*IND*+0x0): multiple definition of `_end' Have the multiple definitions in the libsmbconf.so library (as reported in comment #2) gone away ?
libndr-standard is part of Samba, yes. I did a patch upstream that is going to hind these three symbols by default in all Samba libraries. Here is a build pipeline: https://gitlab.com/samba-team/devel/samba/pipelines/25612925, once it succeeds, I'll submit that upstream.
A new pipeline after I fixed abi tests at Samba side is this: https://gitlab.com/samba-team/devel/samba/pipelines/25615509. I submitted patch upstream. I also have rebuilt samba in Rawhide using the patch: https://koji.fedoraproject.org/koji/taskinfo?taskID=28154430 A check for libraries shows that now these symbols aren't public anymore.
I still can see linking failures with latest update in koji samba-client-libs x86_64 2:4.8.3-4.fc29 rawhide 4.9 M samba-common noarch 2:4.8.3-4.fc29 rawhide 139 k samba-common-libs x86_64 2:4.8.3-4.fc29 rawhide 101 k samba-common-tools x86_64 2:4.8.3-4.fc29 rawhide 388 k samba-dc-libs x86_64 2:4.8.3-4.fc29 rawhide 522 k samba-devel x86_64 2:4.8.3-4.fc29 rawhide 238 k samba-libs x86_64 2:4.8.3-4.fc29 rawhide 216 k samba-winbind x86_64 2:4.8.3-4.fc29 rawhide 485 k samba-winbind-modules x86_64 2:4.8.3-4.fc29 rawhide 53 k /bin/dash ./libtool --tag=CC --mode=link gcc -Wall -Wshadow -Wstrict-prototypes -Wpointer-arith -Wcast-qual -Wcast-align -Wwrite-strings -Wundef -Werror-implicit-function-declaration -Winit-self -Wmissing-include-dirs -fno-strict-aliasing -std=gnu99 -g3 -O2 -Werror -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -avoid-version -module -o libsss_krb 5.la -rpath /usr/local/lib/sssd src/providers/krb5/libsss_krb5_la-krb5_init.lo -ltalloc -ldhash -lkrb5 -lk5crypto -lcom_err -lpcre libsss_util.la libsss_crypt.la libsss_debug.la libsss _child.la libsss_krb5_common.la /usr/bin/ld: /usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/libndr-standard.so:(*IND*+0x0): multiple definition of `__bss_start' collect2: error: ld returned 1 exit status libtool: link: ( cd ".libs" && rm -f "_py2sss.la" && ln -s "../_py2sss.la" "_py2sss.la" ) make[2]: *** [Makefile:16678: sssd_pac] Error 1 sh$ rpm -qf /usr/lib64/libndr-standard.so samba-devel-4.8.3-4.fc29.x86_64 samba-client-libs-4.8.3-4.fc29.x86_64
Yes, that's with a typo _bss_start (one, not two underscores at the beginning). I'm going to ship a new build.
Samba build samba-4.8.3-4.1.fc29 fixed the problem for me. FreeIPA compiles again on F29 rawhide. Good work, Alexander!
I'd like to know how the bad samba libraries were built. If you can manage to have both _end.1 and _end@@SAMBA_UTIL_0.0.1 defined then we can probably get into the same mess with other symbols. Every trick I tried to construct a testcase failed..
When samba builds, it generates a vscript for each library it has. The code to generate it is defined in buildtools/wafsamba/samba_abi.py. It is tested in buildtools/wafsamba/tests/test_abi.py You can run that test suite with the following sequence inside samba source code: $ PYTHONPATH=third_party/waf/wafadmin python2 -mpytest buildtools/wafsamba/tests/test_abi.py ==================================================================================== test session starts ===================================================================================== platform linux2 -- Python 2.7.15, pytest-3.4.2, py-1.5.3, pluggy-0.6.0 rootdir: /home/abokovoy/src/samba, inifile: plugins: sourceorder-0.5, multihost-3.0 collected 9 items buildtools/wafsamba/tests/test_abi.py ......... [100%] ================================================================================== 9 passed in 0.03 seconds ==================================================================================
If you want to see all generated vscripts, you need to run a samba build, then do $ find bin -name '*vscript' in the source tree. $srcdir/bin is the default target.
So I found a reproducer for this gold bug, which appears to be present in at least 2.29, 2.30 and 2.31. https://sourceware.org/bugzilla/show_bug.cgi?id=23409
This bug appears to have been reported against 'rawhide' during the Fedora 29 development cycle. Changing version to '29'.
This message is a reminder that Fedora 29 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora 29 on 2019-11-26. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '29'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 29 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Fedora 29 changed to end-of-life (EOL) status on 2019-11-26. Fedora 29 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed.