Bug 1470692 - gcc: symbol _ZSt11__once_call, version GLIBCXX_3.4.11 not defined in file libstdc++.so.6 (on ppc64le)
gcc: symbol _ZSt11__once_call, version GLIBCXX_3.4.11 not defined in file lib...
Status: CLOSED RAWHIDE
Product: Fedora
Classification: Fedora
Component: gcc (Show other bugs)
rawhide
ppc64le Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Jakub Jelinek
Fedora Extras Quality Assurance
:
: 1472424 (view as bug list)
Depends On: 1456261
Blocks: PPCTracker 1471258
  Show dependency treegraph
 
Reported: 2017-07-13 09:12 EDT by Gwyn Ciesla
Modified: 2017-07-20 04:51 EDT (History)
24 users (show)

See Also:
Fixed In Version: gcc-7.1.1-6.fc27
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-07-20 04:10:43 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Build log (508.63 KB, text/plain)
2017-07-13 09:12 EDT, Gwyn Ciesla
no flags Details

  None (edit)
Description Gwyn Ciesla 2017-07-13 09:12:52 EDT
Created attachment 1297614 [details]
Build log

emacs: relocation error: /lib64/libjavascriptcoregtk-4.0.so.18: symbol _ZSt11__once_call, version GLIBCXX_3.4.11 not defined in file libstdc++.so.6 with link time reference
Comment 1 Dan Horák 2017-07-13 09:22:12 EDT
This might be related to the shuffling with gcc and glibc on ppc64le because the IFUNC support was broken. It could go away during or after the mass-rebuild once webkit gets rebuilt.
Comment 2 Dan Horák 2017-07-17 03:39:35 EDT
same problem when building evolution-data-server-3.25.4-1.fc27

/usr/lib64/libwebkit2gtk-4.0.so.37: undefined reference to `std::__once_callable@GLIBCXX_3.4.11'
/usr/lib64/libwebkit2gtk-4.0.so.37: undefined reference to `std::__once_call@GLIBCXX_3.4.11'

glibc team - I suppose we just need to rebuild the webkit package, right?
Comment 3 Milan Crha 2017-07-17 04:44:38 EDT
I had a chat on #fedora-admin and puiterwijk pointed me to bug #1456261, which prevents update of webkitgtk4 in rawhide. Note the version evolution-data-server build in rawhide [1] picked was webkitgtk4-2.16.2-2.fc27, which is neither the latest unstable version (due to bug #1456261), nor the latest stable version.

Thus, from my point of view, either the issue with debuginfo will be fixed soon, or the rawhide should get the latest stable version to not block the rest of the system from update.

An interesting thing, from my point of view, is that this missing symbol issue strikes only on ppc64le, but not anywhere else (see [1]), while the package versions I checked are the same on x86_64 and ppc64le (it was a rough check, nothing in-deep).

[1] https://koji.fedoraproject.org/koji/taskinfo?taskID=20570888
Comment 4 Florian Weimer 2017-07-17 05:03:37 EDT
libstdc++.so.6 comes from gcc, reassinging.
Comment 5 Jakub Jelinek 2017-07-17 05:21:49 EDT
The problem is:
 /* Define to 1 if the target supports thread-local storage. */
-#define _GLIBCXX_HAVE_TLS 1
+/* #undef _GLIBCXX_HAVE_TLS */
 
in between gcc-7.1.1-4.fc26 and gcc-7.1.1-5.fc27.  As there is no related change between -4 and -5 in this, this must be again some glibc headers related change in rawhide.
Comment 6 Florian Weimer 2017-07-17 10:09:20 EDT
(In reply to Jakub Jelinek from comment #5)
> The problem is:
>  /* Define to 1 if the target supports thread-local storage. */
> -#define _GLIBCXX_HAVE_TLS 1
> +/* #undef _GLIBCXX_HAVE_TLS */
>  
> in between gcc-7.1.1-4.fc26 and gcc-7.1.1-5.fc27.  As there is no related
> change between -4 and -5 in this, this must be again some glibc headers
> related change in rawhide.

I think this is derived from the run-time check here:

AC_DEFUN([GCC_CHECK_TLS], [
  AC_REQUIRE([AC_CANONICAL_HOST])
  GCC_ENABLE(tls, yes, [], [Use thread-local storage])
  AC_CACHE_CHECK([whether the target supports thread-local storage],
                 gcc_cv_have_tls, [
    AC_RUN_IFELSE([__thread int a; int b; int main() { return a = b; }],
      [dnl If the test case passed with dynamic linking, try again with
       dnl static linking, but only if static linking is supported (not
       dnl on Solaris 10).  This fails with some older Red Hat releases.
      chktls_save_LDFLAGS="$LDFLAGS"

I suspect the run-time check failed due to the bug we were trying to fix.  The build log is garbled due to the concurrency (maybe you should switch to “make -O”?), but I see some instances of

checking whether the target supports thread-local storage... no

Which, I think, should not happen.  It does not in the old logs from gcc-7.1.1-2.fc27.

I'm not sure if the configure check will pass with the fixed glibc and the current gcc in rawhide.  If not, you'll have to build with an older version, or override the check to always succeed.
Comment 7 Jakub Jelinek 2017-07-17 10:16:54 EDT
I've tried to reproduce it today but it works now:
https://koji.fedoraproject.org/koji/getfile?taskID=20573264&volume=DEFAULT&name=build.log
This was a scratch build with hacked up spec file so that it would just stop after bootstrap and uuencode libstdc++-v3/config.log
Today it was with glibc-2.25.90-25.fc27, the problematic build on ppc64le was against glibc-2.25.90-22.fc27.
So have you fixed some bug on the glibc side?
Comment 8 Florian Weimer 2017-07-17 10:35:50 EDT
(In reply to Jakub Jelinek from comment #7)
> I've tried to reproduce it today but it works now:
> https://koji.fedoraproject.org/koji/
> getfile?taskID=20573264&volume=DEFAULT&name=build.log
> This was a scratch build with hacked up spec file so that it would just stop
> after bootstrap and uuencode libstdc++-v3/config.log
> Today it was with glibc-2.25.90-25.fc27, the problematic build on ppc64le
> was against glibc-2.25.90-22.fc27.
> So have you fixed some bug on the glibc side?

We changed the TCB initialization for statically-linked binaries, and considering that there's a check for TLS with -static, we might have already fixed it.
Comment 9 Carlos O'Donell 2017-07-17 10:51:27 EDT
(In reply to Florian Weimer from comment #8)
> (In reply to Jakub Jelinek from comment #7)
> > I've tried to reproduce it today but it works now:
> > https://koji.fedoraproject.org/koji/
> > getfile?taskID=20573264&volume=DEFAULT&name=build.log
> > This was a scratch build with hacked up spec file so that it would just stop
> > after bootstrap and uuencode libstdc++-v3/config.log
> > Today it was with glibc-2.25.90-25.fc27, the problematic build on ppc64le
> > was against glibc-2.25.90-22.fc27.
> > So have you fixed some bug on the glibc side?
> 
> We changed the TCB initialization for statically-linked binaries, and
> considering that there's a check for TLS with -static, we might have already
> fixed it.

Yes on ppc64le for statically-linked binaries we now initialize the TCB and TLS *before* running the IFUNC resolvers, which was done to support the libgcc_s.so/.a IFUNC which uses a builtin to check the TCB's AT_PLATFORM/AT_HWCAP  entries. This is a ppc64le-only difference.

All statically-linked applications in ppc64le were broken until we fixed the bug. This is reason we had to do the circular builds last week to get ready for the mass rebuild.
Comment 10 Florian Weimer 2017-07-18 17:06:57 EDT
*** Bug 1472424 has been marked as a duplicate of this bug. ***
Comment 11 Richard W.M. Jones 2017-07-18 18:39:01 EDT
I did a scratch build of webkitgtk4 against current Rawhide
(changing nothing else) and it appears to have fixed emacs on
ppc64le.  My scratch build is here:

https://koji.fedoraproject.org/koji/taskinfo?taskID=20602070

I intend to bump and rebuild the real webkitgtk4 package now.
Comment 12 Milan Crha 2017-07-19 05:01:59 EDT
(In reply to Richard W.M. Jones from comment #11)
> I intend to bump and rebuild the real webkitgtk4 package now.

It'll possibly fail during package signing, as it did so for several latest builds:
https://koji.fedoraproject.org/koji/rpminfo?rpmID=10003418
which is an "info" link of x86_64 build of debuginfo subpackage at
https://koji.fedoraproject.org/koji/buildinfo?buildID=909563

That's why this depends on bug #1456261.
Comment 13 Tomas Popela 2017-07-19 06:10:13 EDT
(In reply to Milan Crha from comment #12)
> (In reply to Richard W.M. Jones from comment #11)
> > I intend to bump and rebuild the real webkitgtk4 package now.
> 
> It'll possibly fail during package signing

It's completed..
Comment 14 Jakub Jelinek 2017-07-19 06:18:54 EDT
Rebuilding webkitgtk4 or any other affected package is just wrong.
This is a gcc bug caused by glibc bug, the real fix is to rebuild gcc, not the other packages.  I'm trying to do that since Monday, but it takes long due to the s390x builders being broken because of a binutils bug.
If you rebuilt anything in order to "cure" this issue, you'll need to rebuild it again once fixed gcc is in the buildroots.
Comment 15 Richard W.M. Jones 2017-07-19 06:42:03 EDT
(In reply to Milan Crha from comment #12)
> (In reply to Richard W.M. Jones from comment #11)
> > I intend to bump and rebuild the real webkitgtk4 package now.
> 
> It'll possibly fail during package signing, as it did so for several latest
> builds:
> https://koji.fedoraproject.org/koji/rpminfo?rpmID=10003418
> which is an "info" link of x86_64 build of debuginfo subpackage at
> https://koji.fedoraproject.org/koji/buildinfo?buildID=909563
> 
> That's why this depends on bug #1456261.

Does this explain why the webkitgtk4 package which is pulled
into the buildroot is the much older version (2.16.2-2.fc27)
and not the latest version (2.17.4-3.fc27)?
Comment 16 Milan Crha 2017-07-19 07:39:43 EDT
(In reply to Richard W.M. Jones from comment #15)
> (In reply to Milan Crha from comment #12)
> > That's why this depends on bug #1456261.
> 
> Does this explain why the webkitgtk4 package which is pulled
> into the buildroot is the much older version (2.16.2-2.fc27)
> and not the latest version (2.17.4-3.fc27)?

Yes, it's the reason. I've been told so on IRC too (see comment #3).
Comment 17 Tomas Popela 2017-07-19 08:18:15 EDT
(In reply to Richard W.M. Jones from comment #15)
> (In reply to Milan Crha from comment #12)
> > (In reply to Richard W.M. Jones from comment #11)
> > > I intend to bump and rebuild the real webkitgtk4 package now.
> > 
> > It'll possibly fail during package signing, as it did so for several latest
> > builds:
> > https://koji.fedoraproject.org/koji/rpminfo?rpmID=10003418
> > which is an "info" link of x86_64 build of debuginfo subpackage at
> > https://koji.fedoraproject.org/koji/buildinfo?buildID=909563
> > 
> > That's why this depends on bug #1456261.
> 
> Does this explain why the webkitgtk4 package which is pulled
> into the buildroot is the much older version (2.16.2-2.fc27)
> and not the latest version (2.17.4-3.fc27)?

It does pull latest version otherwise the epiphany build won't succeeded - (but I manually ran koji wait-repo webkitgtk4-2.17.4-3.fc27 f27-build)
Comment 18 Milan Crha 2017-07-19 09:15:05 EDT
According to Jakub (comment #14), it's not correct to rebuild everything (what depends on webkitgtk4) now, thus I'm still on hold with evolution packages.

That the webkitgtk4-2.17.4-3.fc27 finally got into the build root is because it worked with the package/debuginfo signing. Each version before it had been broken (too large debuginfo) and 2.16.2 had been used in rawhide till now (and
on Monday [1]).

[1] https://kojipkgs.fedoraproject.org//work/tasks/910/20570910/root.log
    which is from:
    https://koji.fedoraproject.org/koji/taskinfo?taskID=20570888
Comment 19 Jakub Jelinek 2017-07-20 04:10:43 EDT
libstdc++.so.6 in libstdc++-7.1.1-6.fc27.ppc64le contains the TLS symbols again.
Packages that used to refer to those symbols and were rebuilt with the broken gcc-7.1.1-5.fc27.ppc64le should be rebuilt.
Comment 20 Richard W.M. Jones 2017-07-20 04:51:52 EDT
Thanks for rebuilding GCC.  Do we have a test for which packages
might be affected?  Would this work?

  readelf -s /usr/lib64/<library>.so.? |
      fgrep '_ZSt11__once_call@GLIBCXX_3.4.11'

I rebuilt webkitgtk4 before the GCC fix, and it's now working for me.
I don't know if that's just luck
.

Note You need to log in before you can comment on or make changes to this bug.