Fedora Account System
Red Hat Associate
Red Hat Customer
Hi Everyone, My apologies for this issue report. We caught a bug report for Fedora 28, ppc64-le and GCC 8.0.1-0.14.fc28. Also see https://github.com/weidai11/cryptopp/issues/588. We attempted to duplicate on GCC112, which is ppc64-le. Also see https://gcc.gnu.org/wiki/CompileFarm. GCC112 offers GCC 4.8.5, GCC 7.3.1 and now GCC 8-r257824. We were not able to duplicate the reported issues under the compilers. We think it might be prudent to update GCC 8.0.1-0.14.fc28 before going further with our issue. Please consider updating GCC 8.
8.0.1-0.14.fc28 is r257795 based, I'd think you just used different configure options or different gcc options because it is very unlikely exactly your issue has been fixed in the last 28 hours. Were you e.g. testing with the /usr/lib/rpm/redhat/redhat-hardened-{cc1,ld} options (-fpie for objects not built with -f{pic,PIC,no-pic,pie,PIE} and -pie for linking binaries)?
(In reply to Jakub Jelinek from comment #1) > 8.0.1-0.14.fc28 is r257795 based, I'd think you just used different > configure options or different gcc options because it is very unlikely > exactly your issue has been fixed in the last 28 hours. Were you e.g. > testing with the > /usr/lib/rpm/redhat/redhat-hardened-{cc1,ld} options (-fpie for objects not > built with -f{pic,PIC,no-pic,pie,PIE} and -pie for linking binaries)? Thanks Jakub. I was kind of afraid of that. Regarding /usr/lib/rpm/redhat/redhat-hardened-{cc1,ld} options, NO, no testing. The compile farm machine does not have them. (I did try to use the exact command before removing the options). We do build with -fPIC, however. The compiler uses -fPIC to drive the linker so the binary should be relocatable, too. This may as well be clsed out. It looks like a dead end. I'm going to try to get access to that testing machine. Let me ping Nicolas.
You could try rebuilding everything with -O0 to see if it passes testsuite then (e.g. in --scratch build), if yes and if you have access somewhere to a machine where you can use mock on ppc64le, you could bisect in between *.o files built with -O0 vs. -O2 to narrow it to a single translation unit. You can use -v to see full cc1/cc1plus command lines and try to replicate that on compile farm.
(In reply to Jakub Jelinek from comment #3) > You could try rebuilding everything with -O0 to see if it passes testsuite > then (e.g. in --scratch build), if yes and if you have access somewhere to a > machine where you can use mock on ppc64le, you could bisect in between *.o > files built with -O0 vs. -O2 to narrow it to a single translation unit. You > can use -v to see full cc1/cc1plus command lines and try to replicate that > on compile farm. Thanks again Jakub. We've isolated problem, but it is not reduced to the point it would make a useful bug report. It seems CXXFLAGS with '-O3 -fstack-protector-strong' is OK, but '-O2 -fstack-protector-strong' is bad. I'm going to ping the gcc-devs on the mailing list to see where to go next. Thanks again for the help.
Do you know which *.o file is affected, do you know it either from debugging or bisection? For the latter, the usual way is to build the tree with one set of flags that work, confirm it works, save a copy of the build tree, build it with the other set of flags that don't work, confirm it fails, save a copy of the build tree and then always for some set of *.o files copy them from the good and/or bad build tree + touch them and make to link the stuff, test. Start with half of the *.o files "good" and half "bad" and try to narrow it if possible to a case where everything "good" but one "bad" fails and everything "bad" but one "good" (the same file) works. One can additionally use __attribute__((optimize (2))) or __attribute__(((optimize (3))) on various functions/methods within the single (hopefully) problematic file to narrow it further.
(In reply to Jakub Jelinek from comment #5) > Do you know which *.o file is affected, do you know it either from debugging > or bisection? For the latter, the usual way is to build the tree with one > set of flags that work, confirm it works, save a copy of the build tree, > build it with the other set of flags that don't work, confirm it fails, save > a copy of the build tree and then always for some set of *.o files copy them > from the good and/or bad build tree + touch them and make to link the stuff, > test. Start with half of the *.o files "good" and half "bad" and try to > narrow it if possible to a case where everything "good" but one "bad" fails > and everything "bad" but one "good" (the same file) works. > One can additionally use __attribute__((optimize (2))) or > __attribute__(((optimize (3))) on various functions/methods within the > single (hopefully) problematic file to narrow it further. Yes, we know the object file that is complicit. It is the AES source files that uses Power8 extensions/builtins. I can avoid the failure with the steps shown below. I do not have it isolated to a function yet. ********** $ make distclean # -O2 $ CXX=/opt/cfarm/gcc8-r257824/bin/g++ CXXFLAGS="-DNDEBUG -g2 -O2 -fstack-protector-strong" make -j 20 $ rm rijndael-simd.o # -O3 $ CXX=/opt/cfarm/gcc8-r257824/bin/g++ CXXFLAGS="-DNDEBUG -g2 -O3 -fstack-protector-strong" make rijndael-simd.o # Link $ CXX=/opt/cfarm/gcc8-r257824/bin/g++ CXXFLAGS="-DNDEBUG -g2 -O2 -fstack-protector-strong" make -j 20 $ LD_LIBRARY_PATH=/opt/cfarm/gcc8-r257824/lib64/ ./cryptest.exe tv aes Using seed: 1519116775 Testing SymmetricCipher algorithm AES/ECB. .... Testing SymmetricCipher algorithm AES/CBC. ........ Testing SymmetricCipher algorithm AES/CFB. ....... Testing SymmetricCipher algorithm AES/OFB. .... Testing SymmetricCipher algorithm AES/CTR. .............
Thanks again for the help Januk. This is far from a simple reporducer. I have my AES proof of concept code at https://github.com/noloader/AES-Intrinsics/blob/master/aes-p8.c . It does not witness the problem. The real source file is awful. It is located at https://github.com/weidai11/cryptopp/blob/master/rijndael-simd.cpp#L536 . Nearly all the Power8 AES functions are inlined into AdvancedProcessBlocks128_6x1_ALTIVEC . The source code for AdvancedProcessBlocks128_6x1_ALTIVEC is located at https://github.com/weidai11/cryptopp/blob/master/adv-simd.h#L1257 .
This bug appears to have been reported against 'rawhide' during the Fedora 28 development cycle. Changing version to '28'.
Jakub, I feel like an asshole... We had an ODR violation. Take a look at adv-simd.h. It is intended to be included in exactly one translation unit, like rijndael-simd.cpp or sha-simd.cpp. It does some unsavory things... It had several things wrong with it that are now fixed: 1. Header guard is wrong - #if defined(XXX) present - #define XXX missing 2. Constants in unnamed namespace 3. Missing inline declaration on 2 functions I fixed adv-simd.h incrementally. The last one - missing inline declaration - was the one that fixed the problem under GCC 8. Sorry to waste your time with this.
(In reply to Jeffrey Walton from comment #9) > I fixed adv-simd.h incrementally. The last one - missing inline declaration > - was the one that fixed the problem under GCC 8. That seems odd, because inline functions and function templates have the same exemption from the One-Definition Rule. So making them inline shouldn't "fix" anything.