Bug 1208467 - libdwarf-20150310-2.fc22 is FTBFS on ppc64 and ppc64le
Summary: libdwarf-20150310-2.fc22 is FTBFS on ppc64 and ppc64le
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: binutils
Version: 22
Hardware: ppc64le
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Nick Clifton
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: F-ExcludeArch-ppc64le, PPC64LETracker
TreeView+ depends on / blocked
 
Reported: 2015-04-02 10:52 UTC by Dan Horák
Modified: 2015-08-14 07:51 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-08-14 07:51:57 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
reproducer (1.86 KB, application/x-gzip)
2015-04-15 17:50 UTC, Dan Horák
no flags Details

Description Dan Horák 2015-04-02 10:52:47 UTC
from build.log at http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=2454515

...
cd dwarfdump && make
make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
make[1]: Entering directory '/builddir/build/BUILD/dwarf-20150310/dwarfdump'
gcc  -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches  -m64 -mcpu=power7 -mtune=power8 -I. -I. -I./../libdwarf -DCONFPREFIX=/usr/local/lib  -c ./common.c
gcc  -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches  -m64 -mcpu=power7 -mtune=power8 -I. -I. -I./../libdwarf -DCONFPREFIX=/usr/local/lib    -c -o tag_common.o tag_common.c
gcc  -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches  -m64 -mcpu=power7 -mtune=power8 -I. -I. -I./../libdwarf -DCONFPREFIX=/usr/local/lib  -c ./makename.c
gcc  -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches  -m64 -mcpu=power7 -mtune=power8 -I. -I. -I./../libdwarf -DCONFPREFIX=/usr/local/lib  -DTRIVIAL_NAMING  -c ./naming.c -o trivial_naming.o 
gcc  -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches  -m64 -mcpu=power7 -mtune=power8 -I. -I. -I./../libdwarf -DCONFPREFIX=/usr/local/lib  -c ./dwgetopt.c
gcc  -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches  -m64 -mcpu=power7 -mtune=power8 -I. -I. -I./../libdwarf -DCONFPREFIX=/usr/local/lib  ./tag_tree.c tag_common.o common.o makename.o trivial_naming.o dwgetopt.o ../libdwarf/libdwarf.a  -Wl,-z,relro   -L../libdwarf -ldwarf -lelf  -o tag_tree_build 
/usr/bin/ld: dwgetopt.o: In function `dwgetoptresetfortestingonly':
/builddir/build/BUILD/dwarf-20150310/dwarfdump/./dwgetopt.c:70:(.text+0x20): unresolvable R_PPC64_TOC16_HA against `optopt@@GLIBC_2.17'
/usr/bin/ld: /builddir/build/BUILD/dwarf-20150310/dwarfdump/./dwgetopt.c:70:(.text+0x28): unresolvable R_PPC64_TOC16_LO against `optopt@@GLIBC_2.17'
/usr/bin/ld: dwgetopt.o: In function `dwgetopt':
/builddir/build/BUILD/dwarf-20150310/dwarfdump/./dwgetopt.c:107:(.text+0xf8): unresolvable R_PPC64_TOC16_HA against `optopt@@GLIBC_2.17'
/usr/bin/ld: /builddir/build/BUILD/dwarf-20150310/dwarfdump/./dwgetopt.c:107:(.text+0xfc): unresolvable R_PPC64_TOC16_HA against `optopt@@GLIBC_2.17'
/usr/bin/ld: /builddir/build/BUILD/dwarf-20150310/dwarfdump/./dwgetopt.c:107:(.text+0x100): unresolvable R_PPC64_TOC16_LO against `optopt@@GLIBC_2.17'
/usr/bin/ld: /builddir/build/BUILD/dwarf-20150310/dwarfdump/./dwgetopt.c:107:(.text+0x110): unresolvable R_PPC64_TOC16_LO against `optopt@@GLIBC_2.17'
/usr/bin/ld: /builddir/build/BUILD/dwarf-20150310/dwarfdump/./dwgetopt.c:185:(.text+0x184): unresolvable R_PPC64_TOC16_HA against `optopt@@GLIBC_2.17'
/usr/bin/ld: /builddir/build/BUILD/dwarf-20150310/dwarfdump/./dwgetopt.c:185:(.text+0x188): unresolvable R_PPC64_TOC16_LO_DS against `optopt@@GLIBC_2.17'
/usr/bin/ld: /builddir/build/BUILD/dwarf-20150310/dwarfdump/./dwgetopt.c:124:(.text+0x1cc): unresolvable R_PPC64_TOC16_HA against `optopt@@GLIBC_2.17'
/usr/bin/ld: /builddir/build/BUILD/dwarf-20150310/dwarfdump/./dwgetopt.c:124:(.text+0x1d4): unresolvable R_PPC64_TOC16_LO against `optopt@@GLIBC_2.17'
/usr/bin/ld: dwgetopt.o: In function `fprintf':
/usr/include/bits/stdio2.h:97:(.text+0x2b4): unresolvable R_PPC64_TOC16_HA against `optopt@@GLIBC_2.17'
/usr/bin/ld: /usr/include/bits/stdio2.h:97:(.text+0x2c4): unresolvable R_PPC64_TOC16_LO_DS against `optopt@@GLIBC_2.17'
/usr/bin/ld: /usr/include/bits/stdio2.h:97:(.text+0x404): unresolvable R_PPC64_TOC16_HA against `optopt@@GLIBC_2.17'
/usr/bin/ld: /usr/include/bits/stdio2.h:97:(.text+0x414): unresolvable R_PPC64_TOC16_LO_DS against `optopt@@GLIBC_2.17'
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
Makefile:121: recipe for target 'tag_tree_build' failed
make[1]: Leaving directory '/builddir/build/BUILD/dwarf-20150310/dwarfdump'
Makefile:70: recipe for target 'basic' failed
RPM build errors:
make[1]: *** [tag_tree_build] Error 1


I think it might be related to relro and missing -fPIC.

Version-Release number of selected component (if applicable):
libdwarf-20150310-2.fc22

Comment 1 Tom Hughes 2015-04-02 10:59:38 UTC
That looks like what x86 was doing in -1 which is why hardended builds were disabled in -2 so are you sure that's from -2?

That is a program it is trying to link, not a library, so -fPIC should not be necessary on any of the code.

Comment 2 Dan Horák 2015-04-02 19:16:03 UTC
(In reply to Tom Hughes from comment #1)
> That looks like what x86 was doing in -1 which is why hardended builds were
> disabled in -2 so are you sure that's from -2?

yes, it is -2 as can be seen from the task URL
 
> That is a program it is trying to link, not a library, so -fPIC should not
> be necessary on any of the code.

the link command for the tag_tree_build executable uses -Wl,-z,relro, but I'm not a toolchain expert

Comment 3 Jaromír Cápík 2015-04-15 14:55:31 UTC
Happens on big endian too.

Comment 4 Jaromír Cápík 2015-04-15 16:17:54 UTC
Tried to return the explicit CFLAGS and hardened build and their combinations and without effect. Also tried to rebuild glibc with the latest gcc and nothing seems to help.

Comment 5 Jaromír Cápík 2015-04-15 16:18:54 UTC
Found similar upstream bugs for gcc : 

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=32219
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65074

Comment 6 Jaromír Cápík 2015-04-15 16:57:26 UTC
Strange .... the optopt variable defined in dwgetopt.c/h somehow conflicts with the optopt@@GLIBC_2.17 symbol. I renamed optopt to ooptopt in the libdwarf and the build passed.

Comment 7 Jaromír Cápík 2015-04-15 16:59:40 UTC
The question is, why that happens on the PPC only. Looks like a toolchain issue.

Comment 8 Jaromír Cápík 2015-04-15 17:17:52 UTC
NOTE: Even initializing the optopt variable in the libdwarf with zero value helps.

Comment 9 Jaromír Cápík 2015-04-15 17:42:56 UTC
Re-building libdwarf with the above workaround as it blocks important packages in f22. The patch can be removed once the linker is fixed.

Comment 10 Tom Hughes 2015-04-15 17:47:42 UTC
How about you let the maintainer look at the patch before using your superpowers? You could at least make it conditional to the affected platforms!

Comment 11 Fedora Update System 2015-04-15 17:48:29 UTC
libdwarf-20150310-3.fc22 has been submitted as an update for Fedora 22.
https://admin.fedoraproject.org/updates/libdwarf-20150310-3.fc22

Comment 12 Dan Horák 2015-04-15 17:50:04 UTC
Created attachment 1014915 [details]
reproducer

build with
gcc -O2 -o test dwarfdump.c dwgetopt.c

and should get
[sharkcz@tyan-openpower-01 rhbz1208467]$ gcc -O2 -o test dwarfdump.c dwgetopt.c 
/usr/bin/ld: /tmp/cczmjUFk.o: In function `dwgetoptresetfortestingonly':
dwgetopt.c:(.text+0x1a): unresolvable R_PPC64_TOC16_HA against `optopt@@GLIBC_2.3'
/usr/bin/ld: dwgetopt.c:(.text+0x22): unresolvable R_PPC64_TOC16_LO against `optopt@@GLIBC_2.3'
/usr/bin/ld: /tmp/cczmjUFk.o: In function `dwgetopt':
dwgetopt.c:(.text+0xf2): unresolvable R_PPC64_TOC16_HA against `optopt@@GLIBC_2.3'
/usr/bin/ld: dwgetopt.c:(.text+0xf6): unresolvable R_PPC64_TOC16_HA against `optopt@@GLIBC_2.3'
/usr/bin/ld: dwgetopt.c:(.text+0xfa): unresolvable R_PPC64_TOC16_LO against `optopt@@GLIBC_2.3'
/usr/bin/ld: dwgetopt.c:(.text+0x10a): unresolvable R_PPC64_TOC16_LO against `optopt@@GLIBC_2.3'
/usr/bin/ld: dwgetopt.c:(.text+0x17e): unresolvable R_PPC64_TOC16_HA against `optopt@@GLIBC_2.3'
/usr/bin/ld: dwgetopt.c:(.text+0x182): unresolvable R_PPC64_TOC16_LO_DS against `optopt@@GLIBC_2.3'
/usr/bin/ld: dwgetopt.c:(.text+0x1be): unresolvable R_PPC64_TOC16_HA against `optopt@@GLIBC_2.3'
/usr/bin/ld: dwgetopt.c:(.text+0x1c6): unresolvable R_PPC64_TOC16_LO against `optopt@@GLIBC_2.3'
/usr/bin/ld: dwgetopt.c:(.text+0x2a6): unresolvable R_PPC64_TOC16_HA against `optopt@@GLIBC_2.3'
/usr/bin/ld: dwgetopt.c:(.text+0x2b2): unresolvable R_PPC64_TOC16_LO_DS against `optopt@@GLIBC_2.3'
/usr/bin/ld: dwgetopt.c:(.text+0x3f6): unresolvable R_PPC64_TOC16_HA against `optopt@@GLIBC_2.3'
/usr/bin/ld: dwgetopt.c:(.text+0x402): unresolvable R_PPC64_TOC16_LO_DS against `optopt@@GLIBC_2.3'
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status


package versions:
gcc-5.0.0-0.21.fc22.ppc64
glibc-2.21-5.fc22.ppc64p7
binutils-2.25-5.fc22.ppc64p7

Comment 13 Jaromír Cápík 2015-04-15 17:59:39 UTC
(In reply to Tom Hughes from comment #10)
> How about you let the maintainer look at the patch before using your
> superpowers? You could at least make it conditional to the affected
> platforms!

Sorry Tom. There was no answer from you for 2 weeks. I was looking for you on the IRC channel, but you're offline and this blocks important stuff. The workaround is a minor and harmless modification. Feel free to revert it if you don't like the change. However, the third release can already be built on the secondary and that's sufficient for us.

Comment 14 Tom Hughes 2015-04-15 19:28:01 UTC
Two weeks?!?!

You identified the problem at 12:57:26 today, that initialising optopt fixes it at 13:17:52 and at 13:42:56 said you had done the rebuild. That's less than an hour from start to finish, and at no point was the actual patch you were proposing attached.

I don't have access to a machine where I could attempt to debug this myself, nor was I aware it was blocking anything, so I was waiting on the mass rebuild before giving any further thought to this on the basis that it was likely a result of inconsistent compiler flags.

Obviously as it turns out that isn't the case for this one, and I would have been happy to review your proposed fix had you posted it.

For the record I don't see any problem about patching the dwarfdump tool, but patching library code to initialise C library global variables seems like a very scary thing to do.

Comment 15 Tom Hughes 2015-04-15 19:35:20 UTC
Ah I see the dwgetopt.c in the library is only included when building the gennames tool, not in the library itself, so that is likely fine then.

Comment 16 Jakub Jelinek 2015-04-15 21:39:05 UTC
Must be a linker bug.
Short testcase (a.c):
int optopt;

int
main ()
{
  optopt = 4;
  return 0;
}

gcc -O2 -o a a.c -fno-common; echo $?; gcc -O2 -o a a.c -Doptopt=optoptx; echo $?; gcc -O2 -o a a.c; echo $?
0
0
/usr/bin/ld: /tmp/ccZQLKGW.o: In function `main':
a.c:(.text.startup+0x6): unresolvable R_PPC64_TOC16_HA against `optopt@@GLIBC_2.3'
/usr/bin/ld: a.c:(.text.startup+0xe): unresolvable R_PPC64_TOC16_LO against `optopt@@GLIBC_2.3'
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
1

I don't see anything wrong on the generated assembly, and as the above command line shows, when optopt is not a common symbol, or when the common symbol doesn't have the same name as some symbol in some shared library linked in (-lc in this case), it works just fine.
For gcc -O2 -o a a.c e.g. on x86_64 the linker creates a copy relocation for the optopt@@GLIBC_something symbol.
Or is it some PowerPC speciality that common symbols, when there is a var definition with the same name in dependent shared libraries, aren't going to be defined necessarily in the binary but somewhere else (i.e. that common symbols defined in the executable don't necessarily bind locally)?

Comment 17 Jakub Jelinek 2015-04-15 22:31:05 UTC
Looking at it some more, I really don't know, perhaps it is a linker bug, perhaps a gcc bug.  Filed http://gcc.gnu.org/PR65780

Comment 18 Martin Sebor 2015-04-15 22:48:13 UTC
I can reproduce it with gcc 5.0 and binutils 2.23.52.0.1-30 on RHEL 7.1 but not with gcc 4.8.3-9.  Here's a standalone test case independent of GLIBC.  As the objdump output shows, there's a difference in foo's relocation type between the two compilers.  It looks to me like gcc 5 is optimizing the common object by putting directly into the TOC, while the earlier gcc stores its address in the TOC.  My guess is that since it's common object it shouldn't go directly in the TOC because the linker needs to be able to merge all instances of it in the program.

$ (set -x; cat a.c && for cc in /usr/bin/gcc "./gcc/xgcc -Bgcc"; do $cc -DDSO -shared -o liba.so a.c && $cc -c a.c && objdump -r liba.so a.o && $cc -L. -la a.o && LD_LIBRARY_PATH=. ./a.out ; echo $?; done)
+ cat a.c
int foo;
extern int bar (void);
#if DSO
int foo = 123;
int bar (void) { return foo; }
#else
int main (void) {
    foo = 0;
    return bar ();
}
#endif
+ for cc in /usr/bin/gcc '"./gcc/xgcc -Bgcc"'
+ /usr/bin/gcc -DDSO -shared -o liba.so a.c
+ /usr/bin/gcc -c a.c
+ objdump -r liba.so a.o

liba.so:     file format elf64-powerpcle


a.o:     file format elf64-powerpcle

RELOCATION RECORDS FOR [.text]:
OFFSET           TYPE              VALUE 
0000000000000000 R_PPC64_REL16_HA  .TOC.
0000000000000004 R_PPC64_REL16_LO  .TOC.+0x0000000000000004
000000000000001c R_PPC64_TOC16_HA  .toc
0000000000000020 R_PPC64_TOC16_LO_DS  .toc
000000000000002c R_PPC64_REL24     bar


RELOCATION RECORDS FOR [.toc]:
OFFSET           TYPE              VALUE 
0000000000000000 R_PPC64_ADDR64    foo


+ /usr/bin/gcc -L. -la a.o
+ LD_LIBRARY_PATH=.
+ ./a.out
+ echo 0
0
+ for cc in /usr/bin/gcc '"./gcc/xgcc -Bgcc"'
+ ./gcc/xgcc -Bgcc -DDSO -shared -o liba.so a.c
+ ./gcc/xgcc -Bgcc -c a.c
+ objdump -r liba.so a.o

liba.so:     file format elf64-powerpcle


a.o:     file format elf64-powerpcle

RELOCATION RECORDS FOR [.text]:
OFFSET           TYPE              VALUE 
0000000000000000 R_PPC64_REL16_HA  .TOC.
0000000000000004 R_PPC64_REL16_LO  .TOC.+0x0000000000000004
000000000000001c R_PPC64_TOC16_HA  foo
0000000000000020 R_PPC64_TOC16_LO  foo
000000000000002c R_PPC64_REL24     bar


+ ./gcc/xgcc -Bgcc -L. -la a.o
/usr/bin/ld: a.o: In function `main':
a.c:(.text+0x1c): unresolvable R_PPC64_TOC16_HA against `foo'
/usr/bin/ld: a.c:(.text+0x20): unresolvable R_PPC64_TOC16_LO against `foo'
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
+ echo 1
1

Comment 19 Jaromír Cápík 2015-04-16 11:49:21 UTC
(In reply to Tom Hughes from comment #14)
> Two weeks?!?!

Two weeks since your first answer and I thought you have no time for the analysis as there were no messages since that.


> You identified the problem at 12:57:26 today, that initialising optopt fixes
> it at 13:17:52 and at 13:42:56 said you had done the rebuild. That's less
> than an hour from start to finish, and at no point was the actual patch you
> were proposing attached.

In case of complex changes I always wait for the maintainer's approval prior committing. In this case the fix is obviously harmless one-liner and that's why I didn't wait. The worst thing that can happen is that you revert the commit and delete the update from Bodhi, right? No need to be upset, I'm on the same boat.


> I don't have access to a machine where I could attempt to debug this myself,

The easiest thing is to ask. We can give you the access in few minutes. No problem with that.


> nor was I aware it was blocking anything,

Are you aware of the F22 release schedule? libdwarf blocks qemu, libguestfs, ... Keeping important deps FTBFS is quite unwanted at this point.


> so I was waiting on the mass
> rebuild before giving any further thought to this on the basis that it was
> likely a result of inconsistent compiler flags.

Ok, understand. Maybe you could mention your plans in the comments next time, so that we could be aware of your activities and avoid interfering with whatever you wanna do.


> Obviously as it turns out that isn't the case for this one, and I would have
> been happy to review your proposed fix had you posted it.

You still can do. Submitting the build in testing is harmless and the build can always be reverted. The bodhi submit allowed the build scripts to continue with building the missing subtree on the PPC koji and you can revert or change it accorrding to your needs now.

Comment 20 Tom Hughes 2015-04-16 11:56:21 UTC
Sorry, I don't think I had registered this was F22 rather than Rawhide. I had it down in my mind as being related to the hardening changes you see.

I also hadn't realised it was only a scratch build you had done, probably mostly because you had pushed the change to git and that was the email I saw, plus again I didn't scroll down the email far enough to see you had pushed it to f22 as well as rawhide so I still had it in my mind as a rawhide issue.

Comment 21 Tom Hughes 2015-04-16 11:58:25 UTC
Ah, it wasn't just a scratch build, and it has been submitted as an update.

Comment 22 Tom Hughes 2015-04-16 12:01:29 UTC
Anyway I think my key point is that my understanding of the PP policy is based on:

  "Provenpackagers should try to communicate with owners of a package in bugzilla, irc or email prior to making changes"

Which I take as meaning that the specific change you want to make should be proposed to the owner, but maybe that's my misunderstanding.

Comment 23 Tom Hughes 2015-04-20 07:33:54 UTC
Looks like gcc-5.0.1-0.2 fixes this, it has the patch from gcc and certainly x86 now works with hardening enabled, which looks like it was the same bug.

Comment 24 Fedora Update System 2015-04-24 22:49:11 UTC
libdwarf-20150310-3.fc22 has been pushed to the Fedora 22 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 25 Rafael Fonseca 2015-08-13 13:18:54 UTC
So can we close this bug?


Note You need to log in before you can comment on or make changes to this bug.