Bug 1659295 - kernel: Empty GNU_EH_FRAME segment in vDSO breaks libgcc unwinder
Summary: kernel: Empty GNU_EH_FRAME segment in vDSO breaks libgcc unwinder
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 28
Hardware: i686
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Keywords:
: 1651010 (view as bug list)
Depends On:
Blocks: x86Tracker
TreeView+ depends on / blocked
 
Reported: 2018-12-14 02:37 UTC by Carlos O'Donell
Modified: 2019-01-29 17:02 UTC (History)
29 users (show)

(edit)
Clone Of:
(edit)
Last Closed: 2019-01-29 17:02:51 UTC


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Linux Kernel 201741 None None None 2019-01-29 16:40 UTC

Description Carlos O'Donell 2018-12-14 02:37:48 UTC
We have a series of new failures on i686-pc-linux-gnu

Building glibc-2.27-37.fc28 for i686.

BUILDSTDERR: FAIL: debug/tst-backtrace5
BUILDSTDERR: FAIL: debug/tst-backtrace6
BUILDSTDERR: FAIL: nptl/tst-cancel24
BUILDSTDERR: FAIL: nptl/tst-cancel24-static
BUILDSTDERR: FAIL: nptl/tst-cancelx16
BUILDSTDERR: FAIL: nptl/tst-cancelx18
BUILDSTDERR: FAIL: nptl/tst-cancelx20
BUILDSTDERR: FAIL: nptl/tst-cancelx21
BUILDSTDERR: FAIL: nptl/tst-cancelx4
BUILDSTDERR: FAIL: nptl/tst-cancelx5
BUILDSTDERR: FAIL: nptl/tst-cleanupx4
BUILDSTDERR: FAIL: nptl/tst-oncex3
BUILDSTDERR: FAIL: nptl/tst-oncex4

BUILDSTDERR: =====FAIL: debug/tst-backtrace5.out=====
BUILDSTDERR: Obtained backtrace with 3 functions
BUILDSTDERR: Failure on line 53
BUILDSTDERR: =====FAIL: debug/tst-backtrace5.test-result=====
BUILDSTDERR: FAIL: debug/tst-backtrace5
BUILDSTDERR: original exit status 1
BUILDSTDERR: =====FAIL: debug/tst-backtrace6.out=====
BUILDSTDERR: Obtained backtrace with 3 functions
BUILDSTDERR: Failure on line 53
BUILDSTDERR: =====FAIL: debug/tst-backtrace6.test-result=====
BUILDSTDERR: FAIL: debug/tst-backtrace6
BUILDSTDERR: original exit status 1

BUILDSTDERR: =====FAIL: nptl/tst-cancel24.out=====
BUILDSTDERR: exception not caught
BUILDSTDERR: =====FAIL: nptl/tst-cancel24.test-result=====
BUILDSTDERR: FAIL: nptl/tst-cancel24
BUILDSTDERR: original exit status 1
BUILDSTDERR: =====FAIL: nptl/tst-cancel24-static.out=====
BUILDSTDERR: exception not caught
BUILDSTDERR: =====FAIL: nptl/tst-cancel24-static.test-result=====
BUILDSTDERR: FAIL: nptl/tst-cancel24-static
BUILDSTDERR: original exit status 1
BUILDSTDERR: =====FAIL: nptl/tst-cancelx16.out=====
BUILDSTDERR: cleanup handler not called
BUILDSTDERR: =====FAIL: nptl/tst-cancelx16.test-result=====
BUILDSTDERR: FAIL: nptl/tst-cancelx16
BUILDSTDERR: original exit status 1
BUILDSTDERR: =====FAIL: nptl/tst-cancelx18.out=====
BUILDSTDERR: going to cancel in-time
BUILDSTDERR: cleanup handler not called
BUILDSTDERR: =====FAIL: nptl/tst-cancelx18.test-result=====
BUILDSTDERR: FAIL: nptl/tst-cancelx18
BUILDSTDERR: original exit status 1
BUILDSTDERR: =====FAIL: nptl/tst-cancelx20.out=====
BUILDSTDERR: sa_flags = 0 test
BUILDSTDERR: called cleanups 0
BUILDSTDERR: =====FAIL: nptl/tst-cancelx20.test-result=====
BUILDSTDERR: FAIL: nptl/tst-cancelx20
BUILDSTDERR: original exit status 1
BUILDSTDERR: =====FAIL: nptl/tst-cancelx21.out=====
BUILDSTDERR: sa_flags = 0 test
BUILDSTDERR: called cleanups 0
BUILDSTDERR: =====FAIL: nptl/tst-cancelx21.test-result=====
BUILDSTDERR: FAIL: nptl/tst-cancelx21
BUILDSTDERR: original exit status 1
BUILDSTDERR: =====FAIL: nptl/tst-cancelx4.out=====
BUILDSTDERR: cleanup handler not called for 'read'
BUILDSTDERR: cleanup handler not called for 'readv'
BUILDSTDERR: cleanup handler not called for 'select'
BUILDSTDERR: cleanup handler not called for 'pselect'
BUILDSTDERR: cleanup handler not called for 'poll'
BUILDSTDERR: cleanup handler not called for 'ppoll'
BUILDSTDERR: cleanup handler not called for 'write'
BUILDSTDERR: cleanup handler not called for 'writev'
BUILDSTDERR: cleanup handler not called for 'sleep'
BUILDSTDERR: cleanup handler not called for 'usleep'
BUILDSTDERR: cleanup handler not called for 'nanosleep'
BUILDSTDERR: cleanup handler not called for 'wait'
BUILDSTDERR: cleanup handler not called for 'waitid'
BUILDSTDERR: cleanup handler not called for 'waitpid'
BUILDSTDERR: cleanup handler not called for 'sigpause'
BUILDSTDERR: cleanup handler not called for 'sigsuspend'
BUILDSTDERR: cleanup handler not called for 'sigwait'
BUILDSTDERR: cleanup handler not called for 'sigwaitinfo'
BUILDSTDERR: cleanup handler not called for 'sigtimedwait'
BUILDSTDERR: cleanup handler not called for 'pause'
BUILDSTDERR: cleanup handler not called for 'accept'
BUILDSTDERR: cleanup handler not called for 'send'
BUILDSTDERR: cleanup handler not called for 'recv'
BUILDSTDERR: cleanup handler not called for 'recvfrom'
BUILDSTDERR: cleanup handler not called for 'recvmsg'
BUILDSTDERR: cleanup handler not called for 'msgrcv'
BUILDSTDERR: early cancel test of 'read' successful
BUILDSTDERR: early cancel test of 'readv' successful
BUILDSTDERR: early cancel test of 'select' successful
BUILDSTDERR: early cancel test of 'pselect' successful
BUILDSTDERR: early cancel test of 'poll' successful
BUILDSTDERR: early cancel test of 'ppoll' successful
BUILDSTDERR: early cancel test of 'write' successful
BUILDSTDERR: early cancel test of 'writev' successful
BUILDSTDERR: early cancel test of 'sleep' successful
BUILDSTDERR: early cancel test of 'usleep' successful
BUILDSTDERR: early cancel test of 'nanosleep' successful
BUILDSTDERR: early cancel test of 'wait' successful
BUILDSTDERR: early cancel test of 'waitid' successful
BUILDSTDERR: early cancel test of 'waitpid' successful
BUILDSTDERR: early cancel test of 'sigpause' successful
BUILDSTDERR: early cancel test of 'sigsuspend' successful
BUILDSTDERR: early cancel test of 'sigwait' successful
BUILDSTDERR: early cancel test of 'sigwaitinfo' successful
BUILDSTDERR: early cancel test of 'sigtimedwait' successful
BUILDSTDERR: early cancel test of 'pause' successful
BUILDSTDERR: early cancel test of 'accept' successful
BUILDSTDERR: early cancel test of 'send' successful
BUILDSTDERR: early cancel test of 'recv' successful
BUILDSTDERR: early cancel test of 'recvfrom' successful
BUILDSTDERR: early cancel test of 'recvmsg' successful
BUILDSTDERR: early cancel test of 'preadv' successful
BUILDSTDERR: early cancel test of 'preadv2' successful
BUILDSTDERR: early cancel test of 'pwritev' successful
BUILDSTDERR: early cancel test of 'pwritev2' successful
BUILDSTDERR: early cancel test of 'open' successful
BUILDSTDERR: early cancel test of 'close' successful
BUILDSTDERR: early cancel test of 'pread' successful
BUILDSTDERR: early cancel test of 'pwrite' successful
BUILDSTDERR: early cancel test of 'fsync' successful
BUILDSTDERR: early cancel test of 'fdatasync' successful
BUILDSTDERR: early cancel test of 'msync' successful
BUILDSTDERR: early cancel test of 'sendto' successful
BUILDSTDERR: early cancel test of 'sendmsg' successful
BUILDSTDERR: early cancel test of 'creat' successful
BUILDSTDERR: early cancel test of 'connect' successful
BUILDSTDERR: early cancel test of 'tcdrain' successful
BUILDSTDERR: early cancel test of 'msgrcv' successful
BUILDSTDERR: early cancel test of 'msgsnd' successful
BUILDSTDERR: =====FAIL: nptl/tst-cancelx4.test-result=====
BUILDSTDERR: FAIL: nptl/tst-cancelx4
BUILDSTDERR: original exit status 1
BUILDSTDERR: =====FAIL: nptl/tst-cancelx5.out=====
BUILDSTDERR: cleanup handler not called for 'read'
BUILDSTDERR: cleanup handler not called for 'readv'
BUILDSTDERR: cleanup handler not called for 'select'
BUILDSTDERR: cleanup handler not called for 'pselect'
BUILDSTDERR: cleanup handler not called for 'poll'
BUILDSTDERR: cleanup handler not called for 'ppoll'
BUILDSTDERR: cleanup handler not called for 'write'
BUILDSTDERR: cleanup handler not called for 'writev'
BUILDSTDERR: cleanup handler not called for 'sleep'
BUILDSTDERR: cleanup handler not called for 'usleep'
BUILDSTDERR: cleanup handler not called for 'nanosleep'
BUILDSTDERR: cleanup handler not called for 'wait'
BUILDSTDERR: cleanup handler not called for 'waitid'
BUILDSTDERR: cleanup handler not called for 'waitpid'
BUILDSTDERR: cleanup handler not called for 'sigpause'
BUILDSTDERR: cleanup handler not called for 'sigsuspend'
BUILDSTDERR: cleanup handler not called for 'sigwait'
BUILDSTDERR: cleanup handler not called for 'sigwaitinfo'
BUILDSTDERR: cleanup handler not called for 'sigtimedwait'
BUILDSTDERR: cleanup handler not called for 'pause'
BUILDSTDERR: cleanup handler not called for 'accept'
BUILDSTDERR: cleanup handler not called for 'send'
BUILDSTDERR: cleanup handler not called for 'recv'
BUILDSTDERR: cleanup handler not called for 'recvfrom'
BUILDSTDERR: cleanup handler not called for 'recvmsg'
BUILDSTDERR: cleanup handler not called for 'msgrcv'
BUILDSTDERR: early cancel test of 'read' successful
BUILDSTDERR: early cancel test of 'readv' successful
BUILDSTDERR: early cancel test of 'select' successful
BUILDSTDERR: early cancel test of 'pselect' successful
BUILDSTDERR: early cancel test of 'poll' successful
BUILDSTDERR: early cancel test of 'ppoll' successful
BUILDSTDERR: early cancel test of 'write' successful
BUILDSTDERR: early cancel test of 'writev' successful
BUILDSTDERR: early cancel test of 'sleep' successful
BUILDSTDERR: early cancel test of 'usleep' successful
BUILDSTDERR: early cancel test of 'nanosleep' successful
BUILDSTDERR: early cancel test of 'wait' successful
BUILDSTDERR: early cancel test of 'waitid' successful
BUILDSTDERR: early cancel test of 'waitpid' successful
BUILDSTDERR: early cancel test of 'sigpause' successful
BUILDSTDERR: early cancel test of 'sigsuspend' successful
BUILDSTDERR: early cancel test of 'sigwait' successful
BUILDSTDERR: early cancel test of 'sigwaitinfo' successful
BUILDSTDERR: early cancel test of 'sigtimedwait' successful
BUILDSTDERR: early cancel test of 'pause' successful
BUILDSTDERR: early cancel test of 'accept' successful
BUILDSTDERR: early cancel test of 'send' successful
BUILDSTDERR: early cancel test of 'recv' successful
BUILDSTDERR: early cancel test of 'recvfrom' successful
BUILDSTDERR: early cancel test of 'recvmsg' successful
BUILDSTDERR: early cancel test of 'preadv' successful
BUILDSTDERR: early cancel test of 'preadv2' successful
BUILDSTDERR: early cancel test of 'pwritev' successful
BUILDSTDERR: early cancel test of 'pwritev2' successful
BUILDSTDERR: early cancel test of 'open' successful
BUILDSTDERR: early cancel test of 'close' successful
BUILDSTDERR: early cancel test of 'pread' successful
BUILDSTDERR: early cancel test of 'pwrite' successful
BUILDSTDERR: early cancel test of 'fsync' successful
BUILDSTDERR: early cancel test of 'fdatasync' successful
BUILDSTDERR: early cancel test of 'msync' successful
BUILDSTDERR: early cancel test of 'sendto' successful
BUILDSTDERR: early cancel test of 'sendmsg' successful
BUILDSTDERR: early cancel test of 'creat' successful
BUILDSTDERR: early cancel test of 'connect' successful
BUILDSTDERR: early cancel test of 'tcdrain' successful
BUILDSTDERR: early cancel test of 'msgrcv' successful
BUILDSTDERR: early cancel test of 'msgsnd' successful
BUILDSTDERR: =====FAIL: nptl/tst-cancelx5.test-result=====
BUILDSTDERR: FAIL: nptl/tst-cancelx5
BUILDSTDERR: original exit status 1
BUILDSTDERR: =====FAIL: nptl/tst-cleanupx4.out=====
BUILDSTDERR: test 0
BUILDSTDERR: clh (2)
BUILDSTDERR: global = 2, expected 15
BUILDSTDERR: test 1
BUILDSTDERR: clh (4)
BUILDSTDERR: clh (6)
BUILDSTDERR: global = 30, expected 276
BUILDSTDERR: test 2
BUILDSTDERR: clh (8)
BUILDSTDERR: global = 8, expected 120
BUILDSTDERR: test 3
BUILDSTDERR: clh (2)
BUILDSTDERR: clh (10)
BUILDSTDERR: global = 30, expected 460
BUILDSTDERR: =====FAIL: nptl/tst-cleanupx4.test-result=====
BUILDSTDERR: FAIL: nptl/tst-cleanupx4
BUILDSTDERR: original exit status 1

BUILDSTDERR: =====FAIL: nptl/tst-oncex3.out=====
BUILDSTDERR: once_handler1: locked
BUILDSTDERR: once_handler1: going to wait on cond
BUILDSTDERR: going to cancel
BUILDSTDERR: joined successfully
BUILDSTDERR: once = 0
BUILDSTDERR: cleanup handler not called
BUILDSTDERR: =====FAIL: nptl/tst-oncex3.test-result=====
BUILDSTDERR: FAIL: nptl/tst-oncex3
BUILDSTDERR: original exit status 1
BUILDSTDERR: =====FAIL: nptl/tst-oncex4.out=====
BUILDSTDERR: joined first thread
BUILDSTDERR: cl_called = 0
BUILDSTDERR: =====FAIL: nptl/tst-oncex4.test-result=====
BUILDSTDERR: FAIL: nptl/tst-oncex4
BUILDSTDERR: original exit status 1

This looks like a problem with unwind information.

However, it doesn't look like binutils or gcc changed. So I don't know what's wrong with the tests. A rebuild doesn't change the results.

I'm rebuilding glibc-2.27-36.fc28 (previous version) to see what happens. I've filed this bug report just to record the failure for future review.

Comment 1 Carlos O'Donell 2018-12-14 02:54:00 UTC
When I rebuild glibc-2.27-36.fc28 (previous version) all of the same i686 tests fail.

Comment 2 Florian Weimer 2018-12-14 17:17:44 UTC
For debug/tst-backtrace5, I see this:

Obtained backtrace with 3 functions

But:

  /*  Check that there are at least seven functions.  */
  if (n < NUM_FUNCTIONS)
    {
      FAIL ();
      return;
    }

NUM_FUNCTIONS is 7.

And:

(gdb) print addresses
$1 = {0x8049482 <handle_signal+18>, 0xf7fd2060 <__kernel_sigreturn>, 
  0xf7fd2049 <__kernel_vsyscall+9>, 0x0, 0x1000, 0x13c0, 0x0}
(gdb) bt
#0  0xf7e6d920 in printf () from /lib/libc.so.6
#1  0x080495b7 in handle_signal (signum=10) at tst-backtrace5.c:53
#2  <signal handler called>
#3  0xf7fd2049 in __kernel_vsyscall ()
#4  0xf7ef33ab in read () from /lib/libc.so.6
#5  0x08049719 in fn (c=0, flags=0) at tst-backtrace5.c:127
#6  0x08049763 in fn (c=1, flags=0) at tst-backtrace5.c:103
#7  0x08049763 in fn (c=2, flags=0) at tst-backtrace5.c:103
#8  0x0804978c in do_test () at tst-backtrace5.c:135
#9  0x08049d72 in support_test_main (argc=1, argv=<optimized out>, 
    config=0xffffcd64) at support_test_main.c:350
#10 0x0804935e in main (argc=2, argv=0xffffce54)
    at ../support/test-driver.c:168

So glibc's backtrace can no longer unwind through signal handlers, which of course breaks cancellation tests and more.  Downgrading libgcc does not fix this.  Downgrading to kernel-4.18.16-300.fc29.x86_64 fixes the problem.

Comment 3 Florian Weimer 2018-12-14 18:00:05 UTC
I believe the unwind information in the kernel vDSO is broken.  The unwinding information has been stripped.

/lib/modules/4.18.16-300.fc29.x86_64/vdso/vdso32.so has this:

  GNU_EH_FRAME   0x0005e4 0x000005e4 0x000005e4 0x000024 0x000024 R   0x4

But /lib/modules/4.19.8-300.fc29.x86_64/vdso/vdso32.so has this:

  GNU_EH_FRAME   0x000000 0x00000000 0x00000000 0x000000 0x000000     0x4

vdso64.so seems affected as well, but the libgcc fallback code in x86_frob_update_context seems to work there.

Comment 4 Laura Abbott 2018-12-14 18:37:38 UTC
Is this i686 only? If so, I want to update the proper tracker so the i686 SIG can track this.

Comment 5 Florian Weimer 2018-12-14 18:48:12 UTC
(In reply to Laura Abbott from comment #4)
> Is this i686 only? If so, I want to update the proper tracker so the i686
> SIG can track this.

No, the segment is missing from vdso64.so as well.  The only difference appears to be that the unwinder is able to cope, but this may be because x86-64 using __kernel_rt_sigreturn (which is recognized by the unwinder), not __kernel_sigreturn.

Comment 6 Laura Abbott 2018-12-14 20:28:20 UTC
Issue appears to be caused by https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=379d98ddf41344273d9718556f761420f4dc80b3 , "x86: vdso: Use $LD instead of $CC to link". I don't know enough details to guess what's happening on but I'll report this to the maintainers unless someone beats me to it.

Comment 7 Florian Weimer 2018-12-14 20:43:58 UTC
Oops.  Now I'm remembering where I saw this before.

Comment 8 H.J. Lu 2018-12-14 22:01:25 UTC
*** Bug 1651010 has been marked as a duplicate of this bug. ***

Comment 9 Justin M. Forbes 2019-01-29 16:24:43 UTC
*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There are a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 28 kernel bugs.

Fedora 28 has now been rebased to 4.20.5-100.fc28.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you have moved on to Fedora 29, and are still experiencing this issue, please change the version to Fedora 29.

If you experience different issues, please open a new bug report for those.

Comment 10 Florian Weimer 2019-01-29 17:02:51 UTC
I can confirm that this is fixed in kernel-4.20.4-200.fc29.x86_64.


Note You need to log in before you can comment on or make changes to this bug.