Bug 589954

Summary: Kernel crash at kernel/cred.c:875 while rebuilding glibc-2.11.90-20.el6.src.rpm
Product: Red Hat Enterprise Linux 6 Reporter: Michal Nowak <mnowak>
Component: kernelAssignee: Red Hat Kernel Manager <kernel-mgr>
Status: CLOSED DUPLICATE QA Contact: Red Hat Kernel QE team <kernel-qe>
Severity: high Docs Contact:
Priority: medium    
Version: 6.0CC: notting, ohudlick, simon.derr
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-05-14 08:06:48 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Michal Nowak 2010-05-07 11:41:51 UTC
Description of problem:

I was rebuilding glibc-2.11.90-20.el6.src.rpm and this point

make -s subdir=nptl -C nptl ..=../ tests
make[2]: Entering directory `/root/rpmbuild/BUILD/glibc-2.11-382-g1cdb215/nptl'

machine hanged (one for each i686, s390x, x86_64 and ppc64). From x86_64 I found a vmcore.

[root@auto-x86-64-001 ~]# crash /var/crash/2010-05-07-06\:21/vmcore /usr/lib/debug/lib/modules/`uname -r`/vmlinux

crash 5.0.0-13.el6
GNU gdb (GDB) 7.0

      KERNEL: /usr/lib/debug/lib/modules/2.6.32-23.el6.x86_64/vmlinux
    DUMPFILE: /var/crash/2010-05-07-06:21/vmcore
        CPUS: 2
        DATE: Fri May  7 06:21:09 2010
      UPTIME: 18:52:47
LOAD AVERAGE: 1.86, 2.06, 1.87
       TASKS: 166
    NODENAME: auto-x86-64-001.ss.eng.bos.redhat.com
     RELEASE: 2.6.32-23.el6.x86_64
     VERSION: #1 SMP Tue Apr 27 21:17:28 EDT 2010
     MACHINE: x86_64  (3400 Mhz)
      MEMORY: 3.9 GB
       PANIC: "kernel BUG at kernel/cred.c:875!"
         PID: 15784
     COMMAND: "ld-linux-x86-64"
        TASK: ffff880017467500  [THREAD_INFO: ffff880040c72000]
         CPU: 1
       STATE: TASK_RUNNING (PANIC)

crash> bt -al
PID: 14775  TASK: ffff8800754d9500  CPU: 0   COMMAND: "ld-linux-x86-64"
 #0 [ffff880028207e80] crash_nmi_callback at ffffffff8102bd06
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/arch/x86/include/asm/paravirt.h: 115
 #1 [ffff880028207e90] notifier_call_chain at ffffffff814cc4d5
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/kernel/notifier.c: 95
 #2 [ffff880028207ed0] atomic_notifier_call_chain at ffffffff814cc53a
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/kernel/notifier.c: 192
 #3 [ffff880028207ee0] notify_die at ffffffff81092dae
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/kernel/notifier.c: 573
 #4 [ffff880028207f10] do_nmi at ffffffff814ca5ec
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/arch/x86/kernel/traps.c: 493
 #5 [ffff880028207f50] nmi at ffffffff814c9ee0
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/arch/x86_64/kernel/entry.S
    [exception RIP: nsec_to_clock_t+26]
    RIP: ffffffff8106ddca  RSP: ffff88003d6a1e50  RFLAGS: 00000803
    RAX: 4f63eb61e34a0b62  RBX: ffff8800754d9500  RCX: d6bf94d5e57a42bd
    RDX: 000000000000284c  RSI: 0000000000000000  RDI: 000000000000300a
    RBP: ffff88003d6a1e78   R8: ffff8800754d9b20   R9: ffff8800754d9b30
    R10: ffff8800754d9b20  R11: 0000000000000001  R12: 0000000000000000
    R13: 0000000000000000  R14: ffff880114c70dc0  R15: 0000000000000000
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
--- <NMI exception stack> ---
 #6 [ffff88003d6a1e50] nsec_to_clock_t at ffffffff8106ddca
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/kernel/time.c: 650
 #7 [ffff88003d6a1e50] task_utime at ffffffff81056a57
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/kernel/sched.c: 5227
 #8 [ffff88003d6a1e80] task_stime at ffffffff81056ad3
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/kernel/sched.c: 5246
 #9 [ffff88003d6a1eb0] release_task at ffffffff8106aa3a
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/kernel/exit.c: 115
#10 [ffff88003d6a1ef0] do_exit at ffffffff8106cabe
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/kernel/exit.c: 860
#11 [ffff88003d6a1f70] sys_exit at ffffffff8106ce67
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/kernel/exit.c: 1039
#12 [ffff88003d6a1f80] system_call_fastpath at ffffffff81013132
    /usr/src/debug////////kernel-2.6.32/linux-2.6.32.x86_64/arch/x86/kernel/entry_64.S: 488
    RIP: 00007fa4e3000afe  RSP: 00007fa4c21fbff0  RFLAGS: 00000206
    RAX: 000000000000003c  RBX: ffffffff81013132  RCX: ffffffffffffffff
    RDX: 0000000000000004  RSI: 00000000009fb000  RDI: 0000000000000000
    RBP: 0000000000000000   R8: 00007fa4c21fc710   R9: 00007fa4c21fc710
    R10: 00007fa4c21fc9e0  R11: 0000000000000246  R12: ffffffff8106ce67
    R13: ffff88003d6a1f78  R14: 0000000000000003  R15: 0000000000000000
    ORIG_RAX: 000000000000003c  CS: 0033  SS: 002b

PID: 15784  TASK: ffff880017467500  CPU: 1   COMMAND: "ld-linux-x86-64"
 #0 [ffff880040c73b70] machine_kexec at ffffffff8103433b
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/arch/x86/kernel/machine_kexec_64.c: 336
 #1 [ffff880040c73bd0] crash_kexec at ffffffff810b5728
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/kernel/kexec.c: 1083
 #2 [ffff880040c73ca0] oops_end at ffffffff814ca8d8
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/arch/x86/kernel/dumpstack.c: 226
 #3 [ffff880040c73cd0] die at ffffffff8101721b
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/arch/x86/kernel/dumpstack.c: 304
 #4 [ffff880040c73d00] do_trap at ffffffff814ca1b4
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/arch/x86/kernel/traps.c: 248
 #5 [ffff880040c73d60] do_invalid_op at ffffffff81014e85
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/arch/x86/kernel/traps.c: 289
 #6 [ffff880040c73e00] invalid_op at ffffffff81013f1b
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/arch/x86_64/kernel/entry.S
    [exception RIP: __validate_process_creds+219]
    RIP: ffffffff810942db  RSP: ffff880040c73eb8  RFLAGS: 00010292
    RAX: 000000000000002a  RBX: ffff880017467500  RCX: 0000000000000948
    RDX: 0000000000000000  RSI: 0000000000000046  RDI: 0000000000000246
    RBP: ffff880040c73ed8   R8: 00000000ffffffff   R9: 0000000000000000
    R10: 0000000000000002  R11: 0000000000000000  R12: ffffffff816518da
    R13: 0000000000000379  R14: 0000000000000000  R15: 0000000000000003
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
 #7 [ffff880040c73ee0] validate_creds_for_do_exit at ffffffff81094316
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/kernel/cred.c: 890
 #8 [ffff880040c73ef0] do_exit at ffffffff8106c5b6
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/kernel/exit.c: 910
 #9 [ffff880040c73f70] sys_exit at ffffffff8106ce67
    /usr/src/debug/kernel-2.6.32/linux-2.6.32.x86_64/kernel/exit.c: 1039
#10 [ffff880040c73f80] system_call_fastpath at ffffffff81013132
    /usr/src/debug////////kernel-2.6.32/linux-2.6.32.x86_64/arch/x86/kernel/entry_64.S: 488
    RIP: 00007fa4e3000afe  RSP: 00007fa4d97faff0  RFLAGS: 00000206
    RAX: 000000000000003c  RBX: ffffffff81013132  RCX: ffffffffffffffff
    RDX: 0000000000000004  RSI: 00000000009fb000  RDI: 0000000000000000
    RBP: 0000000000000000   R8: 00007fa4d97fb710   R9: 00007fa4d97fb710
    R10: 00007fa4d97fb9e0  R11: 0000000000000246  R12: ffffffff8106ce67
    R13: ffff880040c73f78  R14: 0000000000000003  R15: 0000000000000000
    ORIG_RAX: 000000000000003c  CS: 0033  SS: 002b


Version-Release number of selected component (if applicable):

2.6.32-23.el6.x86_64

How reproducible:

I guess always, happened 4 times on different archs.

Steps to Reproduce:
1. Rebuild mentioned glibc srpm.
  
Additional info:

The box with vmcore is auto-x86-64-001.ss.eng.bos.redhat.com.

Comment 2 RHEL Program Management 2010-05-07 12:59:10 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.

Comment 3 Guy Streeter 2010-05-13 20:19:30 UTC
This is most likely the same as bug 578268
The line number has changed in this more recent kernel.

Comment 4 Michal Nowak 2010-05-14 08:06:48 UTC
I guess this is dupe, thanks Guy.

*** This bug has been marked as a duplicate of bug 578268 ***

Comment 5 Simon Derr 2010-07-02 08:27:35 UTC
This can be fixed by applying this patch to the kernel:

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=e134d200d57d43b171dcb0b55c178a1a0c7db14a