Bug 458417

Summary: SIGSEGV when search -k on IA64
Product: Red Hat Enterprise Linux 5 Reporter: Qian Cai <qcai>
Component: crashAssignee: Dave Anderson <anderson>
Status: CLOSED ERRATA QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: 5.2CC: duck, syeghiay
Target Milestone: rc   
Target Release: ---   
Hardware: ia64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-01-20 22:13:45 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Qian Cai 2008-08-08 03:49:10 UTC
Description of problem:
When analysing vmcore from IA64 machines, search -k caused crash SIGSEGV.

[root@hp-matterhorn1 ~]# crash /usr/lib/debug/lib/modules/2.6.18-92.el5/vmlinux /var/crash/127.0.0.1-2008-08-07-22\:25\:27/vmcore 

crash 4.0-5.0.3
Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007, 2008  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.
 
GNU gdb 6.1
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "ia64-unknown-linux-gnu"...

      KERNEL: /usr/lib/debug/lib/modules/2.6.18-92.el5/vmlinux
    DUMPFILE: /var/crash/127.0.0.1-2008-08-07-22:25:27/vmcore  [PARTIAL DUMP]
        CPUS: 8
        DATE: Thu Aug  7 22:24:40 2008
      UPTIME: 00:11:49
LOAD AVERAGE: 0.29, 0.28, 0.18
       TASKS: 144
    NODENAME: hp-matterhorn1.rhts.bos.redhat.com
     RELEASE: 2.6.18-92.el5
     VERSION: #1 SMP Tue Apr 29 13:18:26 EDT 2008
     MACHINE: ia64  (1300 Mhz)
      MEMORY: 31.4 GB
       PANIC: "SysRq : Trigger a crashdump"
         PID: 3096
     COMMAND: "bash"
        TASK: e0000040f4978000  [THREAD_INFO: e0000040f4979040]
         CPU: 1
       STATE: TASK_RUNNING (SYSRQ)

crash> search -k deadbeef
Segmentation fault (core dumped)

[root@hp-matterhorn1 ~]# gdb /usr/bin/crash core.4583 
GNU gdb Red Hat Linux (6.5-37.el5rh)
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "ia64-redhat-linux-gnu"...(no debugging symbols found)
Using host libthread_db library "/lib/libthread_db.so.1".


warning: Can't read pathname for load map: Input/output error.
Reading symbols from /lib/libm.so.6.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libm.so.6.1
Reading symbols from /usr/lib/libncurses.so.5...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libncurses.so.5
Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /usr/lib/libz.so.1...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libz.so.1
Reading symbols from /lib/libc.so.6.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6.1
Reading symbols from /lib/ld-linux-ia64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux-ia64.so.2
Reading symbols from /lib/libthread_db.so.1...
(no debugging symbols found)...done.
Loaded symbols for /lib/libthread_db.so.1
(no debugging symbols found)
Core was generated by `crash /usr/lib/debug/lib/modules/2.6.18-92.el5/vmlinux /var/crash/127.0.0.1-200'.
Program terminated with signal 11, Segmentation fault.
#0  0x4000000000106ee0 in nr_to_section ()
(gdb) bt
#0  0x4000000000106ee0 in nr_to_section ()
#1  0x40000000001078d0 in valid_section_nr ()
#2  0x4000000000107a50 in pfn_to_map ()
#3  0x40000000000f2930 in is_page_ptr ()
#4  0x40000000000f9660 in cmd_search ()
#5  0x40000000000f9190 in cmd_search ()
#6  0x400000000005dd30 in exec_command ()
#7  0x400000000005d910 in main_loop ()
#8  0x400000000054a4d0 in current_interp_command_loop ()
#9  0x40000000003870b0 in gdb_main_entry ()
#10 0x400000000040c560 in f_val_print ()
#11 0x4000000000411420 in throw_exception ()
#12 0x40000000004114e0 in catch_errors ()
#13 0x4000000000388020 in gdb_main_entry ()
#14 0x400000000040c560 in f_val_print ()
#15 0x4000000000411420 in throw_exception ()
#16 0x40000000004114e0 in catch_errors ()
#17 0x4000000000386fd0 in gdb_main ()
#18 0x4000000000387070 in gdb_main_entry ()
#19 0x40000000001b9de0 in gdb_main_loop ()
#20 0x400000000005d2f0 in main ()


Version-Release number of selected component (if applicable):
kernel-2.6.18-92.el5
crash-4.0-5.0.3
kexec-tools-1.102pre-21.el5

How reproducible:
always

Steps to Reproduce:
1. configure Kdump with 512M@256M
2. SysRq-C
3. analyse vmcore, and run search -k
  
Actual results:
SIGSEGV

Expected results:
Search the value at the base of Kernel space address.

Additional info:
vmcore/vmlinux pair could be found at,
porkchop.devel.redhat.com:/mnt/redhat/qa/qa/qcai/bz/search_ia64

Comment 1 Dave Anderson 2008-08-08 12:40:56 UTC
I believe that has been fixed in the upstream version of crash:

  http://people.redhat.com/anderson/crash.changelog.html#4_0_6_2


         - Fix/workaround for the "search -k" command option on relocatable
           2.6-era ia64 machines configured with CONFIG_SPARSEMEM.  Without 
           the patch, an immediate segmentation violation occurs.
           (anderson, yzgcsu.com)

Please download and test the upstream version of crash (4.0-6.3) and build it
like so:

  # tar xvzmf crash-4.0-6.3.tar.gz
  ...
  # cd crash-4.0-6.3
  # make
  ...
  # ./crash <vmlinux> <vmcore>

Comment 2 Qian Cai 2008-08-11 03:08:36 UTC
Confirmed the upstream version fixed the problem. Thanks!

Comment 7 Dave Anderson 2008-09-24 13:40:08 UTC
*** Bug 463726 has been marked as a duplicate of this bug. ***

Comment 8 Dave Anderson 2008-09-24 13:46:49 UTC
Although crash version 4.0-7.2.1 (and 4.0-7.2.2) fixes the SIGSEGV problem
with the 2.6.18-92.el5 dumpfile attached to this bugzilla (as shown above
in comment #6), Cai Quan has uncovered another bug with the same
command when run on a 2.6.18-116.el5 kernel.  So the fix is incomplete:

This is from duplicate BZ #463726:

# crash  /usr/lib/debug/lib/modules/2.6.18-116.el5/vmlinux
/var/crash/2008-09-24-04:41/vmcore

crash 4.0-7.2.2
Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007, 2008  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.

GNU gdb 6.1
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "ia64-unknown-linux-gnu"...

      KERNEL: /usr/lib/debug/lib/modules/2.6.18-116.el5/vmlinux
    DUMPFILE: /var/crash/2008-09-24-04:41/vmcore
        CPUS: 2
        DATE: Wed Sep 24 04:40:47 2008
      UPTIME: 00:28:49
LOAD AVERAGE: 0.63, 0.25, 0.09
       TASKS: 98
    NODENAME: hp-lp1.rhts.bos.redhat.com
     RELEASE: 2.6.18-116.el5
     VERSION: #1 SMP Thu Sep 18 18:13:01 EDT 2008
     MACHINE: ia64  (900 Mhz)
      MEMORY: 1.5 GB
       PANIC: "SysRq : Trigger a crashdump"
         PID: 2416
     COMMAND: "bash"
        TASK: e0000040471c8000  [THREAD_INFO: e0000040471c9040]
         CPU: 1
       STATE: TASK_RUNNING (SYSRQ)

crash> search -k deadbeef
search: ia64_VTOP(a000000200000000): unexpected region 5 address

crash>

Comment 13 errata-xmlrpc 2009-01-20 22:13:45 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2009-0240.html