Bug 1516954

Summary: crash: cannot resolve "init_level4_pgt"
Product: [Fedora] Fedora Reporter: Daniel Bristot de Oliveira <daolivei>
Component: crashAssignee: Dave Anderson <anderson>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 27CC: anderson, cz172638, lherbolt
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: crash-7.2.4-1.fc30 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-09-24 19:06:57 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Daniel Bristot de Oliveira 2017-11-23 16:38:54 UTC 
Description of problem:

 Crash is crashing with the following error:

   crash: cannot resolve "init_level4_pgt"

Version-Release number of selected component (if applicable):
  crash-7.1.9-3.fc27.x86_64

  [root@deskbase crash]# rpm -qa | grep `uname -r`
  kernel-devel-4.13.13-300.fc27.x86_64
  kernel-debuginfo-common-x86_64-4.13.13-300.fc27.x86_64
  kernel-core-4.13.13-300.fc27.x86_64
  kernel-debuginfo-4.13.13-300.fc27.x86_64
  kernel-headers-4.13.13-300.fc27.x86_64
  kernel-modules-extra-4.13.13-300.fc27.x86_64
  kernel-modules-4.13.13-300.fc27.x86_64
  kernel-4.13.13-300.fc27.x86_64

How reproducible:
  always

Steps to Reproduce:
  1. update & reboot the system
  2. install the kernel debuginfo
     # debuginfo install kernel
  3. run crash

Actual results:
------------------ %< -------------------------------
[root@deskbase crash]# crash

crash 7.1.9-3.fc27
Copyright (C) 2002-2016  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.
 
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...

WARNING: kernel relocated [608MB]: patching 89989 gdb minimal_symbol values

                                                                       
crash: cannot resolve "init_level4_pgt"
------------------ >% -------------------------------

Expected results:
  Crash working.

Additional info:
  It seems that this bug was already fixed upstream, in the following commit:

https://github.com/crash-utility/crash/commit/a16324a2f05c0947a83e26a5de7c756de4603da9#diff-2e50715241f69338dda2b8adf90db04a
Comment 1 Dave Anderson 2017-11-23 16:55:10 UTC 
(In reply to Daniel Bristot de Oliveira from comment #0)
>
> Additional info:
>   It seems that this bug was already fixed upstream, in the following commit:
> 
> https://github.com/crash-utility/crash/commit/
> a16324a2f05c0947a83e26a5de7c756de4603da9#diff-
> 2e50715241f69338dda2b8adf90db04a

Try rawhide version crash-7.2.0-1.fc28:

  https://koji.fedoraproject.org/koji/packageinfo?packageID=307
Comment 2 Jiri Kastner 2017-11-23 20:18:55 UTC 
launched build in copr for f26/27
https://copr.fedorainfracloud.org/coprs/jkastner/various/build/678834/

just verified:

[root@localhost ~]# crash 

crash 7.2.0-1.fc27
Copyright (C) 2002-2017  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.
 
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...

WARNING: kernel relocated [768MB]: patching 91234 gdb minimal_symbol values

      KERNEL: /usr/lib/debug/lib/modules/4.14.1-1.rt1.1.fc27.x86_64+rt/vmlinux
    DUMPFILE: /dev/crash
        CPUS: 4
        DATE: Thu Nov 23 21:17:50 2017
      UPTIME: 1 days, 22:01:35
LOAD AVERAGE: 0.48, 0.46, 0.45
       TASKS: 521
    NODENAME: localhost.localdomain
     RELEASE: 4.14.1-1.rt1.1.fc27.x86_64+rt
     VERSION: #1 SMP PREEMPT RT Tue Nov 21 14:12:30 UTC 2017
     MACHINE: x86_64  (2793 Mhz)
      MEMORY: 31.6 GB
         PID: 7615
     COMMAND: "crash"
        TASK: ffff8c281c2b5e80  [THREAD_INFO: ffff8c281c2b5e80]
         CPU: 3
       STATE: TASK_RUNNING (ACTIVE)

crash>
Comment 3 Lukas Herbolt 2018-07-05 08:24:54 UTC 
Just run over this bug, and after update to 7.2.0-1.fc27

[root@test ~]# uname -r
4.17.3-100.fc27.x86_64
[root@test ~]# crash

crash 7.2.0-1.fc27
Copyright (C) 2002-2017  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.
 
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...

WARNING: kernel relocated [976MB]: patching 97906 gdb minimal_symbol values

                                                                       
crash: cannot resolve "init_task_union"

[root@test ~]#
Comment 4 Dave Anderson 2018-07-05 13:41:50 UTC 
(In reply to Lukas Herbolt from comment #3)
> Just run over this bug, and after update to 7.2.0-1.fc27
> 
> [root@test ~]# uname -r
> 4.17.3-100.fc27.x86_64
> [root@test ~]# crash
> 
> crash 7.2.0-1.fc27
> Copyright (C) 2002-2017  Red Hat, Inc.
> Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
> Copyright (C) 1999-2006  Hewlett-Packard Co
> Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
> Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
> Copyright (C) 2005, 2011  NEC Corporation
> Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
> Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
> This program is free software, covered by the GNU General Public License,
> and you are welcome to change it and/or distribute copies of it under
> certain conditions.  Enter "help copying" to see the conditions.
> This program has absolutely no warranty.  Enter "help warranty" for details.
>  
> GNU gdb (GDB) 7.6
> Copyright (C) 2013 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-unknown-linux-gnu"...
> 
> WARNING: kernel relocated [976MB]: patching 97906 gdb minimal_symbol values
> 
>                                                                        
> crash: cannot resolve "init_task_union"
> 
> [root@test ~]#

It's a different bug, which one was fixed here in crash-7.2.1:

 http://people.redhat.com/anderson/crash.changelog.html

     - Fix to support Linux 4.15 and later kernels that contain kernel
       commit e8cfbc245e24887e3c30235f71e9e9405e0cfc39, titled "pid: remove
       pidhash".  The kernel's traditional usage of a pid_hash[] array to
       store PIDs has been replaced by an IDR radix tree, requiring a new
       crash plug-in function to gather the system's task set.  Without the
       patch, the crash session fails during initialization with the error
       message "crash: cannot resolve init_task_union".
       (anderson)

Try the latest Fedora package (crash-7.2.3-1.fc29):

  https://koji.fedoraproject.org/koji/packageinfo?packageID=307

It should work, but there are never guarantees with the crash package,
given that it is at the mercy of the shifting sands of new kernel
versions being introduced into a fedora stream.  It's a losing battle.

If that one doesn't work, you can download the latest git repo and
rebuild with the most recent sources:

  https://github.com/crash-utility/crash