Bug 1483934
| Summary: | [Pegas] crash: crash cmd reports "WARNING: cannot access vmalloc'd module memory" | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Emma Wu <xiawu> | ||||
| Component: | crash | Assignee: | Dave Anderson <anderson> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Yuming Liu <yumliu> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 7.4-Alt | CC: | blc, bugproxy, ernunes, hannsj_uhl, hkrzesin, jbastian, jfeeney, jjarvis, mahesh.salgaonkar, pbunyan, qzhao, xiawu, yselkowi | ||||
| Target Milestone: | rc | Keywords: | Patch | ||||
| Target Release: | 7.4-Alt | ||||||
| Hardware: | ppc64le | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | crash-7.1.9-2.p1.el7a | Doc Type: | If docs needed, set a value | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | |||||||
| : | 1485391 (view as bug list) | Environment: | |||||
| Last Closed: | 2017-11-09 11:33:54 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1381083, 1440030, 1485391, 1516606 | ||||||
| Attachments: |
|
||||||
|
Description
Emma Wu
2017-08-22 10:07:15 UTC
Created attachment 1316571 [details]
log_live.txt
attached log (log_live.txt) of cmd `crash -d7`
Seems the warning is reported after:
....
<readmem: c0000004b9280000, KVADDR, "page table", 65536, (FOE), 100285a7eb0>
<read_memory_device: addr: c0000004b9280000 paddr: 4b9280000 cnt: 65536>
<read_memory_device: addr: d0000000078e0900 paddr: 800004b19a0900 cnt: 896>
crash: read error: kernel virtual address: d0000000078e0900 type: "module struct"
WARNING: cannot access vmalloc'd module memory
...
(In reply to Emma Wu from comment #3) > Feel free to change the component. Just not very sure where I should place > it.. The crash-trace-command is irrelevant, it is simply trying to read a vmalloc'd memory address, and the base crash utility's virtual address translation (page-table-walkthrough) is failing for some reason. (In reply to Emma Wu from comment #0) > > Version-Release number of selected component (if applicable): > > distro: RHEL-ALT-7.4-20170808.0 > kernel-4.11.0-24.el7a.ppc64le Kernel version 4.11.0-23.el7a.ppc64le works OK: [root@ibm-p8-05-lp6 ~]# crash crash 7.1.9-2.el7 Copyright (C) 2002-2016 Red Hat, Inc. Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation Copyright (C) 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. Copyright (C) 2005, 2011 NEC Corporation Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. This program is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Enter "help copying" to see the conditions. This program has absolutely no warranty. Enter "help warranty" for details. GNU gdb (GDB) 7.6 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "powerpc64le-unknown-linux-gnu"... KERNEL: /usr/lib/debug/lib/modules/4.11.0-23.el7a.ppc64le/vmlinux DUMPFILE: /dev/crash CPUS: 16 DATE: Tue Aug 22 09:48:52 2017 UPTIME: 00:04:45 LOAD AVERAGE: 0.17, 0.05, 0.01 TASKS: 282 NODENAME: ibm-p8-05-lp6.rhts.eng.bos.redhat.com RELEASE: 4.11.0-23.el7a.ppc64le VERSION: #1 SMP Fri Aug 11 11:52:59 EDT 2017 MACHINE: ppc64le (3425 Mhz) MEMORY: 24 GB PID: 2169 COMMAND: "crash" TASK: c0000005eaae9400 [THREAD_INFO: c0000005f463c000] CPU: 9 STATE: TASK_RUNNING (ACTIVE) crash> So it looks like this kernel patch: * Wed Aug 16 2017 Herton R. Krzesinski <herton> [4.11.0-24.el7a] ... [ cut ] ... - [powerpc] mm: Move hash specific pte bits to be top bits of RPN (Scott Wood) [1475882] - [powerpc] mm: Lower the max real address to 53 bits (Scott Wood) [1475882] - [powerpc] mm: Define all PTE bits based on radix definitions (Scott Wood) [1475882] - [powerpc] mm: Define _PAGE_SOFT_DIRTY unconditionally (Scott Wood) [1475882] - [powerpc] mm: Cleanup bits definition between hash and radix (Scott Wood) [1475882] ... This is going to require assistance from IBM. It really needs to be viewed by Hari Krishna Bathini (hbathini.com), but he does not have a bugzilla account under that email address. Hopefully the cc: additions of bugproxy.com or mahesh.salgaonkar.com can get this to his attention. pbunyan adds hkrzesin so its on his radar. (In reply to Dave Anderson from comment #6) > This is going to require assistance from IBM. It really needs to be > viewed by Hari Krishna Bathini (hbathini.com), but he does not > have a bugzilla account under that email address. Hopefully the cc: > additions of bugproxy.com or mahesh.salgaonkar.com > can get this to his attention. Can you mirror this bug to IBM bugzilla so that Hari can take a look ? ------- Comment From vaish123.com 2017-08-23 02:04 EDT------- Please reverse mirror Red Hat Bug 1483934 - crash: crash cmd reports "WARNING: cannot access vmalloc'd module memory" ------- Comment From hbathini.com 2017-08-24 07:06 EDT------- Hello Dave, The below patch should fix the error during vmalloc address translation: https://www.redhat.com/archives/crash-utility/2017-August/msg00005.html ..which reads "[Crash-utility] [PATCH] book3s/ppc64: Lower the max real address to 53 bits" Also, there seem be slight changes in how hugepages are handled. Will look into it and post a separate patch for it but this fix should at least help ppl not using huge pages.. Thanks Hari Hari, Thanks for the quick response! The patch is queued upstream: https://github.com/crash-utility/crash/commit/d7b43c2a527503415dd66dff20daec01c139c6f1 PPC64 kernel commit 2f18d533757da3899f4bedab0b2c051b080079dc lowered the max real address to 53 bits. Without this patch, the warning message "WARNING: cannot access vmalloc'd module memory" appears during initialization, and any command that attempts to read a vmalloc'd kernel virtual address will fail and display "read error" messages. (hbathini.ibm.com) Brendan and/or John, Despite my best efforts to keep the RHEL7.4 version of the crash utility working with Pegas kernels, the recent kernel commit referenced in comment #5 has crippled the crash utility. Can you give me a pointer to the process for creating a Pegas branch of a package? Thanks, Dave Created attachment 1322785 [details]
crash analysis log on kernel-4.11.0-28.el7a.ppc64le.rpm with crash-7.1.9-2.p1.el7a.ppc64le.rpm
------- Comment on attachment From hbathini.com 2017-09-06 15:23 EDT-------
Able to access vmalloc region successfully on kernel-4.11.0-28.el7a.ppc64le.rpm
with crash-7.1.9-2.p1.el7a.ppc64le.rpm
Thanks
Hari
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:3178 ------- Comment From sthoufee.com 2017-11-27 04:56 EDT------- *** Bug 161663 has been marked as a duplicate of this bug. *** |