Bug 1440725
| Summary: | crash: KASLR: gdb cannot find text block for address: ring_buffer_read when extending trace.so | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Emma Wu <xiawu> |
| Component: | crash | Assignee: | Dave Anderson <anderson> |
| Status: | CLOSED ERRATA | QA Contact: | Emma Wu <xiawu> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 7.4 | CC: | feij.fnst, ksanagi, qzhao, tumeya, yishimat |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | crash-7.1.9-1.el7 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-08-01 22:04:38 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Comment 2
Dave Anderson
2017-04-10 14:48:56 UTC
With kernel-3.10.0-632.el7: crash> extend extensions/trace.so extend: gdb cannot find text block for address: ring_buffer_read ./extensions/trace.so: shared object loaded crash> sym ring_buffer_read ffffffff8f145360 (T) ring_buffer_read /usr/src/debug/kernel-3.10.0-632.el7/linux-3.10.0-632.el7.x86_64/kernel/trace/ring_buffer.c: 4081 crash> May be KASLR-related? (In reply to Dave Anderson from comment #3) > With kernel-3.10.0-632.el7: > > crash> extend extensions/trace.so > extend: gdb cannot find text block for address: ring_buffer_read > ./extensions/trace.so: shared object loaded > crash> sym ring_buffer_read > ffffffff8f145360 (T) ring_buffer_read > /usr/src/debug/kernel-3.10.0-632.el7/linux-3.10.0-632.el7.x86_64/kernel/ > trace/ring_buffer.c: 4081 > crash> > > May be KASLR-related? Ok, so this is actually a crash issue, associated with KASLR. The crash-trace-command needs to set the gdb text "scope" to that of the ring_buffer_read() text function, or else it utilizes the address of an incorrect, duplicate, "ring_buffer" kernel symbol: BZ #1265553 - crash-trace-command: failed to init the offset, struct:ftrace_event_call, member:list https://bugzilla.redhat.com/show_bug.cgi?id=1265553 Any, the crash-trace-command module utilizes the crash utility function gdb_set_crash_scope(), which is what is generating this error message: extend: gdb cannot find text block for address: ring_buffer_read The embedded gdb maintains "text blocks" based upon the contents of the "vmlinux" file, which has kernel virtual addresses that have been subsequently modified by KASLR when the kernel is loaded. The fix is in the crash utility, which must revert the KASLR address of the ring_buffer_read symbol back to the value seen in the vmlinux file. (It does the same thing for generating text line numbers.) Note to QE: It is not necessary to utilize the crash-trace-command "trace.so" extension module to see/test this problem. On a recent KASLR-enabled kernel, and any text symbol will fail: $ crash crash 7.1.8-2.el7 Copyright (C) 2002-2016 Red Hat, Inc. Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation Copyright (C) 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. Copyright (C) 2005, 2011 NEC Corporation Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. This program is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Enter "help copying" to see the conditions. This program has absolutely no warranty. Enter "help warranty" for details. GNU gdb (GDB) 7.6 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-unknown-linux-gnu"... WARNING: kernel relocated [224MB]: patching 77374 gdb minimal_symbol values KERNEL: /usr/lib/debug/lib/modules/3.10.0-632.el7.x86_64/vmlinux DUMPFILE: /dev/crash CPUS: 8 DATE: Mon Apr 10 12:22:09 2017 UPTIME: 01:02:57 LOAD AVERAGE: 0.43, 0.14, 0.09 TASKS: 199 NODENAME: hp-xw6400-02.khw.lab.eng.bos.redhat.com RELEASE: 3.10.0-632.el7.x86_64 VERSION: #1 SMP Mon Mar 27 22:53:17 EDT 2017 MACHINE: x86_64 (1595 Mhz) MEMORY: 2 GB PID: 14099 COMMAND: "crash" TASK: ffff8800780fde20 [THREAD_INFO: ffff88006aa98000] CPU: 5 STATE: TASK_RUNNING (ACTIVE) crash> set scope sys_read set: gdb cannot find text block for address: sys_read crash> Fix posted to the upstream git repository: https://github.com/crash-utility/crash/commit/270d8b40a4a0fb72e952e4db523420f83dc85d32 Fix for the "set scope" option if the kernel was configured with CONFIG_RANDOMIZE_BASE. Without the patch, the command fails with the message "set: gdb cannot find text block for address: <symbol>". This also affects extension modules that call gdb_set_crash_scope() when running with KASLR kernels. (anderson) With the patch applied: crash> sys | grep RELEASE RELEASE: 3.10.0-644.el7.x86_64 crash> set scope sys_read scope: ffffffff8cdfefc0 (sys_read) crash> extend trace.so ./extensions/trace.so: shared object loaded crash> Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:2019 |