Bug 2203082 - [aarch64] Kernel stacktraces from perf don't show function names unless kernel-debuginfo is installed
Summary: [aarch64] Kernel stacktraces from perf don't show function names unless kerne...
Keywords:
Status: NEW
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: kernel
Version: 9.2
Hardware: aarch64
OS: Linux
unspecified
medium
Target Milestone: rc
: ---
Assignee: Michael Petlan
QA Contact: Qiao Zhao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-05-11 07:57 UTC by Ondrej Mosnacek
Modified: 2023-07-20 07:22 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-157084 0 None None None 2023-05-12 06:55:43 UTC

Description Ondrej Mosnacek 2023-05-11 07:57:28 UTC
Description of problem:
Somewhere between RHEL-9.1 and RHEL-9.2 perf became unable to print function names in kernel stack traces from events on aarch64 unless kernel-debuginfo is installed. On RHEL-9.1 and also on all other arches on RHEL-9.2/9.3 it is able to infer the function names without it (presumably using /proc/kallsyms).

The issue also appears when I boot a RHEL-9.2 kernel on RHEL-9.1 userspace, so it seems to be a regression in the kernel rather than the userspace tools/libraries.

Version-Release number of selected component (if applicable):
Last known good: kernel-5.14.0-162.23.1.el9_1.aarch64
First known bad: kernel-5.14.0-284.14.1.el9_2.aarch64

How reproducible:
Always.

Steps to Reproduce:
1. perf record -g sleep 1
2. perf script

Actual results:
perf-exec 21437   257.001297:          1 cycles: 
        ffffa2362e8ce7d0 [unknown] ([unknown])
        ffffa2362e9f90e0 [unknown] ([unknown])
        ffffa2362ea6a728 [unknown] ([unknown])
        ffffa2362e9f6714 [unknown] ([unknown])
        ffffa2362e9f785c [unknown] ([unknown])
        ffffa2362e9f7fc4 [unknown] ([unknown])
        ffffa2362e9f80b0 [unknown] ([unknown])
        ffffa2362e9f84e8 [unknown] ([unknown])
        ffffa2362e9f85bc [unknown] ([unknown])
        ffffa2362e67636c [unknown] ([unknown])
        ffffa2362e676514 [unknown] ([unknown])
        ffffa2362e67654c [unknown] ([unknown])
        ffffa2362f279348 [unknown] ([unknown])
        ffffa2362f279cb4 [unknown] ([unknown])
        ffffa2362e6615c0 [unknown] ([unknown])
            ffff8147c9cc [unknown] ([unknown])
[...]

Expected results:
perf-exec 21046   143.835852:          1 cycles: 
        ffffb7d2558f43a0 perf_event_remove_on_exec+0x0 ([kernel.kallsyms])
        ffffb7d255a02614 begin_new_exec+0x2e4 ([kernel.kallsyms])
        ffffb7d255a73ae0 load_elf_binary+0x65c ([kernel.kallsyms])
        ffffb7d2559ffb34 search_binary_handler+0xd4 ([kernel.kallsyms])
        ffffb7d255a0191c exec_binprm+0x5c ([kernel.kallsyms])
        ffffb7d255a01c24 bprm_execve.part.0+0x184 ([kernel.kallsyms])
        ffffb7d255a01d10 bprm_execve+0x60 ([kernel.kallsyms])
        ffffb7d255a01ec4 do_execveat_common+0x184 ([kernel.kallsyms])
        ffffb7d255a01f90 __arm64_sys_execve+0x3c ([kernel.kallsyms])
        ffffb7d2556a65cc invoke_syscall.constprop.0+0x7c ([kernel.kallsyms])
        ffffb7d2556a6774 el0_svc_common.constprop.0+0x154 ([kernel.kallsyms])
        ffffb7d2556a67ac do_el0_svc+0x2c ([kernel.kallsyms])
        ffffb7d25625c648 el0_svc+0x38 ([kernel.kallsyms])
        ffffb7d25625cf9c el0t_64_sync_handler+0x9c ([kernel.kallsyms])
        ffffb7d2556915c0 el0t_64_sync+0x174 ([kernel.kallsyms])
            ffffaf9cc9cc [unknown] ([unknown])
[...]

Comment 1 Michael Petlan 2023-07-19 14:33:12 UTC
5.14.0-284.11.1.el9_2.aarch64 is also bad

Comment 2 Michael Petlan 2023-07-20 07:22:39 UTC
perf-5.14.0-340.el9.aarch64 works OK, so RHEL-9.3 solves this problem


Note You need to log in before you can comment on or make changes to this bug.