1946948 – gdb crashes with: ../../gdb/dwarf2-frame.c:1061: internal-error: Unknown CFA rule.

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1946948 - gdb crashes with: ../../gdb/dwarf2-frame.c:1061: internal-error: Unknown CFA rule.

Summary: gdb crashes with: ../../gdb/dwarf2-frame.c:1061: internal-error: Unknown CFA ...

Keywords:
Status:	CLOSED WORKSFORME
Alias:	None
Product:	Red Hat Enterprise Linux 8
Classification:	Red Hat
Component:	gdb
Sub Component:
Version:	8.5
Hardware:	aarch64
OS:	Unspecified
Priority:	unspecified
Severity:	unspecified
Target Milestone:	beta
Target Release:	---
Assignee:	Keith Seitz
QA Contact:	qe-baseos-tools-bugs
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2021-04-07 10:33 UTC by Richard W.M. Jones
Modified:	2023-09-15 01:04 UTC (History)
CC List:	5 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2021-05-17 15:37:59 UTC
Type:	Bug
Target Upstream Version:
Embargoed:
Dependent Products:
Flags:	pm-rhel: mirror+

Attachments	(Terms of Use)

Description Richard W.M. Jones 2021-04-07 10:33:13 UTC

Description of problem:

gdb crashes when displaying a stack trace (ie "bt" command):

# coredumpctl gdb
...
GNU gdb (GDB) Red Hat Enterprise Linux 8.2-15.el8
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "aarch64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/rjones/d/nbdkit/tests/test-layers...done.
[New LWP 349487]
[New LWP 349486]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `./test-layers'.
Program terminated with signal SIGABRT, Aborted.
#0  0x0000ffffab292bec in raise () from /lib64/libc.so.6
[Current thread is 1 (Thread 0xffffaa83efc0 (LWP 349487))]
Missing separate debuginfos, use: yum debuginfo-install glibc-2.28-154.el8.aarch64 gmp-6.1.2-10.el8.aarch64 gnutls-3.6.14-7.el8_3.aarch64 libffi-3.1-22.el8.aarch64 libgcc-8.4.1-1.el8.aarch64 libidn2-2.2.0-1.el8.aarch64 libnbd-1.6.0-3.module+el8.4.0+9856+bbc47853.aarch64 libtasn1-4.13-3.el8.aarch64 libunistring-0.9.9-3.el8.aarch64 libxml2-2.9.7-9.el8.aarch64 nettle-3.4.1-2.el8.aarch64 p11-kit-0.23.22-1.el8.aarch64 xz-libs-5.2.4-3.el8.aarch64 zlib-1.2.11-17.el8.aarch64
(gdb) bt
#0  0x0000ffffab292bec in raise () from /lib64/libc.so.6
#1  0x0000ffffab28096c in abort () from /lib64/libc.so.6
#2  0x0000ffffa9ffd594 in uw_update_context_1 () from /lib64/libgcc_s.so.1
#3  0x0000ffffa9ffd8e0 in uw_update_context () from /lib64/libgcc_s.so.1
#4  0x0000ffffa9ffdaf4 in _Unwind_ForcedUnwind_Phase2 ()
   from /lib64/libgcc_s.so.1
#5  0x0000ffffa9ffdeb8 in _Unwind_ForcedUnwind () from /lib64/libgcc_s.so.1
#6  0x0000ffffab3f0e30 in __pthread_unwind () from /lib64/libpthread.so.0
#7  0x0000ffffab3e87e8 in pthread_exit () from /lib64/libpthread.so.0
../../gdb/dwarf2-frame.c:1061: internal-error: Unknown CFA rule.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Quit this debugging session? (y or n) y

This is a bug, please report it.  For instructions, see:
<http://www.gnu.org/software/gdb/bugs/>.

../../gdb/dwarf2-frame.c:1061: internal-error: Unknown CFA rule.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Create a core file of GDB? (y or n) y
gdb terminated by signal ABRT.

This produces a coredump for gdb itself which has:

(gdb) bt
#0  0x0000ffffa5d92bec in raise () from /lib64/libc.so.6
#1  0x0000ffffa5d8096c in abort () from /lib64/libc.so.6
#2  0x0000aaaabdacfcb4 in dump_core() ()
#3  0x0000aaaabdad293c in internal_vproblem(internal_problem*, char const*, int, char const*, std::__va_list) ()
#4  0x0000aaaabdad2a54 in internal_verror(char const*, int, char const*, std::__va_list) ()
#5  0x0000aaaabd921a34 in internal_error(char const*, int, char const*, ...) ()
#6  0x0000aaaabd955228 in dwarf2_frame_cache(frame_info*, void**) ()
#7  0x0000aaaabd955320 in dwarf2_frame_this_id(frame_info*, void**, frame_id*)
    ()
#8  0x0000aaaabd99e3d0 in compute_frame_id(frame_info*) ()
#9  0x0000aaaabd99ea4c in get_prev_frame_if_no_cycle(frame_info*) ()
#10 0x0000aaaabd9a0810 in get_prev_frame_always_1(frame_info*) ()
#11 0x0000aaaabd9a0dec in get_prev_frame_always(frame_info*) ()
#12 0x0000aaaabd9a1138 in get_prev_frame(frame_info*) ()
#13 0x0000aaaabda6fe44 in backtrace_command(char const*, int) ()
#14 0x0000aaaabd81d448 in cmd_func(cmd_list_element*, char const*, int) ()
#15 0x0000aaaabdab4298 in execute_command(char const*, int) ()
#16 0x0000aaaabd994a9c in command_handler(char const*) ()
#17 0x0000aaaabd995824 in command_line_handler(char*) ()
#18 0x0000aaaabd9940c4 in gdb_rl_callback_handler(char*) ()
#19 0x0000ffffa6a80928 in rl_callback_read_char () from /lib64/libreadline.so.7
#20 0x0000aaaabd993ff4 in gdb_rl_callback_read_char_wrapper_noexcept() ()
#21 0x0000aaaabd99406c in gdb_rl_callback_read_char_wrapper(void*) ()
#22 0x0000aaaabd994648 in stdin_event_handler(int, void*) ()
#23 0x0000aaaabd993170 in gdb_wait_for_event(int) ()
#24 0x0000aaaabd99347c in gdb_do_one_event() [clone .part.11] ()
#25 0x0000aaaabd99357c in start_event_loop() ()
#26 0x0000aaaabd9f50bc in captured_command_loop() ()
#27 0x0000aaaabd9f6634 in gdb_main(captured_main_args*) ()
#28 0x0000aaaabd76d080 in main ()

Version-Release number of selected component (if applicable):

gdb-8.2-15.el8.aarch64

How reproducible:

100% - I've had this problem with many different core dumps
from this aarch64 RHEL 8.5 machine.

Steps to Reproduce:
1. Open a coredump in gdb.
2. Ask for a stack trace.

I found this reference online to a bug that looks superficially
similar: https://www.sourceware.org/bugzilla/show_bug.cgi?id=26876

Comment 1 Richard W.M. Jones 2021-04-07 10:37:28 UTC

Here's the gdb internal coredump with debug symbols:

#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x0000ffffa5d8096c in __GI_abort () at abort.c:79
#2  0x0000aaaabdacfcb4 in dump_core () at ../../gdb/utils.c:262
#3  0x0000aaaabdad293c in internal_vproblem (
    problem=problem@entry=0xaaaabdf02f38 <internal_error_problem>, 
    file=file@entry=0xaaaabdc868d0 "../../gdb/dwarf2-frame.c", 
    line=line@entry=1061, fmt=<optimized out>, 
    ap=<error reading variable: Cannot access memory at address 0x0>)
    at ../../gdb/utils.c:472
#4  0x0000aaaabdad2a54 in internal_verror (
    file=file@entry=0xaaaabdc868d0 "../../gdb/dwarf2-frame.c", 
    line=line@entry=1061, fmt=<optimized out>, ap=...) at ../../gdb/utils.c:497
#5  0x0000aaaabd921a34 in internal_error (
    file=file@entry=0xaaaabdc868d0 "../../gdb/dwarf2-frame.c", 
    line=line@entry=1061, fmt=<optimized out>) at ../../gdb/common/errors.c:55
#6  0x0000aaaabd955228 in dwarf2_frame_cache (this_frame=this_frame@entry=
    0xaaaaed27b910, this_cache=<optimized out>)
    at ../../gdb/dwarf2-frame.c:1061
#7  0x0000aaaabd955320 in dwarf2_frame_this_id (this_frame=0xaaaaed27b910, 
    this_cache=<optimized out>, this_id=0xaaaaed27b970)
    at ../../gdb/dwarf2-frame.c:1197
#8  0x0000aaaabd99e3d0 in compute_frame_id (fi=fi@entry=0xaaaaed27b910)
    at ../../gdb/frame.c:521
#9  0x0000aaaabd99ea4c in get_prev_frame_if_no_cycle (
    this_frame=this_frame@entry=0xaaaaed279340) at ../../gdb/frame.c:1902
#10 0x0000aaaabd9a0810 in get_prev_frame_always_1 (
    this_frame=this_frame@entry=0xaaaaed279340) at ../../gdb/frame.c:2084
#11 0x0000aaaabd9a0dec in get_prev_frame_always (
    this_frame=this_frame@entry=0xaaaaed279340) at ../../gdb/frame.c:2100
#12 0x0000aaaabd9a1138 in get_prev_frame (
    this_frame=this_frame@entry=0xaaaaed279340) at ../../gdb/frame.c:2353
#13 0x0000aaaabda6fe44 in backtrace_command_1 (from_tty=1, 
    no_filters=<optimized out>, flags=..., count_exp=<optimized out>)
    at ../../gdb/stack.c:1808
#14 backtrace_command (arg=<optimized out>, from_tty=1)
    at ../../gdb/stack.c:1865
#15 0x0000aaaabd81d448 in cmd_func (cmd=<optimized out>, args=<optimized out>, 
    from_tty=<optimized out>) at ../../gdb/cli/cli-decode.c:1857
#16 0x0000aaaabdab4298 in execute_command (p=<optimized out>, 
    p@entry=0xaaaaeb1f0a50 "bt", from_tty=1) at ../../gdb/top.c:630
#17 0x0000aaaabd994a9c in command_handler (command=0xaaaaeb1f0a50 "bt")
    at ../../gdb/event-top.c:586
#18 0x0000aaaabd995824 in command_line_handler (rl=<optimized out>)
    at ../../gdb/event-top.c:777
#19 0x0000aaaabd9940c4 in gdb_rl_callback_handler (rl=0xaaaaecdd0d00 "")
    at ../../gdb/event-top.c:214
#20 0x0000ffffa6a80928 in rl_callback_read_char () at ../callback.c:283
#21 0x0000aaaabd993ff4 in gdb_rl_callback_read_char_wrapper_noexcept ()
    at ../../gdb/event-top.c:176
#22 0x0000aaaabd99406c in gdb_rl_callback_read_char_wrapper (
    client_data=<optimized out>) at ../../gdb/event-top.c:192
#23 0x0000aaaabd994648 in stdin_event_handler (error=<optimized out>, 
    client_data=0xaaaaeb1ee920) at ../../gdb/event-top.c:514
#24 0x0000aaaabd993170 in gdb_wait_for_event (block=<optimized out>)
    at ../../gdb/event-loop.c:859
#25 gdb_wait_for_event (block=<optimized out>) at ../../gdb/event-loop.c:746
#26 0x0000aaaabd99347c in gdb_do_one_event () at ../../gdb/event-loop.c:347
#27 0x0000aaaabd99357c in gdb_do_one_event () at ../../gdb/event-loop.c:371
#28 start_event_loop () at ../../gdb/event-loop.c:371
#29 0x0000aaaabd9f50bc in captured_command_loop () at ../../gdb/main.c:331
#30 0x0000aaaabd9f6634 in captured_main (data=0xffffdcafee20)
    at ../../gdb/main.c:1267
#31 gdb_main (args=0xffffdcafee20) at ../../gdb/main.c:1284
#32 0x0000aaaabd76d080 in main (argc=<optimized out>, argv=<optimized out>)
    at ../../gdb/gdb.c:40

Comment 2 Richard W.M. Jones 2021-04-07 12:30:50 UTC

Reproducer can be found here: https://bugzilla.redhat.com/show_bug.cgi?id=1946977#c0

Comment 3 Keith Seitz 2021-04-08 15:32:32 UTC

I'm looking into this, but I have to ask: Have you tried GTS yet? That might get you up and working again.

Unfortunately, we have very little QE capacity for RHEL8.5, so even when a patch is available, there is low likelihood that we can include it.

Comment 4 Richard W.M. Jones 2021-04-08 15:58:29 UTC

What's GTS?  Anyway if this bug only happens on aarch64 then it might
be caused by the recent ARM v8.6 changes (bug 1875912) which caused
lots of brokenness.

Comment 5 Keith Seitz 2021-04-08 16:17:05 UTC

GCC Toolset aka "DTS for RHEL8" is the "latest and greatest" tools, including GDB.

Available to everyone:
# dnf install gcc-toolset-10-gdb

GTS and DTS 10 both ship gdb-9.2.

Comment 6 Keith Seitz 2021-04-08 16:39:49 UTC

For the record, I just checked GTS10, and that fares no better.

Comment 7 Alexander Larsson 2021-04-21 14:36:37 UTC

I get the same results with gdb 8.2-15.el8 on centos-streams on aarch64:

$ gdb /usr/bin/true
...
(gdb) break main
...
(gdb) r
...
(gbd) bt
../../gdb/dwarf2-frame.c:1061: internal-error: Unknown CFA rule.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Create a core file of GDB? (y or n)

Comment 8 Keith Seitz 2021-05-04 20:30:26 UTC

Update: I've heard there has been binutils(?) issue on aarch64 across our product portfolio,
and I've retested these bugs. Using Alexander's reproducer [with debuginfo for coreutils installed]:

rhel8.5$ gdb /usr/bin/true -ex start -ex bt -batch
GNU gdb (GDB) Red Hat Enterprise Linux 8.2-15.el8
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
[root@hpe-apollo-cn99xx-15-vm-15 ~]# gdb /usr/bin/true
GNU gdb (GDB) Red Hat Enterprise Linux 8.2-15.el8
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "aarch64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/true...Reading symbols from /usr/lib/debug/usr/bin/true-8.30-8.el8.aarch64.debug...done.
done.
(gdb) start
Temporary breakpoint 1 at 0x1600: file ../src/true.c, line 59.
Starting program: /usr/bin/true 
Missing separate debuginfos, use: yum debuginfo-install glibc-2.28-158.el8.aarch64

Temporary breakpoint 1, main (argc=1, argv=0xffffffffeee8) at ../src/true.c:59
59	  if (argc == 2)
(gdb) bt
#0  main (argc=1, argv=0xffffffffeee8) at ../src/true.c:59
(gdb) quit
A debugging session is active.

	Inferior 1 [process 15865] will be killed.

Quit anyway? (y or n) y
rhel8.5$

I get identical results on 9-beta and 8.4.

@alexl, @rjones, can you please update your machines and retest?

I've tested RHEL-8.5.0-20210504.n.0.

Comment 9 stalkerg@gmail.com 2021-05-06 15:29:56 UTC

After update glibc and binutils on centos8 stream this bug is gone. Thanks.

Comment 10 Richard W.M. Jones 2021-05-10 10:39:39 UTC

I don't have time to test this and my test machine has long been returned
to beaker.  I'm happy to go with your fix and other reporters.

Comment 11 Red Hat Bugzilla 2023-09-15 01:04:44 UTC

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days

Note You need to log in before you can comment on or make changes to this bug.