We're getting an illegal instruction while using libasan. GDB is reporting the illegal instruction happens during a branch to a PLT stub: Core was generated by `/usr/bin/ls --color=auto'. Program terminated with signal SIGILL, Illegal instruction. #0 0x000040000093e0e0 in memset () from /usr/lib64/libasan.so.8 Missing separate debuginfos, use: dnf debuginfo-install coreutils-9.4-7.fc40.aarch64 >>> disas Dump of assembler code for function memset: => 0x000040000093e0e0 <+0>: b 0x4000008a0690 <__interceptor_memset@plt> End of assembler dump. The branch instruction looks correct. According to comments in IRC channel #fedora-arm, the issue does not happen when the kernel is booted with parameter arm64.nobti. Looking at the library, it does have all the features for BTI and PAC: $ readelf -aW /usr/lib64/libasan.so.8 | grep BTI 0x0000000070000001 (AARCH64_BTI_PLT) GNU 0x00000010 NT_GNU_PROPERTY_TYPE_0 Properties: AArch64 feature: BTI, PAC But when we disassemble it, most of the PLT stubs do bot have BTI/PAC instructions: /usr/lib64/libasan.so.8: file format elf64-littleaarch64 Disassembly of section .init: 0000000000040000 <.init>: 40000: d503233f paciasp 40004: a9bf7bfd stp x29, x30, [sp, #-16]! 40008: 910003fd mov x29, sp 4000c: 94000b48 bl 42d2c <*ABS*+0x8@plt+0xcc> 40010: a8c17bfd ldp x29, x30, [sp], #16 40014: d50323bf autiasp 40018: d65f03c0 ret Disassembly of section .plt: 0000000000040020 <__asan_report_store4@plt-0x20>: 40020: d503245f bti c 40024: a9bf7bf0 stp x16, x30, [sp, #-16]! 40028: f0000ad0 adrp x16, 19b000 <__asan_cplus_demangle_builtin_types@@Base+0x460> 4002c: f9451a11 ldr x17, [x16, #2608] 40030: 9128c210 add x16, x16, #0xa30 40034: d61f0220 br x17 40038: d503201f nop 4003c: d503201f nop 0000000000040040 <__asan_report_store4@plt>: 40040: f0000ad0 adrp x16, 19b000 <__asan_cplus_demangle_builtin_types@@Base+0x460> 40044: f9451e11 ldr x17, [x16, #2616] 40048: 9128e210 add x16, x16, #0xa38 4004c: d61f0220 br x17 ... 0000000000040690 <__interceptor_memset@plt>: 40690: f0000ad0 adrp x16, 19b000 <__asan_cplus_demangle_builtin_types@@Base+0x460> 40694: f946b211 ldr x17, [x16, #3424] 40698: 91358210 add x16, x16, #0xd60 4069c: d61f0220 br x17 Reproducible: Always Steps to Reproduce: 1. dnf install libasan 2. LD_PRELOAD=/usr/lib64/libasan.so.8 ls Actual Results: Illegal instruction (core dumped) Expected Results: Output from ls
This issue has been reproduced on Snapdragon X1 and QEMU. Looking at other binaries, their PLT stubs also do not have BTI/PAC instructions and they run well on the same processors. So, there is a chance this is not related to Binutils.
It is also reproduced on Apple M2+ (which support BTI/PAC).
I'm starting to believe my initial description of this bug is wrong. B is not an instruction that sets EPSR.B. The previous frame was executing: #1 0x00004000012fc070 in std::locale::_Impl::_Impl(unsigned long) () from /lib64/libstdc++.so.6 More specifically, it calls memset: 0x00004000012fc064 <+68>: mov w1, #0x0 // #0 0x00004000012fc068 <+72>: str x4, [x19, #16] 0x00004000012fc06c <+76>: bl 0x4000012c4200 <memset@plt> => 0x00004000012fc070 <+80>: str x0, [x19, #8] BL doesn't set EPSR.B either. I wonder if PLT code might be causing issues here. I'm trying to get access to real hardware in order to track this.
Hi Tulio, Which build of libasan is involved ? Looking at libasan-14.1.1-7.fc40.aarch64.rpm I see that the plt stubs do indeed appear to be missing the "bti c" instruction at their entry points. eg: 0000000000041070 <memset@plt>: 41070: f0000ad0 adrp x16, 19c000 <__interceptor_pthread_attr_getguardsize@@Base+0xd6460> 41074: f9412a11 ldr x17, [x16, #592] 41078: 91094210 add x16, x16, #0x250 4107c: d61f0220 br x17 I am not sure what is happening here, but I will ask ARM for help. Cheers Nick
I was able to reproduce it with libasan-14.2.1-1.fc40.aarch64
I've just found out a piece of inline asm that is used to generate __interceptor_trampoline_memset and that it does not have support for PACBTI: https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=libsanitizer/interception/interception.h;h=58e969378a90822ce727dc24af74944f225da189;hb=HEAD#l193 So, I'm reassigning this to GCC.
Replicated and can confirm that the ASM_TAIL_CALL macro is the b instruction which is not a valid landing pad.
I also can confirm that adding the proper landing pad there works, will clean up and upstream. Thanks Tullo, that triage work saved me a lot of time
So libsanitizer is out of the clang tree, they have it fixed here: https://github.com/llvm/llvm-project/pull/84061
FEDORA-2024-d47aa726a4 (gcc-14.2.1-3.fc41) has been submitted as an update to Fedora 41. https://bodhi.fedoraproject.org/updates/FEDORA-2024-d47aa726a4
FEDORA-2024-c69055a3b6 (gcc-14.2.1-3.fc40) has been submitted as an update to Fedora 40. https://bodhi.fedoraproject.org/updates/FEDORA-2024-c69055a3b6
FEDORA-2024-d47aa726a4 has been pushed to the Fedora 41 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2024-d47aa726a4` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2024-d47aa726a4 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2024-c69055a3b6 has been pushed to the Fedora 40 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2024-c69055a3b6` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2024-c69055a3b6 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2024-d47aa726a4 (gcc-14.2.1-3.fc41) has been pushed to the Fedora 41 stable repository. If problem still persists, please make note of it in this bug report.
FEDORA-2024-c69055a3b6 (gcc-14.2.1-3.fc40) has been pushed to the Fedora 40 stable repository. If problem still persists, please make note of it in this bug report.