Bug 1639397
| Summary: | 32-bit arm kernel arch_prepare_optimized_kprobe panics due to FORTIFY_SOURCE check for memcpy | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | William Cohen <wcohen> |
| Component: | kernel | Assignee: | Kernel Maintainer List <kernel-maint> |
| Status: | CLOSED INSUFFICIENT_DATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 28 | CC: | airlied, bskeggs, ewk, hdegoede, hongzhi.song, ichavero, itamar, jarodwilson, jglisse, john.j5live, jonathan, josef, kees, kernel-maint, linville, mchehab, mjg59, steved |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | arm | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-02-21 21:13:39 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
This can be replicated without systemtap by running the following perf command as root: perf probe -a kernel_vfs_write_return=vfs_write%return This appears to be a false positive, due to the arm compiler treating inline-asm global labels as unsigned long _values_ instead of an unsigned char *. This fixes it for me:
diff --git a/arch/arm/probes/kprobes/opt-arm.c b/arch/arm/probes/kprobes/opt-arm.c
index b2aa9b32bff2..2c118a6ab358 100644
--- a/arch/arm/probes/kprobes/opt-arm.c
+++ b/arch/arm/probes/kprobes/opt-arm.c
@@ -247,7 +247,7 @@ int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, struct kprobe *or
}
/* Copy arch-dep-instance from template. */
- memcpy(code, &optprobe_template_entry,
+ memcpy(code, (unsigned char *)optprobe_template_entry,
TMPL_END_IDX * sizeof(kprobe_opcode_t));
/* Adjust buffer according to instruction. */
(In reply to Kees Cook from comment #2) > This appears to be a false positive, due to the arm compiler treating > inline-asm global labels as unsigned long _values_ instead of an unsigned > char *. This fixes it for me: > > diff --git a/arch/arm/probes/kprobes/opt-arm.c > b/arch/arm/probes/kprobes/opt-arm.c > index b2aa9b32bff2..2c118a6ab358 100644 > --- a/arch/arm/probes/kprobes/opt-arm.c > +++ b/arch/arm/probes/kprobes/opt-arm.c > @@ -247,7 +247,7 @@ int arch_prepare_optimized_kprobe(struct > optimized_kprobe *op, struct kprobe *or > } > > /* Copy arch-dep-instance from template. */ > - memcpy(code, &optprobe_template_entry, > + memcpy(code, (unsigned char *)optprobe_template_entry, > TMPL_END_IDX * sizeof(kprobe_opcode_t)); > > /* Adjust buffer according to instruction. */ I find a new problem introduced by this way. Issue: register_kprobe() will fail on arm. Reproduce: 1. Machine: ARM 2. CONFIG_SAMPLES=y CONFIG_SAMPLE_KPROBES=m build an arm kernel image with above patch make ARCH=arm CROSS_COMPILE=arm-gcc 3. build kprobe_example.c under kernel-source/samples/kprobe/ make modules 4. run arm kernel with qemu 5. modprobe kprobe_example Then the console will prompt: Unable to handle kernel paging request at virtual address e24dd0fc [ 102.687559] pgd = 109ab7a1 [ 102.688274] [e24dd0fc] *pgd=00000000 [ 102.690136] Internal error: Oops: 5 [#1] PREEMPT ARM [ 102.691848] Modules linked in: kprobe_example(+) ... [ 103.075173] Backtrace: [ 103.081256] [<c001eaf8>] (arch_prepare_optimized_kprobe) from [<c00b0ad8>] (alloc_aggr_kprobe+0x5c/0x6c) [ 103.092807] r9:00000002 r8:bf0001e0 r7:00000000 r6:00000000 r5:bf0001d8 r4:ce8c3a80 [ 103.104176] [<c00b0a7c>] (alloc_aggr_kprobe) from [<c00b058c>] (register_kprobe+0x4a0/0x53c) [ 103.115408] r5:00000000 r4:bf0001d8 [ 103.121722] [<c00b00ec>] (register_kprobe) from [<bf002038>] (kprobe_init+0x38/0xc0 [kprobe_example]) [ 103.132876] r9:00000002 r8:c0b50920 r7:00000000 r6:bf002000 r5:c0abf048 r4:bf0001d8 [ 103.143766] [<bf002000>] (kprobe_init [kprobe_example]) from [<c000b5b4>] (do_one_initcall+0xc8/0x20c) [ 103.154720] r5:c0abf048 r4:ffffe000 *********** MASS BUG UPDATE ************** We apologize for the inconvenience. There are a large number of bugs to go through and several of them have gone stale. Due to this, we are doing a mass bug update across all of the Fedora 28 kernel bugs. Fedora 28 has now been rebased to 4.20.5-100.fc28. Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel. If you have moved on to Fedora 29, and are still experiencing this issue, please change the version to Fedora 29. If you experience different issues, please open a new bug report for those. *********** MASS BUG UPDATE ************** This bug is being closed with INSUFFICIENT_DATA as there has not been a response in 3 weeks. If you are still experiencing this issue, please reopen and attach the relevant data from the latest kernel you are running and any data that might have been requested previously. |
Description of problem: On 32-bit arm when using the the kernels kretprobes the arm kernel panic due to a fortify check. This problem was observed using systemtap, but should also be reproducible using a similar "perf record" command. Version-Release number of selected component (if applicable): kernel-4.18.12-200.fc28.armv7hl How reproducible: always Steps to Reproduce: 1. As root install systemtap, kernel-devel, kernel-debuginfo dnf install systemtap-testsuite kernel-devel dnf debuginfo-install -y kernel-$(uname -r) 2. run the systemtap reproducer as root; stap -ve 'probe kernel.function("vfs_*").return {print("probe fired\n"); exit();}' Actual results: [root@calxeda-arm-soc-04 systemtap]# stap -ve 'probe kernel.function("vfs_*").return {print("probe fired\n"); exit();}' Pass 1: parsed user script and 456 library scripts using 52600virt/36748res/7400shr/29400data kb, in 1040usr/120sys/1174real ms. WARNING: function vfs_caches_init_early is in blacklisted section: keyword at <input>:1:1 source: probe kernel.function("vfs_*").return {print("probe fired\n"); exit();} ^ WARNING: function vfs_caches_init is in blacklisted section: keyword at :1:1 source: probe kernel.function("vfs_*").return {print("probe fired\n"); exit();} ^ Pass 2: analyzed script: 53 probes, 1 function, 0 embeds, 0 globals using 91636virt/76692res/8220shr/68436data kb, in 4910usr/740sys/5756real ms. Pass 3: translated to C into "/tmp/stapfwtbuG/stap_8743d71b2554ca6b7eae2a0ad83bf96f_11834_src.c" using 91636virt/77008res/8536shr/68436data kb, in 130usr/610sys/758real ms. Pass 4: compiled C into "stap_8743d71b2554ca6b7eae2a0ad83bf96f_11834.ko" in 55660usr/8830sys/65541real ms. Pass 5: starting run. [ 3200.659438] stap_8743d71b2554ca6b7eae2a0ad83bf96f__4877: loading out-of-tree module taints kernel. [ 3200.668589] stap_8743d71b2554ca6b7eae2a0ad83bf96f__4877: module verification failed: signature and/or required key missing - tainting kernel [ 3200.742398] detected buffer overflow in memcpy [ 3200.747009] ------------[ cut here ]------------ [ 3200.751622] kernel BUG at lib/string.c:1052! [ 3200.755886] Internal error: Oops - BUG: 0 [#1] SMP ARM [ 3200.761017] Modules linked in: stap_8743d71b2554ca6b7eae2a0ad83bf96f__4877(OE) sunrpc ipmi_si ipmi_devintf ipmi_msghandler highbank_l2_edac xgmac highbank_mc_edac xfs libcrc32c sata_highbank [ 3200.777988] CPU: 3 PID: 4877 Comm: stapio Tainted: G W OE 4.18.12-200.fc28.armv7hl #1 [ 3200.786850] Hardware name: Highbank [ 3200.790343] PC is at fortify_panic+0x18/0x20 [ 3200.794609] LR is at irq_work_queue+0x4c/0x80 [ 3200.798959] pc : [<c0b51530>] lr : [<c044ac54>] psr: 60030013 [ 3200.805216] sp : e1da7d80 ip : e1da7ca0 fp : e1da7d8c [ 3200.810432] r10: bf28cb98 r9 : 00000000 r8 : e17e8800 [ 3200.815650] r7 : 00000060 r6 : bf290a68 r5 : c03248a8 r4 : bf29c000 [ 3200.822168] r3 : edbeec68 r2 : 2caab000 r1 : edbeb510 r0 : 00000022 [ 3200.828689] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none [ 3200.835817] Control: 10c5387d Table: 2319404a DAC: 00000051 [ 3200.841556] Process stapio (pid: 4877, stack limit = 0x83fda77f) [ 3200.847555] Stack: (0xe1da7d80 to 0xe1da8000) [ 3200.851908] 7d80: e1da7db4 e1da7d90 c0324a7c c0b51524 e17e8800 bf290a68 00000000 bf290a70 [ 3200.860081] 7da0: 00000000 00000000 e1da7dcc e1da7db8 c040b194 c03249f0 bf290a68 00000000 [ 3200.868254] 7dc0: e1da7dfc e1da7dd0 c040d3bc c040b144 00000000 bf290a68 00000018 006000c0 [ 3200.876426] 7de0: 00000000 c1204db4 0000000c bf28cb98 e1da7e1c e1da7e00 c040d608 c040ce7c [ 3200.884599] 7e00: bf28f044 bf290a68 bf2904a8 00000000 e1da7e34 e1da7e20 bf287aac c040d57c [ 3200.892771] 7e20: bf290180 bf28f044 e1da7ea4 e1da7e38 bf289e14 bf2879cc 00000028 0000001c [ 3200.900942] 7e40: 00000001 0000080f 00000052 00000035 c0382d48 00000002 00000001 bf28c080 [ 3200.909115] 7e60: e801a600 00000000 00000000 00000000 c4b38ec0 e24e5ee0 e1da7ea4 e41a46c0 [ 3200.917287] 7e80: bf289a34 e24e5ee0 bee95154 0000000c e1da7f70 00000004 e1da7ecc e1da7ea8 [ 3200.925460] 7ea0: c05f6e4c bf289a40 e41a46c0 c05f6df4 00000000 e1da7f70 0000000c e1da6000 [ 3200.933632] 7ec0: e1da7f3c e1da7ed0 c04ff778 c05f6e00 00000002 c1269e90 e41a46c0 00000000 [ 3200.941805] 7ee0: e1da7f70 e1da6000 e1da7f1c c0b57458 00000000 ed4ed988 e1da7f1c e1da7f08 [ 3200.949977] 7f00: c0b57458 c03c2a80 00000000 ed4ed988 e1da7f3c e1da7f20 c050176c e41a46c0 [ 3200.958149] 7f20: 0000000c 00000000 bee95154 e1da7f70 e1da7f6c e1da7f40 c04ffa5c c04ff748 [ 3200.966322] 7f40: e41a46c3 bee95154 e41a46c0 e41a46c3 bee95154 0000000c c0301204 e1da6000 [ 3200.974494] 7f60: e1da7f94 e1da7f70 c04ffc6c c04ff99c 00000000 00000000 0000000c bee95154 [ 3200.982665] 7f80: 00000004 00000004 e1da7fa4 e1da7f98 c04ffcbc c04ffc28 00000000 e1da7fa8 [ 3200.990837] 7fa0: c0301000 c04ffcb0 0000000c bee95154 00000004 bee95154 0000000c 00000000 [ 3200.999010] 7fc0: 0000000c bee95154 00000004 00000004 bee95154 00000000 0052da50 bee95778 [ 3201.007182] 7fe0: 00000000 bee95138 b6c8b4d0 b6eb9a88 80030010 00000004 00000000 00000000 [ 3201.015366] [<c0b51530>] (fortify_panic) from [<c0324a7c>] (arch_prepare_optimized_kprobe+0x98/0x1ec) [ 3201.024589] [<c0324a7c>] (arch_prepare_optimized_kprobe) from [<c040b194>] (alloc_aggr_kprobe+0x5c/0x6c) [ 3201.034069] [<c040b194>] (alloc_aggr_kprobe) from [<c040d3bc>] (register_kprobe+0x54c/0x5cc) [ 3201.042506] [<c040d3bc>] (register_kprobe) from [<c040d608>] (register_kretprobe+0x98/0x174) [ 3201.050966] [<c040d608>] (register_kretprobe) from [<bf287aac>] (stapkp_register_probe+0xec/0x174 [stap_8743d71b2554ca6b7eae2a0ad83bf96f__4877]) [ 3201.063949] [<bf287aac>] (stapkp_register_probe [stap_8743d71b2554ca6b7eae2a0ad83bf96f__4877]) from [<bf289e14>] (_stp_ctl_write_cmd+0x3e0/0xce0 [stap_8743d71b2554ca6b7eae2a0ad83bf96f__4877]) [ 3201.080995] [<bf289e14>] (_stp_ctl_write_cmd [stap_8743d71b2554ca6b7eae2a0ad83bf96f__4877]) from [<c05f6e4c>] (full_proxy_write+0x58/0x6c) [ 3201.093427] [<c05f6e4c>] (full_proxy_write) from [<c04ff778>] (__vfs_write+0x3c/0x138) [ 3201.101341] [<c04ff778>] (__vfs_write) from [<c04ffa5c>] (vfs_write+0xcc/0x1a4) [ 3201.108645] [<c04ffa5c>] (vfs_write) from [<c04ffc6c>] (ksys_write+0x50/0x88) [ 3201.115777] [<c04ffc6c>] (ksys_write) from [<c04ffcbc>] (sys_write+0x18/0x1c) [ 3201.122909] [<c04ffcbc>] (sys_write) from [<c0301000>] (ret_fast_syscall+0x0/0x54) [ 3201.130469] Exception stack(0xe1da7fa8 to 0xe1da7ff0) [ 3201.135516] 7fa0: 0000000c bee95154 00000004 bee95154 0000000c 00000000 [ 3201.143688] 7fc0: 0000000c bee95154 00000004 00000004 bee95154 00000000 0052da50 bee95778 [ 3201.151859] 7fe0: 00000000 bee95138 b6c8b4d0 b6eb9a88 [ 3201.156907] Code: e24cb004 e1a01000 e59f0004 ebe1809a (e7f001f2) [ 3201.162996] ---[ end trace b03cac47d3a6a84b ]--- Expected results: $ stap -ve 'probe kernel.function("vfs_*").return {print("probe fired\n"); exit();}' Pass 1: parsed user script and 545 library scripts using 422244virt/219676res/3520shr/218660data kb, in 1650usr/70sys/1721real ms. WARNING: function vfs_caches_init is in blacklisted section: keyword at <input>:1:1 source: probe kernel.function("vfs_*").return {print("probe fired\n"); exit();} ^ WARNING: function vfs_caches_init_early is in blacklisted section: keyword at :1:1 source: probe kernel.function("vfs_*").return {print("probe fired\n"); exit();} ^ Pass 2: analyzed script: 47 probes, 1 function, 0 embeds, 0 globals using 478520virt/276816res/4384shr/274936data kb, in 660usr/90sys/740real ms. Pass 3: translated to C into "/tmp/stapRoTC4N/stap_6fcb0cc693796dbd20b149c3d43ed94b_10492_src.c" using 478520virt/277168res/4736shr/274936data kb, in 20usr/50sys/62real ms. Pass 4: compiled C into "stap_6fcb0cc693796dbd20b149c3d43ed94b_10492.ko" in 1020usr/330sys/1233real ms. Pass 5: starting run. probe fired Pass 5: run completed in 10usr/520sys/854real ms. Additional info: