Cannot compile and run a short stp script RPMs: systemtap-4.9-2.fc39.x86_64 gcc-13.1.1-4.fc39.x86_64 glibc-2.37.9000-13.fc39.x86_64 kernel-6.4.0-0.rc6.20230616git40f71e7cd3c6.50.fc39.x86_64 kernel-core-6.4.0-0.rc6.20230616git40f71e7cd3c6.50.fc39.x86_64 kernel-debuginfo-6.4.0-0.rc6.20230616git40f71e7cd3c6.50.fc39.x86_64 kernel-debuginfo-common-x86_64-6.4.0-0.rc6.20230616git40f71e7cd3c6.50.fc39.x86_64 kernel-devel-6.4.0-0.rc6.20230616git40f71e7cd3c6.50.fc39.x86_64 kernel-modules-6.4.0-0.rc6.20230616git40f71e7cd3c6.50.fc39.x86_64 kernel-modules-core-6.4.0-0.rc6.20230616git40f71e7cd3c6.50.fc39.x86_64 Reproducible: Always Steps to Reproduce: 1. cat skip.stp global f probe timer.profile { f++; snooze() } function snooze() %{ udelay(10000); %} 2. stap skip.stp --disable-cache -vtug Actual Results: Pass 1: parsed user script and 486 library scripts using 137752virt/107056res/15488shr/90856data kb, in 160usr/20sys/185real ms. Pass 2: analyzed script: 2 probes, 1 function, 0 embeds, 1 global using 139204virt/109744res/16640shr/92308data kb, in 10usr/10sys/17real ms. Pass 3: translated to C into "/tmp/stapdHVyOH/stap_746_src.c" using 139340virt/110128res/17024shr/92444data kb, in 0usr/0sys/3real ms. In file included from /usr/share/systemtap/runtime/linux/runtime.h:288, from /usr/share/systemtap/runtime/runtime.h:26, from /tmp/stapdHVyOH/stap_746_src.c:22: /usr/share/systemtap/runtime/sym.c:1159:5: error: conflicting types for ‘kallsyms_on_each_symbol’; have ‘int(int (*)(void *, const char *, struct module *, long unsigned int), void *)’ 1159 | int kallsyms_on_each_symbol(int (*fn)(void *, const char *, struct module *, | ^~~~~~~~~~~~~~~~~~~~~~~ In file included from ./include/linux/ftrace.h:13, from ./include/linux/kprobes.h:28, from /usr/share/systemtap/runtime/linux/runtime.h:21: ./include/linux/kallsyms.h:70:5: note: previous declaration of ‘kallsyms_on_each_symbol’ with type ‘int(int (*)(void *, const char *, long unsigned int), void *)’ 70 | int kallsyms_on_each_symbol(int (*fn)(void *, const char *, unsigned long), | ^~~~~~~~~~~~~~~~~~~~~~~ /usr/share/systemtap/runtime/sym.c: In function ‘kallsyms_on_each_symbol’: /usr/share/systemtap/runtime/sym.c:1166:85: error: passing argument 1 of ‘(int (*)(int (*)(void *, const char *, long unsigned int), void *))_stp_kallsyms_on_each_symbol’ from incompatible pointer type [-Werror=incompatible-pointer-types] 1166 | return (* (kallsyms_on_each_symbol_fn)_stp_kallsyms_on_each_symbol)(fn, data); | ^~ | | | int (*)(void *, const char *, struct module *, long unsigned int) /usr/share/systemtap/runtime/sym.c:1166:85: note: expected ‘int (*)(void *, const char *, long unsigned int)’ but argument is of type ‘int (*)(void *, const char *, struct module *, long unsigned int)’ In file included from ./include/linux/kernel.h:30, from ./arch/x86/include/asm/percpu.h:27, from ./arch/x86/include/asm/preempt.h:6, from ./include/linux/preempt.h:78, from ./include/linux/spinlock.h:56, from ./include/linux/mmzone.h:8, from ./include/linux/gfp.h:7, from /usr/share/systemtap/runtime/linux/runtime_defines.h:20, from /usr/share/systemtap/runtime/runtime_defines.h:8, from /tmp/stapdHVyOH/stap_746_src.c:12: /usr/share/systemtap/runtime/linux/print.c: In function ‘_stp_print_kernel_info’: /usr/share/systemtap/runtime/linux/print.c:365:43: error: ‘struct module’ has no member named ‘module_core’ 365 | (unsigned long) THIS_MODULE->module_core, | ^~ ./include/linux/printk.h:427:33: note: in definition of macro ‘printk_index_wrap’ 427 | _p_func(_fmt, ##__VA_ARGS__); \ | ^~~~~~~~~~~ /usr/share/systemtap/runtime/linux/print.c:348:9: note: in expansion of macro ‘printk’ 348 | printk(KERN_DEBUG | ^~~~~~ /usr/share/systemtap/runtime/linux/print.c:366:44: error: ‘struct module’ has no member named ‘core_size’ 366 | (unsigned long) (THIS_MODULE->core_size - THIS_MODULE->core_text_size)/1024, | ^~ ./include/linux/printk.h:427:33: note: in definition of macro ‘printk_index_wrap’ 427 | _p_func(_fmt, ##__VA_ARGS__); \ | ^~~~~~~~~~~ /usr/share/systemtap/runtime/linux/print.c:348:9: note: in expansion of macro ‘printk’ 348 | printk(KERN_DEBUG | ^~~~~~ /usr/share/systemtap/runtime/linux/print.c:366:71: error: ‘struct module’ has no member named ‘core_text_size’; did you mean ‘kprobes_text_size’? 366 | (unsigned long) (THIS_MODULE->core_size - THIS_MODULE->core_text_size)/1024, | ^~~~~~~~~~~~~~ ./include/linux/printk.h:427:33: note: in definition of macro ‘printk_index_wrap’ 427 | _p_func(_fmt, ##__VA_ARGS__); \ | ^~~~~~~~~~~ /usr/share/systemtap/runtime/linux/print.c:348:9: note: in expansion of macro ‘printk’ 348 | printk(KERN_DEBUG | ^~~~~~ /usr/share/systemtap/runtime/linux/print.c:367:46: error: ‘struct module’ has no member named ‘core_text_size’; did you mean ‘kprobes_text_size’? 367 | (unsigned long) (THIS_MODULE->core_text_size)/1024, | ^~~~~~~~~~~~~~ ./include/linux/printk.h:427:33: note: in definition of macro ‘printk_index_wrap’ 427 | _p_func(_fmt, ##__VA_ARGS__); \ | ^~~~~~~~~~~ /usr/share/systemtap/runtime/linux/print.c:348:9: note: in expansion of macro ‘printk’ 348 | printk(KERN_DEBUG | ^~~~~~ cc1: all warnings being treated as errors make[1]: *** [scripts/Makefile.build:252: /tmp/stapdHVyOH/stap_746_src.o] Error 1 make: *** [Makefile:2044: /tmp/stapdHVyOH] Error 2 WARNING: kbuild exited with status: 2 Pass 4: compiled C into "stap_746.ko" in 17860usr/2550sys/11541real ms. Pass 4: compilation failed. [man error::pass4]
This is due to changes in the kernel's livepatch support code. There are also some changes in module struct in newer kernels. There are some upstream changes in systemtap to address this specific issue. Following systemtap git commits should address running systemtap on linux 6.4 kernels: 33fae2d0107fb6166b4eac3fdffd277829849ab0 5251b3060790faafa9f94c14801baaa76a2bf8ea fc6519089d3f9366470ce442b648d69ed9b56f53 56054abb4efb3ef95808306b2f22339ab5c96352
The current systemtap-5.0~pre16891249ge891a37e-0.2.fc39.x86_64 in Fedora rawhide builds instrumentation for the linux-6.5 kernels: $ uname -a; rpm -q systemtap Linux rawhide 6.5.0-0.rc5.20230809git13b937206866.38.fc40.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Aug 9 16:40:06 UTC 2023 x86_64 GNU/Linux systemtap-5.0~pre16891249ge891a37e-0.2.fc39.x86_64 $ sudo stap skip.stp --disable-cache -vtug [sudo] password for wcohen: Pass 1: parsed user script and 541 library scripts using 543544virt/297264res/16000shr/280616data kb, in 630usr/90sys/1439real ms. Pass 2: analyzed script: 2 probes, 1 function, 0 embeds, 1 global using 556348virt/311216res/17024shr/293420data kb, in 60usr/0sys/130real ms. Pass 3: translated to C into "/tmp/stapN5Luvp/stap_71645_src.c" using 556348virt/311600res/17408shr/293420data kb, in 0usr/0sys/0real ms. /tmp/stapN5Luvp/stap_71645.o: warning: objtool: _stp_vsprint_memory+0x1e9: call to __get_user_nocheck_1() with UACCESS enabled Pass 4: compiled C into "stap_71645.ko" in 19690usr/2340sys/45132real ms. Pass 5: starting run. ERROR: probe overhead (926703704 cycles) exceeded threshold (500000000 cycles) in last 1000000000 cycles ----- probe hit report: perf.type(1).config(0).sample(1000000), (/usr/share/systemtap/tapset/timers.stp:21:46), hits: 34, cycles: 28081515min/28081935avg/28083490max, variance: 28734, from: perf.type(1).config(0).sample(1000000) from: perf.sw.cpu_clock.sample(1000000) from: timer.profile from: timer.profile, index: 0 ----- refresh report: WARNING: Number of errors: 1, skipped probes: 0 WARNING: /usr/bin/staprun exited with status: 1 Pass 5: run completed in 10usr/20sys/707real ms. Pass 5: run failed. [man error::pass5]