Bug 2215975

Summary: mismatched declaration of kallsyms_on_each_symbol
Product: [Fedora] Fedora Reporter: Václav Kadlčík <vkadlcik>
Component: systemtapAssignee: Frank Ch. Eigler <fche>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: rawhideCC: amerey, fche, mcermak, mjw, scox, wcohen
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-08-10 18:14:57 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Václav Kadlčík 2023-06-19 14:29:38 UTC
Cannot compile and run a short stp script

RPMs:
systemtap-4.9-2.fc39.x86_64
gcc-13.1.1-4.fc39.x86_64
glibc-2.37.9000-13.fc39.x86_64
kernel-6.4.0-0.rc6.20230616git40f71e7cd3c6.50.fc39.x86_64
kernel-core-6.4.0-0.rc6.20230616git40f71e7cd3c6.50.fc39.x86_64
kernel-debuginfo-6.4.0-0.rc6.20230616git40f71e7cd3c6.50.fc39.x86_64
kernel-debuginfo-common-x86_64-6.4.0-0.rc6.20230616git40f71e7cd3c6.50.fc39.x86_64
kernel-devel-6.4.0-0.rc6.20230616git40f71e7cd3c6.50.fc39.x86_64
kernel-modules-6.4.0-0.rc6.20230616git40f71e7cd3c6.50.fc39.x86_64
kernel-modules-core-6.4.0-0.rc6.20230616git40f71e7cd3c6.50.fc39.x86_64


Reproducible: Always

Steps to Reproduce:
1. cat skip.stp
global f
probe timer.profile { f++; snooze() }
function snooze() %{ udelay(10000); %}
   
2. stap skip.stp --disable-cache -vtug
Actual Results:  
Pass 1: parsed user script and 486 library scripts using 137752virt/107056res/15488shr/90856data kb, in 160usr/20sys/185real ms.
Pass 2: analyzed script: 2 probes, 1 function, 0 embeds, 1 global using 139204virt/109744res/16640shr/92308data kb, in 10usr/10sys/17real ms.
Pass 3: translated to C into "/tmp/stapdHVyOH/stap_746_src.c" using 139340virt/110128res/17024shr/92444data kb, in 0usr/0sys/3real ms.
In file included from /usr/share/systemtap/runtime/linux/runtime.h:288,
                 from /usr/share/systemtap/runtime/runtime.h:26,
                 from /tmp/stapdHVyOH/stap_746_src.c:22:
/usr/share/systemtap/runtime/sym.c:1159:5: error: conflicting types for ‘kallsyms_on_each_symbol’; have ‘int(int (*)(void *, const char *, struct module *, long unsigned int), void *)’
 1159 | int kallsyms_on_each_symbol(int (*fn)(void *, const char *, struct module *,
      |     ^~~~~~~~~~~~~~~~~~~~~~~
In file included from ./include/linux/ftrace.h:13,
                 from ./include/linux/kprobes.h:28,
                 from /usr/share/systemtap/runtime/linux/runtime.h:21:
./include/linux/kallsyms.h:70:5: note: previous declaration of ‘kallsyms_on_each_symbol’ with type ‘int(int (*)(void *, const char *, long unsigned int), void *)’
   70 | int kallsyms_on_each_symbol(int (*fn)(void *, const char *, unsigned long),
      |     ^~~~~~~~~~~~~~~~~~~~~~~
/usr/share/systemtap/runtime/sym.c: In function ‘kallsyms_on_each_symbol’:
/usr/share/systemtap/runtime/sym.c:1166:85: error: passing argument 1 of ‘(int (*)(int (*)(void *, const char *, long unsigned int), void *))_stp_kallsyms_on_each_symbol’ from incompatible pointer type [-Werror=incompatible-pointer-types]
 1166 |                 return (* (kallsyms_on_each_symbol_fn)_stp_kallsyms_on_each_symbol)(fn, data);
      |                                                                                     ^~
      |                                                                                     |
      |                                                                                     int (*)(void *, const char *, struct module *, long unsigned int)
/usr/share/systemtap/runtime/sym.c:1166:85: note: expected ‘int (*)(void *, const char *, long unsigned int)’ but argument is of type ‘int (*)(void *, const char *, struct module *, long unsigned int)’
In file included from ./include/linux/kernel.h:30,
                 from ./arch/x86/include/asm/percpu.h:27,
                 from ./arch/x86/include/asm/preempt.h:6,
                 from ./include/linux/preempt.h:78,
                 from ./include/linux/spinlock.h:56,
                 from ./include/linux/mmzone.h:8,
                 from ./include/linux/gfp.h:7,
                 from /usr/share/systemtap/runtime/linux/runtime_defines.h:20,
                 from /usr/share/systemtap/runtime/runtime_defines.h:8,
                 from /tmp/stapdHVyOH/stap_746_src.c:12:
/usr/share/systemtap/runtime/linux/print.c: In function ‘_stp_print_kernel_info’:
/usr/share/systemtap/runtime/linux/print.c:365:43: error: ‘struct module’ has no member named ‘module_core’
  365 |                (unsigned long) THIS_MODULE->module_core,
      |                                           ^~
./include/linux/printk.h:427:33: note: in definition of macro ‘printk_index_wrap’
  427 |                 _p_func(_fmt, ##__VA_ARGS__);                           \
      |                                 ^~~~~~~~~~~
/usr/share/systemtap/runtime/linux/print.c:348:9: note: in expansion of macro ‘printk’
  348 |         printk(KERN_DEBUG
      |         ^~~~~~
/usr/share/systemtap/runtime/linux/print.c:366:44: error: ‘struct module’ has no member named ‘core_size’
  366 |                (unsigned long) (THIS_MODULE->core_size - THIS_MODULE->core_text_size)/1024,
      |                                            ^~
./include/linux/printk.h:427:33: note: in definition of macro ‘printk_index_wrap’
  427 |                 _p_func(_fmt, ##__VA_ARGS__);                           \
      |                                 ^~~~~~~~~~~
/usr/share/systemtap/runtime/linux/print.c:348:9: note: in expansion of macro ‘printk’
  348 |         printk(KERN_DEBUG
      |         ^~~~~~
/usr/share/systemtap/runtime/linux/print.c:366:71: error: ‘struct module’ has no member named ‘core_text_size’; did you mean ‘kprobes_text_size’?
  366 |                (unsigned long) (THIS_MODULE->core_size - THIS_MODULE->core_text_size)/1024,
      |                                                                       ^~~~~~~~~~~~~~
./include/linux/printk.h:427:33: note: in definition of macro ‘printk_index_wrap’
  427 |                 _p_func(_fmt, ##__VA_ARGS__);                           \
      |                                 ^~~~~~~~~~~
/usr/share/systemtap/runtime/linux/print.c:348:9: note: in expansion of macro ‘printk’
  348 |         printk(KERN_DEBUG
      |         ^~~~~~
/usr/share/systemtap/runtime/linux/print.c:367:46: error: ‘struct module’ has no member named ‘core_text_size’; did you mean ‘kprobes_text_size’?
  367 |                (unsigned long) (THIS_MODULE->core_text_size)/1024,
      |                                              ^~~~~~~~~~~~~~
./include/linux/printk.h:427:33: note: in definition of macro ‘printk_index_wrap’
  427 |                 _p_func(_fmt, ##__VA_ARGS__);                           \
      |                                 ^~~~~~~~~~~
/usr/share/systemtap/runtime/linux/print.c:348:9: note: in expansion of macro ‘printk’
  348 |         printk(KERN_DEBUG
      |         ^~~~~~
cc1: all warnings being treated as errors
make[1]: *** [scripts/Makefile.build:252: /tmp/stapdHVyOH/stap_746_src.o] Error 1
make: *** [Makefile:2044: /tmp/stapdHVyOH] Error 2
WARNING: kbuild exited with status: 2
Pass 4: compiled C into "stap_746.ko" in 17860usr/2550sys/11541real ms.
Pass 4: compilation failed.  [man error::pass4]

Comment 1 William Cohen 2023-06-20 16:39:08 UTC
This is due to changes in the kernel's livepatch support code.  There are also some changes in module struct in newer kernels.  There are some upstream changes in systemtap to address this specific issue. Following systemtap git commits should address running systemtap on linux 6.4 kernels:

33fae2d0107fb6166b4eac3fdffd277829849ab0
5251b3060790faafa9f94c14801baaa76a2bf8ea
fc6519089d3f9366470ce442b648d69ed9b56f53
56054abb4efb3ef95808306b2f22339ab5c96352

Comment 2 William Cohen 2023-08-10 18:14:57 UTC
The current systemtap-5.0~pre16891249ge891a37e-0.2.fc39.x86_64 in Fedora rawhide builds instrumentation for the linux-6.5 kernels:

$ uname -a; rpm -q systemtap
Linux rawhide 6.5.0-0.rc5.20230809git13b937206866.38.fc40.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Aug  9 16:40:06 UTC 2023 x86_64 GNU/Linux
systemtap-5.0~pre16891249ge891a37e-0.2.fc39.x86_64
$ sudo stap skip.stp --disable-cache -vtug
[sudo] password for wcohen: 
Pass 1: parsed user script and 541 library scripts using 543544virt/297264res/16000shr/280616data kb, in 630usr/90sys/1439real ms.
Pass 2: analyzed script: 2 probes, 1 function, 0 embeds, 1 global using 556348virt/311216res/17024shr/293420data kb, in 60usr/0sys/130real ms.
Pass 3: translated to C into "/tmp/stapN5Luvp/stap_71645_src.c" using 556348virt/311600res/17408shr/293420data kb, in 0usr/0sys/0real ms.
/tmp/stapN5Luvp/stap_71645.o: warning: objtool: _stp_vsprint_memory+0x1e9: call to __get_user_nocheck_1() with UACCESS enabled
Pass 4: compiled C into "stap_71645.ko" in 19690usr/2340sys/45132real ms.
Pass 5: starting run.
ERROR: probe overhead (926703704 cycles) exceeded threshold (500000000 cycles) in last 1000000000 cycles
----- probe hit report: 
perf.type(1).config(0).sample(1000000), (/usr/share/systemtap/tapset/timers.stp:21:46), hits: 34, cycles: 28081515min/28081935avg/28083490max, variance: 28734, from: perf.type(1).config(0).sample(1000000) from: perf.sw.cpu_clock.sample(1000000) from: timer.profile from: timer.profile, index: 0
----- refresh report:
WARNING: Number of errors: 1, skipped probes: 0
WARNING: /usr/bin/staprun exited with status: 1
Pass 5: run completed in 10usr/20sys/707real ms.
Pass 5: run failed.  [man error::pass5]