Description of problem: Libvirt uses systemtaps' dtrace compatibility tool to generate the probe header file definitions based on a dtrace provider script. When we compile Libvirt with CLang the dtrace / systemtap probes cause a slew of compiler errors. I'm unclear whether the fault is with systemtap generating non-portable code, or with clang not honouring some part of the C spec. Picking systemtap to file the bug though, since even if clang doesn't honour the C spec in this area, I think it is still desirable for systemtap to generate working code for clang. The following steps demonstrate the problem on Fedora 19 x86_64 $ cat sdt.c #include <sys/sdt.h> #include "sdt.h" int foo(void) { DEMO_FOO(); return 0; } int main(void) { return foo(); } $ cat sdt.d provider demo { probe foo(); }; $ dtrace -o sdt.h -h -s sdt.d $ clang -o sdt sdt.c sdt.c:7:3: error: unknown flag DEMO_FOO(); ^ ./sdt.h:19:20: note: expanded from macro 'DEMO_FOO' #define DEMO_FOO() \ ^ /usr/include/sys/sdt.h:362:3: note: expanded from macro '\ DTRACE_PROBE' STAP_PROBE(provider,probe) ^ /usr/include/sys/sdt.h:254:3: note: expanded from macro 'STAP_PROBE' _SDT_PROBE(provider, name, 0, ()) ^ note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all) /usr/include/sys/sdt.h:43:31: note: expanded from macro '_SDT_ASM_3' # define _SDT_ASM_3(a, b, c) _SDT_S(a) "," _SDT_S(b) "," \ ^ /usr/include/sys/sdt.h:40:22: note: expanded from macro '_SDT_S' # define _SDT_S(x) #x ^ <scratch space>:4:2: note: expanded from here ".pushsection .note.stapsdt" ^ <inline asm>:2:31: note: instantiated into assembly here .pushsection .note.stapsdt,"?","note" ^ sdt.c:7:3: error: .popsection without corresponding .pushsection DEMO_FOO(); ^ ./sdt.h:19:20: note: expanded from macro 'DEMO_FOO' #define DEMO_FOO() \ ^ /usr/include/sys/sdt.h:362:3: note: expanded from macro '\ DTRACE_PROBE' STAP_PROBE(provider,probe) ^ /usr/include/sys/sdt.h:254:3: note: expanded from macro 'STAP_PROBE' _SDT_PROBE(provider, name, 0, ()) ^ note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all) /usr/include/sys/sdt.h:41:26: note: expanded from macro '_SDT_ASM_1' # define _SDT_ASM_1(x) _SDT_S(x) "\n" ^ /usr/include/sys/sdt.h:40:22: note: expanded from macro '_SDT_S' # define _SDT_S(x) #x ^ <scratch space>:24:2: note: expanded from here ".popsection" ^ <inline asm>:14:12: note: instantiated into assembly here .popsection ^ sdt.c:7:3: error: expected '@' or '%' before type DEMO_FOO(); ^ ./sdt.h:19:20: note: expanded from macro 'DEMO_FOO' #define DEMO_FOO() \ ^ /usr/include/sys/sdt.h:362:3: note: expanded from macro '\ DTRACE_PROBE' STAP_PROBE(provider,probe) ^ /usr/include/sys/sdt.h:254:3: note: expanded from macro 'STAP_PROBE' _SDT_PROBE(provider, name, 0, ()) ^ note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all) /usr/include/sys/sdt.h:45:36: note: expanded from macro '_SDT_ASM_5' # define _SDT_ASM_5(a, b, c, d, e) _SDT_S(a) "," _SDT_S(b) "," \ ^ /usr/include/sys/sdt.h:40:22: note: expanded from macro '_SDT_S' # define _SDT_S(x) #x ^ <scratch space>:27:2: note: expanded from here ".pushsection .stapsdt.base" ^ <inline asm>:2:33: note: instantiated into assembly here .pushsection .stapsdt.base,"aG","progbits",.stapsdt.base,comdat ^ sdt.c:7:3: error: .popsection without corresponding .pushsection DEMO_FOO(); ^ ./sdt.h:19:20: note: expanded from macro 'DEMO_FOO' #define DEMO_FOO() \ ^ /usr/include/sys/sdt.h:362:3: note: expanded from macro '\ DTRACE_PROBE' STAP_PROBE(provider,probe) ^ /usr/include/sys/sdt.h:254:3: note: expanded from macro 'STAP_PROBE' _SDT_PROBE(provider, name, 0, ()) ^ note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all) /usr/include/sys/sdt.h:41:26: note: expanded from macro '_SDT_ASM_1' # define _SDT_ASM_1(x) _SDT_S(x) "\n" ^ /usr/include/sys/sdt.h:40:22: note: expanded from macro '_SDT_S' # define _SDT_S(x) #x ^ <scratch space>:37:2: note: expanded from here ".popsection" ^ <inline asm>:7:12: note: instantiated into assembly here .popsection ^ 4 errors generated. Gcc by comparison works as expected $ gcc -o sdt sdt.c ...no errors... Systemtap support is the only remaining blocker to building a fully featured libvirt with clang. Version-Release number of selected component (if applicable): systemtap-sdt-devel-2.3-1.fc19.x86_64 clang-3.3-0.6.rc3.fc19.x86_64
I think that is just clang not properly supporting the inline assembly directives in /usr/include/sys/sdt.h. .pushsection and .popsection a gas directive [*]. It is needed to put the actual probes in the correct ELF section where gdb or stap can pick them up. Does clang use gas as assembler by default or something else? [*] http://sourceware.org/binutils/docs/as/PushSection.html It is probably easiest to write a little configure script to check the compiler can at least compile SDT probes and if not disable them.
This bug report suggests that clang should be supporting pushsection/popsection in asm blocks by now http://llvm.org/bugs/show_bug.cgi?id=8633
So there seem to be other deficiencies in the clang assembler then. I did find the following bug report: http://sourceware.org/bugzilla/show_bug.cgi?id=13974 One problem is that clang apparently doesn't use the system assembler (gas) by default, so the installed sdt-config.h doesn't match which capabilities the system assembler has. But even with that fixed it looks like there are other troubles with using the clang/llvm assembler. Is there a way for clang to just use the system installed gas?
See also http://sourceware.org/bugzilla/show_bug.cgi?id=13974 I can't think of a way we can work around these clang<->gcc incompatibilities within the sys/sdt.h file. Redirecting to clang for their consideration.
FYI - we did have a little movement on stap PR13974, but ended up reverting as the attempted change caused other issues. However, llvm-mc has gained support for both of the missing constructs: http://llvm.org/bugs/show_bug.cgi?id=17198 http://llvm.org/bugs/show_bug.cgi?id=17270
Dan, please consider reassigning this against llvm, if their upstream patches to implement double-quoted "foo" section-attributes may be backported to f19.
Works with clang-3.4-10.fc20.x86_64 and systemtap-sdt-devel-2.6-2.fc20.x86_64. I'm closing this report as we hardly have resources to fix bugs in current Fedora releases. If this is really needed in F19, feel free to reopen this.