Bug 999419

Summary: Generated dtrace probes do not compile with clang
Product: [Fedora] Fedora Reporter: Daniel Berrangé <berrange>
Component: llvmAssignee: Adam Jackson <ajax>
Status: CLOSED NEXTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 19CC: ajax, berrange, bos, dmalcolm, dsmith, fche, jistone, jv+fedora, lberk, mbenitez, michel, mjw, mjw, nathans, petersen, scottt.tw, scox, wcohen
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-12-26 00:26:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Daniel Berrangé 2013-08-21 09:49:59 UTC
Description of problem:

Libvirt uses systemtaps' dtrace compatibility tool to generate the probe header file definitions based on a dtrace provider script.

When we compile Libvirt with CLang the dtrace / systemtap probes cause a slew of compiler errors.

I'm unclear whether the fault is with systemtap generating non-portable code, or with clang not honouring some part of the C spec. Picking systemtap to file the bug though, since even if clang doesn't honour the C spec in this area, I think it is still desirable for systemtap to generate working code for clang.

The following steps demonstrate the problem on Fedora 19 x86_64

$ cat sdt.c
#include <sys/sdt.h>

#include "sdt.h"

int foo(void)
{
  DEMO_FOO();
  return 0;
}


int main(void)
{
  return foo();
}


$ cat sdt.d

provider demo {
	 probe foo();
};


$ dtrace -o sdt.h -h  -s sdt.d
$ clang -o sdt sdt.c
sdt.c:7:3: error: unknown flag
  DEMO_FOO();
  ^
./sdt.h:19:20: note: expanded from macro 'DEMO_FOO'
#define DEMO_FOO() \
                   ^
/usr/include/sys/sdt.h:362:3: note: expanded from macro '\
DTRACE_PROBE'
  STAP_PROBE(provider,probe)
  ^
/usr/include/sys/sdt.h:254:3: note: expanded from macro 'STAP_PROBE'
  _SDT_PROBE(provider, name, 0, ())
  ^
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
/usr/include/sys/sdt.h:43:31: note: expanded from macro '_SDT_ASM_3'
# define _SDT_ASM_3(a, b, c)            _SDT_S(a) "," _SDT_S(b) "," \
                                        ^
/usr/include/sys/sdt.h:40:22: note: expanded from macro '_SDT_S'
# define _SDT_S(x)                      #x
                                        ^
<scratch space>:4:2: note: expanded from here
".pushsection .note.stapsdt"
 ^
<inline asm>:2:31: note: instantiated into assembly here
.pushsection .note.stapsdt,"?","note"
                              ^
sdt.c:7:3: error: .popsection without corresponding .pushsection
  DEMO_FOO();
  ^
./sdt.h:19:20: note: expanded from macro 'DEMO_FOO'
#define DEMO_FOO() \
                   ^
/usr/include/sys/sdt.h:362:3: note: expanded from macro '\
DTRACE_PROBE'
  STAP_PROBE(provider,probe)
  ^
/usr/include/sys/sdt.h:254:3: note: expanded from macro 'STAP_PROBE'
  _SDT_PROBE(provider, name, 0, ())
  ^
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
/usr/include/sys/sdt.h:41:26: note: expanded from macro '_SDT_ASM_1'
# define _SDT_ASM_1(x)                  _SDT_S(x) "\n"
                                        ^
/usr/include/sys/sdt.h:40:22: note: expanded from macro '_SDT_S'
# define _SDT_S(x)                      #x
                                        ^
<scratch space>:24:2: note: expanded from here
".popsection"
 ^
<inline asm>:14:12: note: instantiated into assembly here
.popsection
           ^
sdt.c:7:3: error: expected '@' or '%' before type
  DEMO_FOO();
  ^
./sdt.h:19:20: note: expanded from macro 'DEMO_FOO'
#define DEMO_FOO() \
                   ^
/usr/include/sys/sdt.h:362:3: note: expanded from macro '\
DTRACE_PROBE'
  STAP_PROBE(provider,probe)
  ^
/usr/include/sys/sdt.h:254:3: note: expanded from macro 'STAP_PROBE'
  _SDT_PROBE(provider, name, 0, ())
  ^
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
/usr/include/sys/sdt.h:45:36: note: expanded from macro '_SDT_ASM_5'
# define _SDT_ASM_5(a, b, c, d, e)      _SDT_S(a) "," _SDT_S(b) "," \
                                        ^
/usr/include/sys/sdt.h:40:22: note: expanded from macro '_SDT_S'
# define _SDT_S(x)                      #x
                                        ^
<scratch space>:27:2: note: expanded from here
".pushsection .stapsdt.base"
 ^
<inline asm>:2:33: note: instantiated into assembly here
.pushsection .stapsdt.base,"aG","progbits",.stapsdt.base,comdat
                                ^
sdt.c:7:3: error: .popsection without corresponding .pushsection
  DEMO_FOO();
  ^
./sdt.h:19:20: note: expanded from macro 'DEMO_FOO'
#define DEMO_FOO() \
                   ^
/usr/include/sys/sdt.h:362:3: note: expanded from macro '\
DTRACE_PROBE'
  STAP_PROBE(provider,probe)
  ^
/usr/include/sys/sdt.h:254:3: note: expanded from macro 'STAP_PROBE'
  _SDT_PROBE(provider, name, 0, ())
  ^
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
/usr/include/sys/sdt.h:41:26: note: expanded from macro '_SDT_ASM_1'
# define _SDT_ASM_1(x)                  _SDT_S(x) "\n"
                                        ^
/usr/include/sys/sdt.h:40:22: note: expanded from macro '_SDT_S'
# define _SDT_S(x)                      #x
                                        ^
<scratch space>:37:2: note: expanded from here
".popsection"
 ^
<inline asm>:7:12: note: instantiated into assembly here
.popsection
           ^
4 errors generated.


Gcc by comparison works as expected

$ gcc -o sdt sdt.c
...no errors...

Systemtap support is the only remaining blocker to building a fully featured libvirt with clang.

Version-Release number of selected component (if applicable):
systemtap-sdt-devel-2.3-1.fc19.x86_64
clang-3.3-0.6.rc3.fc19.x86_64

Comment 1 Mark Wielaard 2013-08-21 10:02:42 UTC
I think that is just clang not properly supporting the inline assembly directives in /usr/include/sys/sdt.h. .pushsection and .popsection a gas directive [*]. It is needed to put the actual probes in the correct ELF section where gdb or stap can pick them up. Does clang use gas as assembler by default or something else?

[*] http://sourceware.org/binutils/docs/as/PushSection.html

It is probably easiest to write a little configure script to check the compiler can at least compile SDT probes and if not disable them.

Comment 2 Daniel Berrangé 2013-08-21 10:07:51 UTC
This bug report suggests that clang should be supporting pushsection/popsection in asm blocks by now  http://llvm.org/bugs/show_bug.cgi?id=8633

Comment 3 Mark Wielaard 2013-08-21 10:14:18 UTC
So there seem to be other deficiencies in the clang assembler then.

I did find the following bug report:
http://sourceware.org/bugzilla/show_bug.cgi?id=13974

One problem is that clang apparently doesn't use the system assembler (gas) by default, so the installed sdt-config.h doesn't match which capabilities the system assembler has. But even with that fixed it looks like there are other troubles with using the clang/llvm assembler. Is there a way for clang to just use the system installed gas?

Comment 4 Frank Ch. Eigler 2013-08-23 02:21:57 UTC
See also http://sourceware.org/bugzilla/show_bug.cgi?id=13974

I can't think of a way we can work around these clang<->gcc incompatibilities
within the sys/sdt.h file.  Redirecting to clang for their consideration.

Comment 5 Josh Stone 2013-09-30 17:33:11 UTC
FYI - we did have a little movement on stap PR13974, but ended up reverting as the attempted change caused other issues.

However, llvm-mc has gained support for both of the missing constructs:
http://llvm.org/bugs/show_bug.cgi?id=17198
http://llvm.org/bugs/show_bug.cgi?id=17270

Comment 6 Frank Ch. Eigler 2013-11-12 18:55:40 UTC
Dan, please consider reassigning this against llvm, if their upstream
patches to implement double-quoted "foo" section-attributes may be
backported to f19.

Comment 7 Jan Včelák 2014-12-26 00:26:56 UTC
Works with clang-3.4-10.fc20.x86_64 and systemtap-sdt-devel-2.6-2.fc20.x86_64.

I'm closing this report as we hardly have resources to fix bugs in current Fedora releases. If this is really needed in F19, feel free to reopen this.