Description of problem:

Consider the following set of source files:

$ cat main.c
#include <stdio.h>
#include <dlfcn.h>

int main(int argc, char **argv) {
  void *mod;
  void (*modf)(void);
  
  fprintf(stderr, "Hello world\n");

  if (!(mod = dlopen("./mod.so", RTLD_LAZY | RTLD_LOCAL))) {
    fprintf(stderr, "dlopen(mod.so): %s\n", dlerror());
    return 1;
  }

  modf = dlsym(mod, "func");

  modf();

  fprintf(stderr, "Goodbye\n");
  return 0;
}


$ cat mod.c
#include <stdio.h>
#include "trace.h"
void func(void) {
  fprintf(stderr, "In mod\n");
  MAIN_MOD_EVENT();
  fprintf(stderr, "Out mod\n");
}



$ cat trace.dtrace 
provider main {
    probe mod_event();
};


Build them with the following set of commands


$ dtrace -o trace.o -G -s trace.dtrace
$ dtrace -o trace.h -h -s trace.dtrace
$ gcc -shared -o mod.so mod.c -fPIC
$ gcc -o main main.c -ldl trace.o -Wl,--export-dynamic


Notice we're using the probe in mod.so, but we're linking the trace.o into the main binary instead of mod.so

Historically this worked, *PROVIDED*, mod.so never used an "if (MAIN_MOD_EVENT_ENABLED())"  conditional check.

$ ./main 
Hello world
In mod
Out mod
Goodbye


Bulding with systemtap 4.4 this now fails

$ ./main 
Hello world
dlopen(mod.so): ./mod.so: undefined symbol: main_mod_event_semaphore


IIUC, this behaviour change was caused by this upstream commit:

commit 82b8e1a07a31bf37ed05d6ebc5162b054c0be9fd
Author: William Cohen <wcohen>
Date:   Wed Aug 5 09:43:17 2020 -0400

    Make dtrace generated code work with LTO (take 2)


As now the SDT_PROBE() macro will always reference the semaphore, even if the caller never used the MAIN_MOD_EVENT_ENABLED() check.

I note that  '-Wl,--export-dynamic' doesn't have any effect on the probes in the dtrace generated .o file which is the root problem that prevents use of the _ENABLED() macros from modules historically. I don't know if there's any way to fix this with the way system creates the .o files in the separate .probes ELF section.

I'm not sure whether I'd call this a bug in systemtap though, rather than a bug in the demonstration code.

Arguably the trace.o really should have been linked to the mod.so instead of 'main' ELF binary, since it was always broken if using the _ENABLED() macros.

The reason I raise this bug though, is that QEMU uses dlopen'd modules and always links all probes to its main binary not the modules. This is because we've not isolated probes related to modules from probes related to the main binary - they're all in the same .dtrace input file.


If the only practical answer is that QEMU needs fixing, we'll accept that, but if you think systemtap should support this scenario explicitly that'd be great.


Version-Release number of selected component (if applicable):
systemtap-sdt-devel-4.4-0.20200805git82b8e1a07.fc33.x86_64