Note: I've filed this under GCC because I don't really knwo where the problem lies here, but I'm reasonably certain it's somewhere low level. Reproducer: $ sudo dnf install uwsgi-plugin-php uwsgi-logger-file php-gd $ rpm -q uwsgi-plugin-php uwsgi-logger-file php-gd php-embedded uwsgi libgomp uwsgi-plugin-php-2.0.21-7.fc38.x86_64 uwsgi-logger-file-2.0.21-7.fc38.x86_64 php-gd-8.2.5-1.fc38.x86_64 php-embedded-8.2.5-1.fc38.x86_64 uwsgi-2.0.21-7.fc38.x86_64 libgomp-13.1.1-1.fc38.x86_64 $ uwsgi --plugins php,logfile --logger file:/dev/tty --req-logger file:/dev/tty *** Starting uWSGI 2.0.21 (64bit) on [Mon May 15 15:32:28 2023] *** compiled with version: 13.0.1 20230401 (Red Hat 13.0.1-0) on 21 April 2023 00:00:00 os: Linux-6.2.14-300.fc38.x86_64 #1 SMP PREEMPT_DYNAMIC Mon May 1 00:55:28 UTC 2023 nodename: kendra machine: x86_64 clock source: unix pcre jit disabled detected number of CPU cores: 4 current working directory: /home/sun detected binary path: /usr/sbin/uwsgi your processes number limit is 63057 your memory page size is 4096 bytes detected max file descriptor number: 1024 lock engine: pthread robust mutexes thunder lock: disabled (you can enable it with --thunder-lock) !!! uWSGI process 43334 got Segmentation Fault !!! *** backtrace of 43334 *** uwsgi(uwsgi_backtrace+0x4b) [0x556e3a1d258b] uwsgi(uwsgi_segfault+0x2f) [0x556e3a1d2a5f] /lib64/libc.so.6(+0x3db70) [0x7f2548cebb70] /lib64/libgomp.so.1(+0xba4e) [0x7f2547038a4e] /lib64/ld-linux-x86-64.so.2(+0x517f) [0x7f25496e917f] /lib64/ld-linux-x86-64.so.2(+0x527d) [0x7f25496e927d] /lib64/ld-linux-x86-64.so.2(_dl_catch_exception+0x142) [0x7f25496e55c2] /lib64/ld-linux-x86-64.so.2(+0xbe3c) [0x7f25496efe3c] /lib64/ld-linux-x86-64.so.2(_dl_catch_exception+0xa3) [0x7f25496e5523] /lib64/ld-linux-x86-64.so.2(+0xc1b4) [0x7f25496f01b4] /lib64/libc.so.6(+0x886d4) [0x7f2548d366d4] /lib64/ld-linux-x86-64.so.2(_dl_catch_exception+0xa3) [0x7f25496e5523] /lib64/ld-linux-x86-64.so.2(+0x1679) [0x7f25496e5679] /lib64/libc.so.6(+0x881b3) [0x7f2548d361b3] /lib64/libc.so.6(dlopen+0x6f) [0x7f2548d3678f] /lib64/libphp-8.2.so(php_load_shlib+0x18) [0x7f254881f548] /lib64/libphp-8.2.so(php_load_extension+0x205) [0x7f254881f785] /lib64/libphp-8.2.so(zend_llist_apply+0x27) [0x7f25488e2637] /lib64/libphp-8.2.so(+0x28dfc5) [0x7f254888dfc5] /lib64/libphp-8.2.so(php_module_startup+0x836) [0x7f25488866e6] /usr/lib64/uwsgi/php_plugin.so(+0x5b2b) [0x7f25496d6b2b] /usr/lib64/uwsgi/php_plugin.so(+0x5eae) [0x7f25496d6eae] uwsgi(uwsgi_start+0x5b7) [0x556e3a1d3637] uwsgi(uwsgi_setup+0x17ac) [0x556e3a1d5f8c] uwsgi(main+0xd) [0x556e3a178ded] /lib64/libc.so.6(+0x27b4a) [0x7f2548cd5b4a] /lib64/libc.so.6(__libc_start_main+0x8b) [0x7f2548cd5c0b] uwsgi(_start+0x25) [0x556e3a178e25] *** end of backtrace *** This blows up in libgomp, in this code: ``` 2057 set to the initial global values. */ 2058 add_initial_icv_to_list (GOMP_DEVICE_NUM_FOR_NO_SUFFIX, 0, NULL); 2059 none = gomp_get_initial_icv_item (GOMP_DEVICE_NUM_FOR_NO_SUFFIX); 2060 initialize_icvs (&none->icvs); 2061 *2062 for (env = environ; *env != 0; env++) 2063 { 2064 if (!startswith (*env, "OMP_")) 2065 continue; 2066 ``` `env` (not `*env`) turns out to be 0x0, at the very start of the `for` loop, the loop never runs. I don't even know how that's possible. I've, so far, been only able to reproduce this when running php under uwsgi, running the php CLI interpreter works fine, even though it also loads the php-gd plugin, which subsequently loads libgomp. The only obvious difference I can see right now is that uwsgi adds some variables to the environment before it loads php (via `setenv`). This might be some relocation gone wrong? I have core dump files, if required, but the issue is easily reprodu Reproducible: Always
If environ is NULL, it is IMNSHO a bug in whatever program changed that pointer to NULL.
There's also a glibc bug in this area, when dlopen is called from a preinit function. Not sure if this applies in this case. Setting a hardware watchpoint on the environ variable should be illuminating.
But `environ` is not NULL, that's the strange thing. `environ` points to a, as far as I can tell, completely valid environment structure. ``` >>> print environ $1 = (char **) 0x55555564ca70 >>> print env $2 = (char **) 0x0 >>> print *(environ) $3 = 0x555555648e60 "SHELL=/bin/bash" ``` This is the disassembly around the error: ``` 0x00007ffff5909a13 <+99>: mov rax,QWORD PTR [rip+0x4559e] # 0x7ffff594efb8 0x00007ffff5909a1a <+106>: pxor xmm0,xmm0 0x00007ffff5909a1e <+110>: mov WORD PTR [rbp+0x54],0x100 0x00007ffff5909a24 <+116>: lea r14,[rbp+0x8] 0x00007ffff5909a28 <+120>: movups XMMWORD PTR [rbp+0x8],xmm0 0x00007ffff5909a2c <+124>: movdqa xmm1,XMMWORD PTR [rip+0x363bc] # 0x7ffff593fdf0 0x00007ffff5909a34 <+132>: lea r13,[rip+0x358d4] # 0x7ffff593f30f 0x00007ffff5909a3b <+139>: mov r15,QWORD PTR [rax] 0x00007ffff5909a3e <+142>: mov QWORD PTR [rbp+0x48],0xffffffffffffffff 0x00007ffff5909a46 <+150>: mov BYTE PTR [rbp+0x56],0x0 0x00007ffff5909a4a <+154>: movups XMMWORD PTR [rbp+0x28],xmm0 => 0x00007ffff5909a4e <+158>: mov r12,QWORD PTR [r15] ``` `[r15]` blows up, because `r15` is 0. This gets loaded from `[rax]` four lines above, and `rax` in turn was loaded from a `rip` relative position at the start. This looks like the assembly expects `[rip+0x4559e]` to be the memory location where `environ` is, but that's not where it is? I'm not really useful with gdb, I admit.
It's due to use of RTLD_DEEPBIND in php: # define DL_LOAD(libname) dlopen(libname, PHP_RTLD_MODE | RTLD_GLOBAL | RTLD_DEEPBIND) RTLD_DEEPBIND changes the binding of environ in libc.so.6 from the main executable (present there because of a copy relocation, and correctly initialized) to the previously dormant symbol in libc.so.6 (which is uninitialized). GDB doesn't really handle this rebinding, which is why environ shows up as not NULL. Ideally, php would stop using RTLD_DEEPBIND. If this is not possible, you can build the main program (uswgi in this case) as PIC, not PIE, which avoids copy relocations.
Is there another can of worms that I'd open with the PIE/PIC change? Is that a change that's required for uwsgi as a whole (binary and dynamically loadable modules), or just for the php module?
(In reply to Ralf Ertzinger from comment #5) > Is there another can of worms that I'd open with the PIE/PIC change? Is that > a change that's required for uwsgi as a whole (binary and dynamically > loadable modules), or just for the php module? Just the main program (uwsgi), the modules are already PIC.
Thanks, I'll try that.
I was tagged into an upstream user hitting problems with RTLD_DEEPBIND recently. https://github.com/php/php-src/issues/10670 It may be time to turn this into a config option and flip the default from On to Off.
I've rebuilt uwsgi with %undefine _hardened_build which should disable PIE according to https://docs.fedoraproject.org/en-US/packaging-guidelines/#_pie And the executable no longer says "LSB pie executable" /usr/sbin/uwsgi: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=160cd2740fd9e07a5a72f574f62d2043a8fa075d, for GNU/Linux 3.2.0, stripped Unfortunately it still blows up in the same way.
(In reply to Ralf Ertzinger from comment #9) > I've rebuilt uwsgi with > > %undefine _hardened_build > > which should disable PIE according to > https://docs.fedoraproject.org/en-US/packaging-guidelines/#_pie > > And the executable no longer says "LSB pie executable" > > /usr/sbin/uwsgi: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), > dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, > BuildID[sha1]=160cd2740fd9e07a5a72f574f62d2043a8fa075d, for GNU/Linux 3.2.0, > stripped > > Unfortunately it still blows up in the same way. You need to build as PIC. Try adding -fPIC to CFLAGS.
I've rebuilt the main binary (and the php plugin for good measure) with -fPIC (both in CFLAGS and LDFLAGS), and it still crashes the same way. Build logs: https://download.copr.fedorainfracloud.org/results/ertzing/scratch_x64/fedora-38-x86_64/05929259-uwsgi/build.log.gz
what are the chances of having php buiit without RTLD_DEEPBIND for Fedora? I've managed to build uwsgi so this isn't an acute issue any more, but this requires disabling `_hardened_build` in the .spec file, which according to https://docs.fedoraproject.org/en-US/packaging-guidelines/#_pie I probably shouldn't be doing.