Bug 2203863 - php: Should not use RTLD_DEEPBIND for opening extension modules
Summary: php: Should not use RTLD_DEEPBIND for opening extension modules
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: php
Version: 38
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Remi Collet
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-05-15 13:39 UTC by Ralf Ertzinger
Modified: 2023-09-12 14:53 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Ralf Ertzinger 2023-05-15 13:39:59 UTC
Note: I've filed this under GCC because I don't really knwo where the problem lies here, but I'm reasonably certain it's somewhere low level.

Reproducer:
$ sudo dnf install uwsgi-plugin-php uwsgi-logger-file  php-gd

$ rpm -q uwsgi-plugin-php uwsgi-logger-file  php-gd php-embedded uwsgi libgomp
uwsgi-plugin-php-2.0.21-7.fc38.x86_64
uwsgi-logger-file-2.0.21-7.fc38.x86_64
php-gd-8.2.5-1.fc38.x86_64
php-embedded-8.2.5-1.fc38.x86_64
uwsgi-2.0.21-7.fc38.x86_64
libgomp-13.1.1-1.fc38.x86_64

$ uwsgi --plugins php,logfile --logger file:/dev/tty --req-logger file:/dev/tty
*** Starting uWSGI 2.0.21 (64bit) on [Mon May 15 15:32:28 2023] ***
compiled with version: 13.0.1 20230401 (Red Hat 13.0.1-0) on 21 April 2023 00:00:00
os: Linux-6.2.14-300.fc38.x86_64 #1 SMP PREEMPT_DYNAMIC Mon May  1 00:55:28 UTC 2023
nodename: kendra
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /home/sun
detected binary path: /usr/sbin/uwsgi
your processes number limit is 63057
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
!!! uWSGI process 43334 got Segmentation Fault !!!
*** backtrace of 43334 ***
uwsgi(uwsgi_backtrace+0x4b) [0x556e3a1d258b]
uwsgi(uwsgi_segfault+0x2f) [0x556e3a1d2a5f]
/lib64/libc.so.6(+0x3db70) [0x7f2548cebb70]
/lib64/libgomp.so.1(+0xba4e) [0x7f2547038a4e]
/lib64/ld-linux-x86-64.so.2(+0x517f) [0x7f25496e917f]
/lib64/ld-linux-x86-64.so.2(+0x527d) [0x7f25496e927d]
/lib64/ld-linux-x86-64.so.2(_dl_catch_exception+0x142) [0x7f25496e55c2]
/lib64/ld-linux-x86-64.so.2(+0xbe3c) [0x7f25496efe3c]
/lib64/ld-linux-x86-64.so.2(_dl_catch_exception+0xa3) [0x7f25496e5523]
/lib64/ld-linux-x86-64.so.2(+0xc1b4) [0x7f25496f01b4]
/lib64/libc.so.6(+0x886d4) [0x7f2548d366d4]
/lib64/ld-linux-x86-64.so.2(_dl_catch_exception+0xa3) [0x7f25496e5523]
/lib64/ld-linux-x86-64.so.2(+0x1679) [0x7f25496e5679]
/lib64/libc.so.6(+0x881b3) [0x7f2548d361b3]
/lib64/libc.so.6(dlopen+0x6f) [0x7f2548d3678f]
/lib64/libphp-8.2.so(php_load_shlib+0x18) [0x7f254881f548]
/lib64/libphp-8.2.so(php_load_extension+0x205) [0x7f254881f785]
/lib64/libphp-8.2.so(zend_llist_apply+0x27) [0x7f25488e2637]
/lib64/libphp-8.2.so(+0x28dfc5) [0x7f254888dfc5]
/lib64/libphp-8.2.so(php_module_startup+0x836) [0x7f25488866e6]
/usr/lib64/uwsgi/php_plugin.so(+0x5b2b) [0x7f25496d6b2b]
/usr/lib64/uwsgi/php_plugin.so(+0x5eae) [0x7f25496d6eae]
uwsgi(uwsgi_start+0x5b7) [0x556e3a1d3637]
uwsgi(uwsgi_setup+0x17ac) [0x556e3a1d5f8c]
uwsgi(main+0xd) [0x556e3a178ded]
/lib64/libc.so.6(+0x27b4a) [0x7f2548cd5b4a]
/lib64/libc.so.6(__libc_start_main+0x8b) [0x7f2548cd5c0b]
uwsgi(_start+0x25) [0x556e3a178e25]
*** end of backtrace ***

This blows up in libgomp, in this code:

```
 2057       set to the initial global values.  */
 2058    add_initial_icv_to_list (GOMP_DEVICE_NUM_FOR_NO_SUFFIX, 0, NULL);
 2059    none = gomp_get_initial_icv_item (GOMP_DEVICE_NUM_FOR_NO_SUFFIX);
 2060    initialize_icvs (&none->icvs);
 2061
*2062    for (env = environ; *env != 0; env++)
 2063      {
 2064        if (!startswith (*env, "OMP_"))
 2065      continue;
 2066
```

`env` (not `*env`) turns out to be 0x0, at the very start of the `for` loop, the loop never runs. I don't even know how that's possible.

I've, so far, been only able to reproduce this when running php under uwsgi, running the php CLI interpreter works fine, even though it also loads the php-gd plugin, which subsequently loads libgomp.

The only obvious difference I can see right now is that uwsgi adds some variables to the environment before it loads php (via `setenv`). This might be some relocation gone wrong?

I have core dump files, if required, but the issue is easily reprodu

Reproducible: Always

Comment 1 Jakub Jelinek 2023-05-15 13:44:58 UTC
If environ is NULL, it is IMNSHO a bug in whatever program changed that pointer to NULL.

Comment 2 Florian Weimer 2023-05-15 13:48:09 UTC
There's also a glibc bug in this area, when dlopen is called from a preinit function. Not sure if this applies in this case.

Setting a hardware watchpoint on the environ variable should be illuminating.

Comment 3 Ralf Ertzinger 2023-05-15 13:59:54 UTC
But `environ` is not NULL, that's the strange thing. `environ` points to a, as far as I can tell, completely valid environment structure.

```
>>> print environ
$1 = (char **) 0x55555564ca70
>>> print env
$2 = (char **) 0x0
>>> print *(environ)
$3 = 0x555555648e60 "SHELL=/bin/bash"
```

This is the disassembly around the error:
```
   0x00007ffff5909a13 <+99>:    mov    rax,QWORD PTR [rip+0x4559e]        # 0x7ffff594efb8
   0x00007ffff5909a1a <+106>:   pxor   xmm0,xmm0
   0x00007ffff5909a1e <+110>:   mov    WORD PTR [rbp+0x54],0x100
   0x00007ffff5909a24 <+116>:   lea    r14,[rbp+0x8]
   0x00007ffff5909a28 <+120>:   movups XMMWORD PTR [rbp+0x8],xmm0
   0x00007ffff5909a2c <+124>:   movdqa xmm1,XMMWORD PTR [rip+0x363bc]        # 0x7ffff593fdf0
   0x00007ffff5909a34 <+132>:   lea    r13,[rip+0x358d4]        # 0x7ffff593f30f
   0x00007ffff5909a3b <+139>:   mov    r15,QWORD PTR [rax]
   0x00007ffff5909a3e <+142>:   mov    QWORD PTR [rbp+0x48],0xffffffffffffffff
   0x00007ffff5909a46 <+150>:   mov    BYTE PTR [rbp+0x56],0x0
   0x00007ffff5909a4a <+154>:   movups XMMWORD PTR [rbp+0x28],xmm0
=> 0x00007ffff5909a4e <+158>:   mov    r12,QWORD PTR [r15]
```

`[r15]` blows up, because `r15` is 0. This gets loaded from `[rax]` four lines above, and `rax` in turn was loaded from a `rip` relative position at the start. This looks like the assembly expects `[rip+0x4559e]` to be the memory location where `environ` is, but that's not where it is?

I'm not really useful with gdb, I admit.

Comment 4 Florian Weimer 2023-05-15 14:04:41 UTC
It's due to use of RTLD_DEEPBIND in php:

#  define DL_LOAD(libname)                      dlopen(libname, PHP_RTLD_MODE | RTLD_GLOBAL | RTLD_DEEPBIND)

RTLD_DEEPBIND changes the binding of environ in libc.so.6 from the main executable (present there because of a copy relocation, and correctly initialized) to the previously dormant symbol in libc.so.6 (which is uninitialized). GDB doesn't really handle this rebinding, which is why environ shows up as not NULL.

Ideally, php would stop using RTLD_DEEPBIND. If this is not possible, you can build the main program (uswgi in this case) as PIC, not PIE, which avoids copy relocations.

Comment 5 Ralf Ertzinger 2023-05-15 14:15:24 UTC
Is there another can of worms that I'd open with the PIE/PIC change? Is that a change that's required for uwsgi as a whole (binary and dynamically loadable modules), or just for the php module?

Comment 6 Florian Weimer 2023-05-15 14:19:34 UTC
(In reply to Ralf Ertzinger from comment #5)
> Is there another can of worms that I'd open with the PIE/PIC change? Is that
> a change that's required for uwsgi as a whole (binary and dynamically
> loadable modules), or just for the php module?

Just the main program (uwsgi), the modules are already PIC.

Comment 7 Ralf Ertzinger 2023-05-15 14:27:57 UTC
Thanks, I'll try that.

Comment 8 Joe Orton 2023-05-15 14:42:37 UTC
I was tagged into an upstream user hitting problems with RTLD_DEEPBIND recently. https://github.com/php/php-src/issues/10670

It may be time to turn this into a config option and flip the default from On to Off.

Comment 9 Ralf Ertzinger 2023-05-15 18:57:18 UTC
I've rebuilt uwsgi with

%undefine _hardened_build

which should disable PIE according to https://docs.fedoraproject.org/en-US/packaging-guidelines/#_pie

And the executable no longer says "LSB pie executable"

/usr/sbin/uwsgi: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=160cd2740fd9e07a5a72f574f62d2043a8fa075d, for GNU/Linux 3.2.0, stripped

Unfortunately it still blows up in the same way.

Comment 10 Florian Weimer 2023-05-15 19:43:28 UTC
(In reply to Ralf Ertzinger from comment #9)
> I've rebuilt uwsgi with
> 
> %undefine _hardened_build
> 
> which should disable PIE according to
> https://docs.fedoraproject.org/en-US/packaging-guidelines/#_pie
> 
> And the executable no longer says "LSB pie executable"
> 
> /usr/sbin/uwsgi: ELF 64-bit LSB executable, x86-64, version 1 (SYSV),
> dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2,
> BuildID[sha1]=160cd2740fd9e07a5a72f574f62d2043a8fa075d, for GNU/Linux 3.2.0,
> stripped
> 
> Unfortunately it still blows up in the same way.

You need to build as PIC. Try adding -fPIC to CFLAGS.

Comment 11 Ralf Ertzinger 2023-05-17 20:19:22 UTC
I've rebuilt the main binary (and the php plugin for good measure) with -fPIC (both in CFLAGS and LDFLAGS), and it still crashes the same way.

Build logs: https://download.copr.fedorainfracloud.org/results/ertzing/scratch_x64/fedora-38-x86_64/05929259-uwsgi/build.log.gz

Comment 12 Ralf Ertzinger 2023-09-12 14:53:59 UTC
what are the chances of having php buiit without RTLD_DEEPBIND for Fedora? I've managed to build uwsgi so this isn't an acute issue any more, but this requires disabling `_hardened_build` in the .spec file, which according to https://docs.fedoraproject.org/en-US/packaging-guidelines/#_pie I probably shouldn't be doing.


Note You need to log in before you can comment on or make changes to this bug.