Hide Forgot
Description of problem: The problem exists since FC34. Sometimes when httpd starts, it core dumps several files but it is able to start up without intervention. -rw------- 1 root apache 271429632 Oct 3 14:45 core.httpd-16891-11-1633243502 -rw------- 1 root apache 179515392 Oct 3 14:45 core.httpd-17782-11-1633243503 -rw------- 1 root apache 356098048 Oct 3 14:45 core.httpd-18384-11-1633243503 -rw------- 1 root apache 398430208 Oct 3 14:45 core.httpd-18999-11-1633243504 -rw------- 1 root apache 440766464 Oct 3 14:45 core.httpd-19383-11-1633243507 -rw------- 1 root apache 483098624 Oct 3 14:45 core.httpd-19737-11-1633243508 -rw------- 1 root apache 525430784 Oct 3 14:45 core.httpd-20154-11-1633243508 -rw------- 1 root apache 567767040 Oct 3 14:45 core.httpd-20633-11-1633243510 -rw------- 1 root apache 610099200 Oct 3 14:45 core.httpd-21085-11-1633243511 -rw------- 1 root apache 652435456 Oct 3 14:45 core.httpd-21519-11-1633243512 -rw------- 1 root apache 694767616 Oct 3 14:45 core.httpd-22906-11-1633243515 -rw------- 1 root apache 743391232 Oct 3 14:45 core.httpd-24520-11-1633243518 -rw------- 1 root apache 785727488 Oct 3 14:45 core.httpd-26026-11-1633243520 -rw------- 1 root apache 828059648 Oct 3 14:45 core.httpd-29431-11-1633243525 Backtrace: Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00007f0fb6c7d659 in apr_atomic_dec32 () from /lib64/libapr-1.so.0 [Current thread is 1 (Thread 0x7f0fb66e9140 (LWP 16891))] (gdb) bt #0 0x00007f0fb6c7d659 in apr_atomic_dec32 () at /lib64/libapr-1.so.0 #1 0x00007f0fb6c90219 in proc_pthread_mutex_unref () at /lib64/libapr-1.so.0 #2 0x00007f0fb6c90cbe in apr_pool_destroy () at /lib64/libapr-1.so.0 #3 0x00007f0fb5b8c52f in clean_child_exit (code=0) at /usr/src/debug/httpd-2.4.49-1.fc35.x86_64/server/mpm/event/event.c:730 #4 0x00007f0fb5b8e99a in child_main (child_num_arg=child_num_arg@entry=0, child_bucket=child_bucket@entry=0) at /usr/src/debug/httpd-2.4.49-1.fc35.x86_64/server/mpm/event/event.c:2556 #5 0x00007f0fb5b8f2cf in make_child (s=0x7f0fb6247110, slot=slot@entry=0, bucket=0) at /usr/src/debug/httpd-2.4.49-1.fc35.x86_64/server/mpm/event/event.c:2754 #6 0x00007f0fb5b8f371 in startup_children (number_to_start=number_to_start@entry=1) at /usr/src/debug/httpd-2.4.49-1.fc35.x86_64/server/mpm/event/event.c:2777 #7 0x00007f0fb5b9013f in event_run (_pconf=<optimized out>, plog=0x7f0fb624b028, s=0x7f0fb6247110) at /usr/src/debug/httpd-2.4.49-1.fc35.x86_64/server/mpm/event/event.c:3150 #8 0x00005618346493f8 in ap_run_mpm (pconf=0x7f0fb621f028, plog=0x7f0fb624b028, s=0x7f0fb6247110) at server/mpm_common.c:95 #9 0x0000561834637484 in main (argc=<optimized out>, argv=<optimized out>) at server/main.c:819 Version-Release number of selected component (if applicable): httpd-2.4.49-1.fc35.x86_64 How reproducible: It is not like this on every time apache starts. But it is occurring about two to three times every week. Steps to Reproduce: System reboot and systemd starts apache
Created attachment 1828813 [details] output of "thread apply all bt full"
Please try httpd 2.4.51 and see if this still occurs. https://bodhi.fedoraproject.org/updates/FEDORA-2021-2a10bc68a4
When I upgrade to httpd 2.4.51, FC35 binary has not yet been released/built. So I use httpd 2.4.51 FC34 rpm on FC35. It still have core dumps. Here's one of it: (gdb) thread apply all bt full Thread 18 (Thread 0x7f77a61fc640 (LWP 1012186)): #0 0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6 #4 0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6 Thread 17 (Thread 0x7f77a09f1640 (LWP 1012197)): #0 0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6 #4 0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6 Thread 16 (Thread 0x7f77a39f7640 (LWP 1012191)): #0 0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6 #4 0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6 Thread 15 (Thread 0x7f77a31f6640 (LWP 1012192)): #0 0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0 --Type <RET> for more, q to quit, c to continue without paging--c #3 0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6 #4 0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6 Thread 14 (Thread 0x7f77a19f3640 (LWP 1012195)): #0 0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6 #4 0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6 Thread 13 (Thread 0x7f77a41f8640 (LWP 1012190)): #0 0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6 #4 0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6 Thread 12 (Thread 0x7f77a51fa640 (LWP 1012188)): #0 0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6 #4 0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6 Thread 11 (Thread 0x7f77a21f4640 (LWP 1012194)): #0 0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6 #4 0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6 Thread 10 (Thread 0x7f77a29f5640 (LWP 1012193)): #0 0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6 #4 0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6 Thread 9 (Thread 0x7f77a59fb640 (LWP 1012187)): #0 0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6 #4 0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6 Thread 8 (Thread 0x7f77a8615640 (LWP 1012182)): #0 0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6 #4 0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6 Thread 7 (Thread 0x7f77a69fd640 (LWP 1012185)): #0 0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6 #4 0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6 Thread 6 (Thread 0x7f77a79ff640 (LWP 1012183)): #0 0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6 #4 0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6 Thread 5 (Thread 0x7f77a49f9640 (LWP 1012189)): #0 0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6 #4 0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6 Thread 4 (Thread 0x7f77a11f2640 (LWP 1012196)): #0 0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6 #4 0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6 Thread 3 (Thread 0x7f77a71fe640 (LWP 1012184)): #0 0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6 #4 0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6 Thread 2 (Thread 0x7f77a8e16640 (LWP 1012181)): #0 0x00007f77b57bb02d in select () at /lib64/libc.so.6 #1 0x00007f77b58f4639 in apr_sleep () at /lib64/libapr-1.so.0 #2 0x00007f77b491ad9e in wd_worker (thread=<optimized out>, data=0x7f77b2db2770) at /usr/src/debug/httpd-2.4.51-1.fc34.x86_64/modules/core/mod_watchdog.c:192 curr = <optimized out> wl = 0x7f77b2db2900 w = 0x7f77b2db2770 rv = <optimized out> locked = 1 probed = <optimized out> inited = 0 mpmq_s = 1 temp_pool = 0x7f77b4fa5028 #3 0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6 #4 0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6 Thread 1 (Thread 0x7f77b533f140 (LWP 1012180)): #0 0x00007f77b58dd659 in apr_atomic_dec32 () at /lib64/libapr-1.so.0 #1 0x00007f77b58f0219 in proc_pthread_mutex_unref () at /lib64/libapr-1.so.0 #2 0x00007f77b58f0cbe in apr_pool_destroy () at /lib64/libapr-1.so.0 #3 0x00007f77b47e152f in clean_child_exit (code=0) at /usr/src/debug/httpd-2.4.51-1.fc34.x86_64/server/mpm/event/event.c:730 #4 0x00007f77b47e396a in child_main (child_num_arg=child_num_arg@entry=0, child_bucket=child_bucket@entry=0) at /usr/src/debug/httpd-2.4.51-1.fc34.x86_64/server/mpm/event/event.c:2555 threads = <optimized out> rv = <optimized out> ts = <optimized out> thread_attr = 0x7f77b2e465e8 start_thread_id = 0x7f77b2e46648 i = <optimized out> #5 0x00007f77b47e429f in make_child (s=0x7f77b4e47110, slot=slot@entry=0, bucket=bucket@entry=0) at /usr/src/debug/httpd-2.4.51-1.fc34.x86_64/server/mpm/event/event.c:2753 pid = <optimized out> #6 0x00007f77b47e5269 in server_main_loop (num_buckets=1, remaining_children_to_start=1) at /usr/src/debug/httpd-2.4.51-1.fc34.x86_64/server/mpm/event/event.c:3052 ps = <optimized out> status = 11 pid = {pid = 1006218, in = 0x7f77b2d4ab00, out = 0x2e3836312e323931, err = 0x38392e32} i = <optimized out> child_slot = 0 exitwhy = (APR_PROC_SIGNAL | APR_PROC_SIGNAL_CORE) processed_status = 0 num_buckets = 1 remaining_children_to_start = <optimized out> i = <optimized out> #7 event_run (_pconf=<optimized out>, plog=<optimized out>, s=<optimized out>) at /usr/src/debug/httpd-2.4.51-1.fc34.x86_64/server/mpm/event/event.c:3171 num_buckets = 1 remaining_children_to_start = <optimized out> i = <optimized out> #8 0x000055f66ebe2428 in ap_run_mpm (pconf=0x7f77b4e1f028, plog=0x7f77b4e4b028, s=0x7f77b4e47110) at server/mpm_common.c:95 pHook = <optimized out> n = 0 rv = -1 #9 0x000055f66ebd0484 in main (argc=<optimized out>, argv=<optimized out>) at server/main.c:819 c = 102 'f' showcompile = <optimized out> showdirectives = 0 confname = 0x7fff37458ea2 "/etc/httpd-web1/conf/httpd.conf" def_server_root = 0x55f66ec1d761 "/etc/httpd" temp_error_log = <optimized out> error = <optimized out> process = 0x7f77b4e1d118 pconf = <optimized out> plog = 0x7f77b4e4b028 ptemp = 0x7f77b4e76028 pcommands = 0x7f77b4e41028 opt = 0x7f77b4e41118 rv = <optimized out> mod = <optimized out> opt_arg = 0x7fff37458ea2 "/etc/httpd-web1/conf/httpd.conf" signal_server = <optimized out> rc = <optimized out> Now I see httpd-2.4.51 FC35 rpm is out in koji. I would install it.
It is still crashing with the httpd 2.4.51 FC35 rpm: Core was generated by `/usr/sbin/httpd -DFOREGROUND -f /etc/httpd-web1/conf/httpd.conf'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00007ff05da33659 in apr_atomic_dec32 () from /lib64/libapr-1.so.0 [Current thread is 1 (Thread 0x7ff05d49f140 (LWP 19523))] (gdb) (gdb) (gdb) thread apply all bt full Thread 18 (Thread 0x7ff0568d6640 (LWP 19527)): #0 0x00007ff05d91b02d in select () at /lib64/libc.so.6 #1 0x00007ff05da4a639 in apr_sleep () at /lib64/libapr-1.so.0 #2 0x00007ff05c98dd9e in wd_worker (thread=<optimized out>, data=0x7ff05af56770) at /usr/src/debug/httpd-2.4.51-1.fc35.x86_64/modules/core/mod_watchdog.c:192 curr = <optimized out> wl = 0x7ff05af56900 w = 0x7ff05af56770 rv = <optimized out> locked = 1 probed = <optimized out> inited = 0 mpmq_s = 1 temp_pool = 0x7ff05d17a028 #3 0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6 #4 0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6 Thread 17 (Thread 0x7ff0533fb640 (LWP 19533)): #0 0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6 #4 0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6 Thread 16 (Thread 0x7ff051bf8640 (LWP 19536)): #0 0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6 #4 0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6 Thread 15 (Thread 0x7ff04ebf2640 (LWP 19542)): --Type <RET> for more, q to quit, c to continue without paging--c #0 0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6 #4 0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6 Thread 14 (Thread 0x7ff04f3f3640 (LWP 19541)): #0 0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6 #4 0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6 Thread 13 (Thread 0x7ff04fbf4640 (LWP 19540)): #0 0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6 #4 0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6 Thread 12 (Thread 0x7ff0503f5640 (LWP 19539)): #0 0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6 #4 0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6 Thread 11 (Thread 0x7ff050bf6640 (LWP 19538)): #0 0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6 #4 0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6 Thread 10 (Thread 0x7ff0513f7640 (LWP 19537)): #0 0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6 #4 0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6 Thread 9 (Thread 0x7ff0523f9640 (LWP 19535)): #0 0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6 #4 0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6 Thread 8 (Thread 0x7ff052bfa640 (LWP 19534)): #0 0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6 #4 0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6 Thread 7 (Thread 0x7ff053bfc640 (LWP 19532)): #0 0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6 #4 0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6 Thread 6 (Thread 0x7ff0543fd640 (LWP 19531)): #0 0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6 #4 0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6 Thread 5 (Thread 0x7ff04e3f1640 (LWP 19543)): #0 0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6 #4 0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6 Thread 4 (Thread 0x7ff054bfe640 (LWP 19530)): #0 0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6 #4 0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6 Thread 3 (Thread 0x7ff0553ff640 (LWP 19529)): #0 0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6 #4 0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6 Thread 2 (Thread 0x7ff0560d5640 (LWP 19528)): #0 0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6 #1 0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 #2 0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0 #3 0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6 #4 0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6 Thread 1 (Thread 0x7ff05d49f140 (LWP 19523)): #0 0x00007ff05da33659 in apr_atomic_dec32 () at /lib64/libapr-1.so.0 #1 0x00007ff05da46219 in proc_pthread_mutex_unref () at /lib64/libapr-1.so.0 #2 0x00007ff05da46cbe in apr_pool_destroy () at /lib64/libapr-1.so.0 #3 0x00007ff05c94a52f in clean_child_exit (code=0) at /usr/src/debug/httpd-2.4.51-1.fc35.x86_64/server/mpm/event/event.c:730 #4 0x00007ff05c94c96a in child_main (child_num_arg=child_num_arg@entry=0, child_bucket=child_bucket@entry=0) at /usr/src/debug/httpd-2.4.51-1.fc35.x86_64/server/mpm/event/event.c:2555 threads = <optimized out> rv = <optimized out> ts = <optimized out> thread_attr = 0x7ff05afc85e8 start_thread_id = 0x7ff05afc8648 i = <optimized out> #5 0x00007ff05c94d29f in make_child (s=0x7ff05d047110, slot=slot@entry=0, bucket=0) at /usr/src/debug/httpd-2.4.51-1.fc35.x86_64/server/mpm/event/event.c:2753 pid = <optimized out> #6 0x00007ff05c94d341 in startup_children (number_to_start=number_to_start@entry=1) at /usr/src/debug/httpd-2.4.51-1.fc35.x86_64/server/mpm/event/event.c:2776 i = 0 #7 0x00007ff05c94e14f in event_run (_pconf=<optimized out>, plog=0x7ff05d04b028, s=0x7ff05d047110) at /usr/src/debug/httpd-2.4.51-1.fc35.x86_64/server/mpm/event/event.c:3152 num_buckets = 1 remaining_children_to_start = 1 i = <optimized out> #8 0x0000560e5a05a428 in ap_run_mpm (pconf=0x7ff05d01f028, plog=0x7ff05d04b028, s=0x7ff05d047110) at server/mpm_common.c:95 pHook = <optimized out> n = 0 rv = -1 #9 0x00007ff05d01f028 in () #10 0x00007ff05d01d118 in () #11 0x0000560e5a0ba4a0 in ap_conftree () #12 0x00007ffecf2916c0 in () #13 0x0000560e5a0957b3 in () #14 0x0000560e5a048484 in main (argc=<optimized out>, argv=<optimized out>) at server/main.c:819 c = 93 ']' showcompile = <optimized out> showdirectives = 1552159360 confname = 0x71f476b4 <error: Cannot access memory at address 0x71f476b4> def_server_root = 0x0 temp_error_log = <optimized out> error = <optimized out> process = 0x7ff05d01f028 pconf = <optimized out> plog = 0x7 ptemp = 0xf4908b9c7cd56700 pcommands = 0x7ff05c841280 <cgi_handle_exec> opt = 0x14 rv = <optimized out> mod = <optimized out> opt_arg = 0x0 signal_server = <optimized out> rc = <optimized out>
Thanks for testing further. Could you do # debuginfo-install apr glibc and regenerate the backtraces?
Created attachment 1831718 [details] output for httpd 2.4.51.txt For https://bugzilla.redhat.com/show_bug.cgi?id=2010166#c5