Bug 2010166 - httpd sometimes core dumps when it is started
Summary: httpd sometimes core dumps when it is started
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: httpd
Version: 35
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Luboš Uhliarik
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-10-04 05:26 UTC by Patrick Dung
Modified: 2022-12-13 15:38 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-12-13 15:38:00 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
output of "thread apply all bt full" (41.38 KB, text/plain)
2021-10-04 05:36 UTC, Patrick Dung
no flags Details
output for httpd 2.4.51.txt (31.88 KB, text/plain)
2021-10-11 07:29 UTC, Patrick Dung
no flags Details

Description Patrick Dung 2021-10-04 05:26:34 UTC
Description of problem:
The problem exists since FC34.
Sometimes when httpd starts, it core dumps several files but it is able to start up without intervention.

-rw-------  1 root          apache        271429632 Oct  3 14:45 core.httpd-16891-11-1633243502
-rw-------  1 root          apache        179515392 Oct  3 14:45 core.httpd-17782-11-1633243503
-rw-------  1 root          apache        356098048 Oct  3 14:45 core.httpd-18384-11-1633243503
-rw-------  1 root          apache        398430208 Oct  3 14:45 core.httpd-18999-11-1633243504
-rw-------  1 root          apache        440766464 Oct  3 14:45 core.httpd-19383-11-1633243507
-rw-------  1 root          apache        483098624 Oct  3 14:45 core.httpd-19737-11-1633243508
-rw-------  1 root          apache        525430784 Oct  3 14:45 core.httpd-20154-11-1633243508
-rw-------  1 root          apache        567767040 Oct  3 14:45 core.httpd-20633-11-1633243510
-rw-------  1 root          apache        610099200 Oct  3 14:45 core.httpd-21085-11-1633243511
-rw-------  1 root          apache        652435456 Oct  3 14:45 core.httpd-21519-11-1633243512
-rw-------  1 root          apache        694767616 Oct  3 14:45 core.httpd-22906-11-1633243515
-rw-------  1 root          apache        743391232 Oct  3 14:45 core.httpd-24520-11-1633243518
-rw-------  1 root          apache        785727488 Oct  3 14:45 core.httpd-26026-11-1633243520
-rw-------  1 root          apache        828059648 Oct  3 14:45 core.httpd-29431-11-1633243525

Backtrace:

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f0fb6c7d659 in apr_atomic_dec32 () from /lib64/libapr-1.so.0
[Current thread is 1 (Thread 0x7f0fb66e9140 (LWP 16891))]
(gdb) bt
#0  0x00007f0fb6c7d659 in apr_atomic_dec32 () at /lib64/libapr-1.so.0
#1  0x00007f0fb6c90219 in proc_pthread_mutex_unref () at /lib64/libapr-1.so.0
#2  0x00007f0fb6c90cbe in apr_pool_destroy () at /lib64/libapr-1.so.0
#3  0x00007f0fb5b8c52f in clean_child_exit (code=0) at /usr/src/debug/httpd-2.4.49-1.fc35.x86_64/server/mpm/event/event.c:730
#4  0x00007f0fb5b8e99a in child_main (child_num_arg=child_num_arg@entry=0, child_bucket=child_bucket@entry=0)
    at /usr/src/debug/httpd-2.4.49-1.fc35.x86_64/server/mpm/event/event.c:2556
#5  0x00007f0fb5b8f2cf in make_child (s=0x7f0fb6247110, slot=slot@entry=0, bucket=0)
    at /usr/src/debug/httpd-2.4.49-1.fc35.x86_64/server/mpm/event/event.c:2754
#6  0x00007f0fb5b8f371 in startup_children (number_to_start=number_to_start@entry=1)
    at /usr/src/debug/httpd-2.4.49-1.fc35.x86_64/server/mpm/event/event.c:2777
#7  0x00007f0fb5b9013f in event_run (_pconf=<optimized out>, plog=0x7f0fb624b028, s=0x7f0fb6247110)
    at /usr/src/debug/httpd-2.4.49-1.fc35.x86_64/server/mpm/event/event.c:3150
#8  0x00005618346493f8 in ap_run_mpm (pconf=0x7f0fb621f028, plog=0x7f0fb624b028, s=0x7f0fb6247110) at server/mpm_common.c:95
#9  0x0000561834637484 in main (argc=<optimized out>, argv=<optimized out>) at server/main.c:819

Version-Release number of selected component (if applicable):
httpd-2.4.49-1.fc35.x86_64

How reproducible:
It is not like this on every time apache starts.
But it is occurring about two to three times every week.

Steps to Reproduce:
System reboot and systemd starts apache

Comment 1 Patrick Dung 2021-10-04 05:36:51 UTC
Created attachment 1828813 [details]
output of "thread apply all bt full"

Comment 2 Joe Orton 2021-10-08 07:24:28 UTC
Please try httpd 2.4.51 and see if this still occurs.

https://bodhi.fedoraproject.org/updates/FEDORA-2021-2a10bc68a4

Comment 3 Patrick Dung 2021-10-08 16:04:00 UTC
When I upgrade to httpd 2.4.51, FC35 binary has not yet been released/built.
So I use httpd 2.4.51 FC34 rpm on FC35.

It still have core dumps. Here's one of it:

(gdb) thread apply all bt full

Thread 18 (Thread 0x7f77a61fc640 (LWP 1012186)):
#0  0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6
#4  0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6

Thread 17 (Thread 0x7f77a09f1640 (LWP 1012197)):
#0  0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6
#4  0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6

Thread 16 (Thread 0x7f77a39f7640 (LWP 1012191)):
#0  0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6
#4  0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6

Thread 15 (Thread 0x7f77a31f6640 (LWP 1012192)):
#0  0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0
--Type <RET> for more, q to quit, c to continue without paging--c
#3  0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6
#4  0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6

Thread 14 (Thread 0x7f77a19f3640 (LWP 1012195)):
#0  0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6
#4  0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6

Thread 13 (Thread 0x7f77a41f8640 (LWP 1012190)):
#0  0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6
#4  0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6

Thread 12 (Thread 0x7f77a51fa640 (LWP 1012188)):
#0  0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6
#4  0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6

Thread 11 (Thread 0x7f77a21f4640 (LWP 1012194)):
#0  0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6
#4  0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6

Thread 10 (Thread 0x7f77a29f5640 (LWP 1012193)):
#0  0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6
#4  0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6

Thread 9 (Thread 0x7f77a59fb640 (LWP 1012187)):
#0  0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6
#4  0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6

Thread 8 (Thread 0x7f77a8615640 (LWP 1012182)):
#0  0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6
#4  0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6

Thread 7 (Thread 0x7f77a69fd640 (LWP 1012185)):
#0  0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6
#4  0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6

Thread 6 (Thread 0x7f77a79ff640 (LWP 1012183)):
#0  0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6
#4  0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6

Thread 5 (Thread 0x7f77a49f9640 (LWP 1012189)):
#0  0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6
#4  0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6

Thread 4 (Thread 0x7f77a11f2640 (LWP 1012196)):
#0  0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6
#4  0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6

Thread 3 (Thread 0x7f77a71fe640 (LWP 1012184)):
#0  0x00007f77b573d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007f77b573fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007f77b598cc0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6
#4  0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6

Thread 2 (Thread 0x7f77a8e16640 (LWP 1012181)):
#0  0x00007f77b57bb02d in select () at /lib64/libc.so.6
#1  0x00007f77b58f4639 in apr_sleep () at /lib64/libapr-1.so.0
#2  0x00007f77b491ad9e in wd_worker (thread=<optimized out>, data=0x7f77b2db2770) at /usr/src/debug/httpd-2.4.51-1.fc34.x86_64/modules/core/mod_watchdog.c:192
        curr = <optimized out>
        wl = 0x7f77b2db2900
        w = 0x7f77b2db2770
        rv = <optimized out>
        locked = 1
        probed = <optimized out>
        inited = 0
        mpmq_s = 1
        temp_pool = 0x7f77b4fa5028
#3  0x00007f77b5740b17 in start_thread () at /lib64/libc.so.6
#4  0x00007f77b57c56a0 in clone3 () at /lib64/libc.so.6


Thread 1 (Thread 0x7f77b533f140 (LWP 1012180)):
#0  0x00007f77b58dd659 in apr_atomic_dec32 () at /lib64/libapr-1.so.0
#1  0x00007f77b58f0219 in proc_pthread_mutex_unref () at /lib64/libapr-1.so.0
#2  0x00007f77b58f0cbe in apr_pool_destroy () at /lib64/libapr-1.so.0
#3  0x00007f77b47e152f in clean_child_exit (code=0) at /usr/src/debug/httpd-2.4.51-1.fc34.x86_64/server/mpm/event/event.c:730
#4  0x00007f77b47e396a in child_main (child_num_arg=child_num_arg@entry=0, child_bucket=child_bucket@entry=0) at /usr/src/debug/httpd-2.4.51-1.fc34.x86_64/server/mpm/event/event.c:2555
        threads = <optimized out>
        rv = <optimized out>
        ts = <optimized out>
        thread_attr = 0x7f77b2e465e8
        start_thread_id = 0x7f77b2e46648
        i = <optimized out>
#5  0x00007f77b47e429f in make_child (s=0x7f77b4e47110, slot=slot@entry=0, bucket=bucket@entry=0) at /usr/src/debug/httpd-2.4.51-1.fc34.x86_64/server/mpm/event/event.c:2753
        pid = <optimized out>
#6  0x00007f77b47e5269 in server_main_loop (num_buckets=1, remaining_children_to_start=1) at /usr/src/debug/httpd-2.4.51-1.fc34.x86_64/server/mpm/event/event.c:3052
        ps = <optimized out>
        status = 11
        pid = {pid = 1006218, in = 0x7f77b2d4ab00, out = 0x2e3836312e323931, err = 0x38392e32}
        i = <optimized out>
        child_slot = 0
        exitwhy = (APR_PROC_SIGNAL | APR_PROC_SIGNAL_CORE)
        processed_status = 0
        num_buckets = 1
        remaining_children_to_start = <optimized out>
        i = <optimized out>
#7  event_run (_pconf=<optimized out>, plog=<optimized out>, s=<optimized out>) at /usr/src/debug/httpd-2.4.51-1.fc34.x86_64/server/mpm/event/event.c:3171
        num_buckets = 1
        remaining_children_to_start = <optimized out>
        i = <optimized out>
#8  0x000055f66ebe2428 in ap_run_mpm (pconf=0x7f77b4e1f028, plog=0x7f77b4e4b028, s=0x7f77b4e47110) at server/mpm_common.c:95
        pHook = <optimized out>
        n = 0
        rv = -1
#9  0x000055f66ebd0484 in main (argc=<optimized out>, argv=<optimized out>) at server/main.c:819
        c = 102 'f'
        showcompile = <optimized out>
        showdirectives = 0
        confname = 0x7fff37458ea2 "/etc/httpd-web1/conf/httpd.conf"
        def_server_root = 0x55f66ec1d761 "/etc/httpd"
        temp_error_log = <optimized out>
        error = <optimized out>
        process = 0x7f77b4e1d118
        pconf = <optimized out>
        plog = 0x7f77b4e4b028
        ptemp = 0x7f77b4e76028
        pcommands = 0x7f77b4e41028
        opt = 0x7f77b4e41118
        rv = <optimized out>
        mod = <optimized out>
        opt_arg = 0x7fff37458ea2 "/etc/httpd-web1/conf/httpd.conf"
        signal_server = <optimized out>
        rc = <optimized out>


Now I see httpd-2.4.51 FC35 rpm is out in koji. I would install it.

Comment 4 Patrick Dung 2021-10-10 06:24:56 UTC
It is still crashing with the httpd 2.4.51 FC35 rpm:

Core was generated by `/usr/sbin/httpd -DFOREGROUND -f /etc/httpd-web1/conf/httpd.conf'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007ff05da33659 in apr_atomic_dec32 () from /lib64/libapr-1.so.0
[Current thread is 1 (Thread 0x7ff05d49f140 (LWP 19523))]
(gdb)
(gdb)
(gdb) thread apply all bt full

Thread 18 (Thread 0x7ff0568d6640 (LWP 19527)):
#0  0x00007ff05d91b02d in select () at /lib64/libc.so.6
#1  0x00007ff05da4a639 in apr_sleep () at /lib64/libapr-1.so.0
#2  0x00007ff05c98dd9e in wd_worker (thread=<optimized out>, data=0x7ff05af56770) at /usr/src/debug/httpd-2.4.51-1.fc35.x86_64/modules/core/mod_watchdog.c:192
        curr = <optimized out>
        wl = 0x7ff05af56900
        w = 0x7ff05af56770
        rv = <optimized out>
        locked = 1
        probed = <optimized out>
        inited = 0
        mpmq_s = 1
        temp_pool = 0x7ff05d17a028
#3  0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6
#4  0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6

Thread 17 (Thread 0x7ff0533fb640 (LWP 19533)):
#0  0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6
#4  0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6

Thread 16 (Thread 0x7ff051bf8640 (LWP 19536)):
#0  0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6
#4  0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6

Thread 15 (Thread 0x7ff04ebf2640 (LWP 19542)):
--Type <RET> for more, q to quit, c to continue without paging--c
#0  0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6
#4  0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6

Thread 14 (Thread 0x7ff04f3f3640 (LWP 19541)):
#0  0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6
#4  0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6

Thread 13 (Thread 0x7ff04fbf4640 (LWP 19540)):
#0  0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6
#4  0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6

Thread 12 (Thread 0x7ff0503f5640 (LWP 19539)):
#0  0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6
#4  0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6

Thread 11 (Thread 0x7ff050bf6640 (LWP 19538)):
#0  0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6
#4  0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6

Thread 10 (Thread 0x7ff0513f7640 (LWP 19537)):
#0  0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6
#4  0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6

Thread 9 (Thread 0x7ff0523f9640 (LWP 19535)):
#0  0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6
#4  0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6

Thread 8 (Thread 0x7ff052bfa640 (LWP 19534)):
#0  0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6
#4  0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6

Thread 7 (Thread 0x7ff053bfc640 (LWP 19532)):
#0  0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6
#4  0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6

Thread 6 (Thread 0x7ff0543fd640 (LWP 19531)):
#0  0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6
#4  0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6

Thread 5 (Thread 0x7ff04e3f1640 (LWP 19543)):
#0  0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6
#4  0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6

Thread 4 (Thread 0x7ff054bfe640 (LWP 19530)):
#0  0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6
#4  0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6

Thread 3 (Thread 0x7ff0553ff640 (LWP 19529)):
#0  0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6
#4  0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6

Thread 2 (Thread 0x7ff0560d5640 (LWP 19528)):
#0  0x00007ff05d89d77a in __futex_abstimed_wait_common () at /lib64/libc.so.6
#1  0x00007ff05d89fef0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6
#2  0x00007ff05dae2c0f in thread_pool_func () at /lib64/libaprutil-1.so.0
#3  0x00007ff05d8a0b17 in start_thread () at /lib64/libc.so.6
#4  0x00007ff05d9256a0 in clone3 () at /lib64/libc.so.6


Thread 1 (Thread 0x7ff05d49f140 (LWP 19523)):
#0  0x00007ff05da33659 in apr_atomic_dec32 () at /lib64/libapr-1.so.0
#1  0x00007ff05da46219 in proc_pthread_mutex_unref () at /lib64/libapr-1.so.0
#2  0x00007ff05da46cbe in apr_pool_destroy () at /lib64/libapr-1.so.0
#3  0x00007ff05c94a52f in clean_child_exit (code=0) at /usr/src/debug/httpd-2.4.51-1.fc35.x86_64/server/mpm/event/event.c:730
#4  0x00007ff05c94c96a in child_main (child_num_arg=child_num_arg@entry=0, child_bucket=child_bucket@entry=0) at /usr/src/debug/httpd-2.4.51-1.fc35.x86_64/server/mpm/event/event.c:2555
        threads = <optimized out>
        rv = <optimized out>
        ts = <optimized out>
        thread_attr = 0x7ff05afc85e8
        start_thread_id = 0x7ff05afc8648
        i = <optimized out>
#5  0x00007ff05c94d29f in make_child (s=0x7ff05d047110, slot=slot@entry=0, bucket=0) at /usr/src/debug/httpd-2.4.51-1.fc35.x86_64/server/mpm/event/event.c:2753
        pid = <optimized out>
#6  0x00007ff05c94d341 in startup_children (number_to_start=number_to_start@entry=1) at /usr/src/debug/httpd-2.4.51-1.fc35.x86_64/server/mpm/event/event.c:2776
        i = 0
#7  0x00007ff05c94e14f in event_run (_pconf=<optimized out>, plog=0x7ff05d04b028, s=0x7ff05d047110) at /usr/src/debug/httpd-2.4.51-1.fc35.x86_64/server/mpm/event/event.c:3152
        num_buckets = 1
        remaining_children_to_start = 1
        i = <optimized out>
#8  0x0000560e5a05a428 in ap_run_mpm (pconf=0x7ff05d01f028, plog=0x7ff05d04b028, s=0x7ff05d047110) at server/mpm_common.c:95
        pHook = <optimized out>
        n = 0
        rv = -1
#9  0x00007ff05d01f028 in  ()
#10 0x00007ff05d01d118 in  ()
#11 0x0000560e5a0ba4a0 in ap_conftree ()
#12 0x00007ffecf2916c0 in  ()
#13 0x0000560e5a0957b3 in  ()
#14 0x0000560e5a048484 in main (argc=<optimized out>, argv=<optimized out>) at server/main.c:819
        c = 93 ']'
        showcompile = <optimized out>
        showdirectives = 1552159360
        confname = 0x71f476b4 <error: Cannot access memory at address 0x71f476b4>
        def_server_root = 0x0
        temp_error_log = <optimized out>
        error = <optimized out>
        process = 0x7ff05d01f028
        pconf = <optimized out>
        plog = 0x7
        ptemp = 0xf4908b9c7cd56700
        pcommands = 0x7ff05c841280 <cgi_handle_exec>
        opt = 0x14
        rv = <optimized out>
        mod = <optimized out>
        opt_arg = 0x0
        signal_server = <optimized out>
        rc = <optimized out>

Comment 5 Joe Orton 2021-10-11 07:13:31 UTC
Thanks for testing further.  Could you do

# debuginfo-install apr glibc

and regenerate the backtraces?

Comment 6 Patrick Dung 2021-10-11 07:29:23 UTC
Created attachment 1831718 [details]
output for httpd 2.4.51.txt

For https://bugzilla.redhat.com/show_bug.cgi?id=2010166#c5

Comment 7 Ben Cotton 2022-11-29 17:06:53 UTC
This message is a reminder that Fedora Linux 35 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 35 on 2022-12-13.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '35'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 35 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 8 Ben Cotton 2022-12-13 15:38:00 UTC
Fedora Linux 35 entered end-of-life (EOL) status on 2022-12-13.

Fedora Linux 35 is no longer maintained, which means that it
will not receive any further security or bug fix updates. As a result we
are closing this bug.

If you can reproduce this bug against a currently maintained version of Fedora Linux
please feel free to reopen this bug against that version. Note that the version
field may be hidden. Click the "Show advanced fields" button if you do not see
the version field.

If you are unable to reopen this bug, please file a new report against an
active release.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.