Bug 1298551 - Segmentation fault in modperl_wbucket_flush
Summary: Segmentation fault in modperl_wbucket_flush
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: mod_perl
Version: 6.7
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: perl-maint-list
QA Contact: BaseOS QE - Apps
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-01-14 12:20 UTC by Martin Frodl
Modified: 2017-10-17 07:36 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-10-17 07:36:25 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Martin Frodl 2016-01-14 12:20:33 UTC
Description of problem:

When running Apache upstream test toolkit with mod_perl and mod_auth_kerb installed, there are occasional segfaults in mod_perl.

Version-Release number of selected component (if applicable):
mod_perl-2.0.4-11.el6_5.x86_64

How reproducible:
occasionally

Steps to Reproduce:
# yum -y install httpd mod_perl mod_auth_kerb subversion perl-Test-Simple
# svn co https://svn.apache.org/repos/asf/perl/Apache-Test/trunk
# chown apache:apache trunk
# cd trunk
# sudo -u apache perl Makefile.PL -apxs /usr/bin/apxs
# make
# sudo -u apache make test

Actual results:
Segmentation fault.

Expected results:
All tests pass, no segmentation faults.

Comment 6 Petr Pisar 2016-11-18 13:12:33 UTC
I can reproduce it with:

httpd-2.2.15-53.el6.x86_64
mod_auth_kerb-5.4-14.el6.x86_64
mod_perl-2.0.4-11.el6_5.x86_64
perl-5.10.1-141.el6_7.1.x86_64

It cashes within t/more/04testmore.t test:

$ prove -b -v t/more/04testmore.t
t/more/04testmore.t .. request has failed (the response code was: 500)
see t/logs/error_log for more details
[  error] oh rats, server dumped core
[  error] for stacktrace, run: gdb /usr/sbin/httpd -core /tmp/trunk/t/core.1861
Dubious, test returned 111 (wstat 28416, 0x6f00)

Comment 7 Petr Pisar 2016-11-18 14:58:31 UTC
Actually it crashes randomly and sometimes sooner.

Smaller reproducer in the trunk tree cloned into /tmp/trunk as apache user:

(1) Build the test suite, especially the configuration for the httpd:
$ perl Makefile.PL
$ make

(2) Start the httpd, it will deamonize:
$ /usr/sbin/httpd  -d /tmp/trunk/t -f /tmp/trunk/t/conf/httpd.conf -D APACHE2 -D PERL_USEITHREADS

(3) Run some tests (from t/alltest/all.t to t/more/04testmore.t):
$ while (prove -b t/alltest/all.t t/alltest2/all.t t/bad_coding.t t/cookies.t t/import.t t/log_watch.t t/log_watch_for_broken_lines.t t/more/01testpm.t t/more/02testmore.t t/more/03testpm.t t/more/04testmore.t); do :;done

When the bug emerges, a test will report that the server crashed. If you created /tmp/trunk/logs directory, you can see in the /tmp/trunk/logs/error_log:

[Fri Nov 18 15:40:58 2016] [notice] child pid 22674 exit signal Segmentation fault (11), possible coredump in /tmp/trunk/t
[Fri Nov 18 15:41:11 2016] [debug] proxy_util.c(1909): proxy: grabbed scoreboard slot 0 in child 22754 for worker proxy:reverse
[Fri Nov 18 15:41:11 2016] [debug] proxy_util.c(1929): proxy: worker proxy:reverse already initialized
[Fri Nov 18 15:41:11 2016] [debug] proxy_util.c(2025): proxy: initialized single connection worker 0 in child 22754 for (*)

It really crashes randomly and very rarely, even with httpd-2.2.15-55.el6_8.2.x86_64.

Comment 8 Petr Pisar 2016-11-18 15:22:34 UTC
Back trace:

#0  0x00007f3fdebf81c3 in PerlIOApache_flush (my_perl=0x7f3feb6cbc30, f=0x7f3feb8db760) at modperl_io_apache.c:167
#1  0x00007f3fde989b15 in Perl_PerlIO_flush (my_perl=0x7f3feb6cbc30, f=<value optimized out>) at perlio.c:1669
#2  0x00007f3fde98a4da in PerlIOBase_close (my_perl=0x7f3feb6cbc30, f=0x7f3feb8db760) at perlio.c:2177
#3  0x00007f3fdebf8239 in PerlIOApache_close (my_perl=<value optimized out>, f=0x7f3feb8db760) at modperl_io_apache.c:189
#4  0x00007f3fde98a5f8 in PerlIO__close (my_perl=<value optimized out>, f=<value optimized out>) at perlio.c:1419
#5  0x00007f3fde98b88f in Perl_PerlIO_close (my_perl=0x7f3feb6cbc30, f=0x7f3feb8db760) at perlio.c:1432
#6  0x00007f3fde96caeb in Perl_do_openn (my_perl=0x7f3feb6cbc30, gv=0x7f3fecb0de30, oname=0x7f3fecaf0a80 ">&STDOUT", len=8, as_raw=0, rawmode=0, rawperm=0, supplied_fp=0x0, 
    svp=0x7f3feca69638, num_svs=0) at doio.c:125
#7  0x00007f3fde963aff in Perl_pp_open (my_perl=0x7f3feb6cbc30) at pp_sys.c:560
#8  0x00007f3fde914b06 in Perl_runops_standard (my_perl=0x7f3feb6cbc30) at run.c:40
#9  0x00007f3fde8bc5df in Perl_call_sv (my_perl=0x7f3feb6cbc30, sv=0x7f3fecb5caf8, flags=4) at perl.c:2721
#10 0x00007f3fdebf30be in modperl_callback (my_perl=0x7f3feb6cbc30, handler=0x7f3feb7d8718, p=0x7f3feb92f2a8, r=0x7f3feb92f328, s=0x7f3feb65f870, args=0x7f3fec9a9bc8)
    at modperl_callback.c:101
#11 0x00007f3fdebf380b in modperl_callback_run_handlers (idx=6, type=<value optimized out>, r=0x7f3feb92f328, c=<value optimized out>, s=0x7f3feb65f870, pconf=<value optimized out>, 
    plog=0x0, ptemp=0x0, run_mode=MP_HOOK_RUN_FIRST) at modperl_callback.c:262
#12 0x00007f3fdebf3e0f in modperl_callback_per_dir (idx=<value optimized out>, r=<value optimized out>, run_mode=<value optimized out>) at modperl_callback.c:369
#13 0x00007f3fdebed75f in modperl_response_handler_run (r=0x7f3feb92f328, finish=0) at mod_perl.c:1000
#14 0x00007f3fdebed913 in modperl_response_handler_cgi (r=0x7f3feb92f328) at mod_perl.c:1100
#15 0x00007f3fea4adfc0 in ap_run_handler (r=0x7f3feb92f328) at /usr/src/debug/httpd-2.2.15/server/config.c:158
#16 0x00007f3fea4b187e in ap_invoke_handler (r=0x7f3feb92f328) at /usr/src/debug/httpd-2.2.15/server/config.c:376
#17 0x00007f3fea4bcfd0 in ap_process_request (r=0x7f3feb92f328) at /usr/src/debug/httpd-2.2.15/modules/http/http_request.c:282
#18 0x00007f3fea4b9e18 in ap_process_http_connection (c=0x7f3feb91b478) at /usr/src/debug/httpd-2.2.15/modules/http/http_core.c:190
#19 0x00007f3fea4b5ae8 in ap_run_process_connection (c=0x7f3feb91b478) at /usr/src/debug/httpd-2.2.15/server/connection.c:43
#20 0x00007f3fea4c1d77 in child_main (child_num_arg=<value optimized out>) at /usr/src/debug/httpd-2.2.15/server/mpm/prefork/prefork.c:670
#21 0x00007f3fea4c2099 in make_child (s=0x7f3feb65f870, slot=0) at /usr/src/debug/httpd-2.2.15/server/mpm/prefork/prefork.c:773
#22 0x00007f3fea4c23cb in startup_children (_pconf=<value optimized out>, plog=<value optimized out>, s=<value optimized out>)
    at /usr/src/debug/httpd-2.2.15/server/mpm/prefork/prefork.c:791
#23 ap_mpm_run (_pconf=<value optimized out>, plog=<value optimized out>, s=<value optimized out>) at /usr/src/debug/httpd-2.2.15/server/mpm/prefork/prefork.c:1012
#24 0x00007f3fea499aa0 in main (argc=9, argv=0x7ffe91f1e8b8) at /usr/src/debug/httpd-2.2.15/server/main.c:763

It crashed in mod_perl's src/modules/perl/modperl_io_apache.c:167:

    rcfg = modperl_config_req_get(st->r);

→   MP_CHECK_WBUCKET_INIT("flush");

The MP_CHECK_WBUCKET_INIT macro does:

/* check whether the response phase has been initialized already */
#define MP_CHECK_WBUCKET_INIT(func) \
    if (!rcfg->wbucket) { \
        Perl_croak(aTHX_ "%s: " func " can't be called "  \
                   "before the response phase", MP_FUNC); \
    }

The rcfg is NULL as disassembly and CPU registers confirm.

Comment 9 Red Hat Bugzilla Rules Engine 2017-10-17 07:36:25 UTC
Development Management has reviewed and declined this request. You may appeal this decision by reopening this request.


Note You need to log in before you can comment on or make changes to this bug.