Created attachment 492313 [details]
PCRE pattern for reproducing

Created attachment 492314 [details]
A lot of 'd's

This problem can be reproduced outside of PHP as well, through
"pcregrep -f pascal d"

This issue looks like a regression to me.

I can reproduce it (using the "pcregrep" test) with pcre-6.6-6.el5_6.1 and 
pcre-6.6-6.el5, but not with pcre-6.6-2.el5_1.7.

It looks like an infinite recursion in match():

Program received signal SIGSEGV, Segmentation fault.
0x00002aaaaaad33d8 in match (
    eptr=0x7fffffff7635 'd' <repeats 28 times>, "\r\n", ecode=0x6068e8 ";",
    offset_top=2, md=0x7fffffff5880, ims=0, eptrb=0x7fffff3ff420, flags=2,
    rdepth=13517) at ./pcre_exec.c:378
378     {
(gdb) where
#0  0x00002aaaaaad33d8 in match (
    eptr=0x7fffffff7635 'd' <repeats 28 times>, "\r\n", ecode=0x6068e8 ";", 
    offset_top=2, md=0x7fffffff5880, ims=0, eptrb=0x7fffff3ff420, flags=2, 
    rdepth=13517) at ./pcre_exec.c:378
#1  0x00002aaaaaad3a8e in match (eptr=<value optimized out>, 
    ecode=<value optimized out>, offset_top=2, md=0x7fffffff5880, ims=0, 
    eptrb=0x7fffff3ff420, flags=<value optimized out>, rdepth=13516)
    at ./pcre_exec.c:647
#2  0x00002aaaaaad3a0b in match (eptr=<value optimized out>, 
    ecode=<value optimized out>, offset_top=2, md=0x7fffffff5880, ims=0, 
    eptrb=0x7fffff3ffb60, flags=<value optimized out>, rdepth=13515)
    at ./pcre_exec.c:1156
#3  0x00002aaaaaad3a8e in match (eptr=<value optimized out>,
    ecode=<value optimized out>, offset_top=2, md=0x7fffffff5880, ims=0,
    eptrb=0x7fffff3ffb60, flags=<value optimized out>, rdepth=13514)
    at ./pcre_exec.c:647
#4  0x00002aaaaaad3a0b in match (eptr=<value optimized out>,
    ecode=<value optimized out>, offset_top=2, md=0x7fffffff5880, ims=0,
    eptrb=0x7fffff4002a0, flags=<value optimized out>, rdepth=13513)
    at ./pcre_exec.c:1156
[...]
#13517 0x00002aaaaaad3a8e in match (eptr=<value optimized out>, ecode=<value optimized out>, offset_top=2, md=0x7fffffff5880, ims=0, eptrb=0x7fffffff57a0, flags=<value optimized out>, rdepth=0) at ./pcre_exec.c:647
#13518 0x00002aaaaaad9e2c in pcre_exec (argument_re=<value optimized out>, extra_data=<value optimized out>, subject=<value optimized out>, length=<value optimized out>, start_offset=<value optimized out>, options=<value optimized out>, offsets=0x7fffffff5a40, offsetcount=99) at ./pcre_exec.c:3765
#13519 0x0000000000401993 in pcregrep (in=0x606670, printname=0x0) at ./pcregrep.c:627
#13520 0x00000000004025dd in grep_or_recurse (pathname=0x0, dir_recurse=0, only_one_at_top=1) at ./pcregrep.c:1001
#13521 0x0000000000403413 in main (argc=4, argv=0x7fffffffe608) at ./pcregrep.c:1711

When I tested the recursion of the on crashing version actually went a few levels deeper then the one that crashed.

See:
Breakpoint 3, match (eptr=0xbeda69 "", ecode=0xbf31cb "\024B\002,",
offset_top=2, md=0x7fffffffbda0, ims=4, eptrb=0x7fffff7e87e0, flags=0,
rdepth=13572) at ./pcre_exec.c:378

The bad version crashed at rdepth=13545

Something is different in Brew build environment as the same source compiled in Fedora works. I guess this is caused by gcc version/flags as this segfault is caused by exhausting stack while nesting call frames at matching implemented as recursion.

I can change implementation to non-recursive (there is a pcre compile time option) or customer can increase stack size limit (ulimit -s). Boundary of provided use case is 10300 KiB in RHEL-5.7.

I guess this rises a few questions

Why did we change the defaults in a minor version? Should that be undone?

Is this really a regression?

Is this a problem that will happen 'in the wild' more often?

Will it happen for other software as well?

Why can pcre not fail gracefully?

(In reply to comment #6)
> I guess this rises a few questions
> 
> Why did we change the defaults in a minor version? Should that be undone?
> 
What defaults? The only thing we done in pcre was enabling Unicode properties (bug #457064) and adding a patch that fixed a bug in non-utf8 mode pattern compilation (bug #680962).

> Is this really a regression?
> 
Actually, it isn't. The size maximal recursion depth depends on stack size and fatness of recursive function. The size of the function can vary according to architecture, compiler and compile options used.


> Is this a problem that will happen 'in the wild' more often?
> 
Yes, we hit similar problem in latest Fedora on s390x, but it disappeared when upgrading compiler. E.g. Gentoo allows user to select which implementation he wants. IMHO the non-recursive one has some performance issues because it needs to track memory allocations instead of moving stack pointer (especially when back-tracking mismatched subpattern).

> Will it happen for other software as well?
> 
Yes.

> Why can pcre not fail gracefully?
>
Because the only way is to install alternative segmentation fault stack to print some fancy message and die either (because unprivileged user cannot increase ulimits). And there can be only one segfault handler and alternative stack that application would like to install too (e.g. Xorg does that).

I will check former SRPMs compiled in current Brew, try to disable the recursion etc. to make sure the only problem is stack size.

x86 build                    lowest working stack size in KiB
-------------------------------------------------------------
original pcre-6.6-6.el5_6.1: 10300
original pcre-6.6-2.el5_1.7:  6268
RHEL-5.7 pcre-6.6-6.el5_6.1: 10300
RHEL-5.7 pcre-6.6-2.el5_1.7:  6268
nonrecur pcre-6.6-6.el5_6.1:    32 (lower causes sh: argument too long)

So toolchain has not changed.

x86 build                     time in ms (average of 5 attempts)
----------------------------------------------------------------
RHEL-5.7 pcre-6.6-6.el5_6.1:  42
nonrecur pcre-6.6-6.el5_6.1:  92

This is reason why recursive implementation is preferred.

(In reply to comment #7)
> (In reply to comment #6)
> > Why can pcre not fail gracefully?
> >
> Because the only way is to install alternative segmentation fault stack to
> print some fancy message and die either (because unprivileged user cannot
> increase ulimits). And there can be only one segfault handler and alternative
> stack that application would like to install too (e.g. Xorg does that).
> 
> I will check former SRPMs compiled in current Brew, try to disable the
> recursion etc. to make sure the only problem is stack size.

Actually there is compile time option --with-match-limit-recursion=10000000 defining default recursion limit (this is not memory limit). Application can change the limit at run time on pcre_exec(, extra, ...) call in extra->match_limit_recursion field. See pcre_exec(3).

Developer can use pcretest(1) with `-M' option to figure out needed recursion depth if recursive implementation is compiled in. Otherwise it returns needed amount of heap memory.

Per comment #9, I'm tempted to close this report as not-a-bug. Application can limit recursion depth, administrator can increase stack size in runt-time limits. Compiling library with heap-based algorithm would introduce significant performance lost. It's natural that library compiled with additional features (Unicode properties) consumes more stack per frame.

This request was evaluated by Red Hat Product Management for inclusion in Red Hat Enterprise Linux 5.7 and Red Hat does not plan to fix this issue the currently developed update.

Contact your manager or support representative in case you need to escalate this bug.

Per pcre documentation, recommended solution is to increase stack size or set stack nesting limit appropriately. This is known feature of pcre when compiled with stack implementation. Closing as not-a-bug.