Bug 1215701 - mod_security with --enable-pcre-jit breaks
Summary: mod_security with --enable-pcre-jit breaks
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: mod_security
Version: 21
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Daniel Kopeček
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-04-27 14:01 UTC by Harald Reindl
Modified: 2016-09-03 15:51 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-12-02 19:40:53 UTC
Type: Bug


Attachments (Terms of Use)
modsec SPEC (3.21 KB, text/x-matlab)
2015-04-28 13:03 UTC, Harald Reindl
no flags Details
src.rpm (4.05 MB, application/x-rpm)
2015-05-04 10:43 UTC, Harald Reindl
no flags Details

Description Harald Reindl 2015-04-27 14:01:40 UTC
while on Fedora mod_security is not build with --enable-pcre-jit it should not crash httpd on F21 while not on F20

[Mon Apr 27 11:09:23.113259 2015] [core:notice] [pid 118704] AH00052: child pid 118706 exit signal Segmentation fault (11)

open("/dev/urandom", O_RDONLY)          = 13
read(13, "\200\376%\23\240\236\n9\373\210\243%i\223\367J\210\271\177\26\207\331\177\r\372RB\342\2223\352="..., 64) = 64
close(13)                               = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fcf70b39000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fcf70b37000
munmap(0x7fcf70b37000, 8192)            = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x8} ---
chdir("/etc/httpd")                     = 0
rt_sigaction(SIGSEGV, {SIG_DFL, [], SA_RESTORER|SA_INTERRUPT, 0x7fcf754c70d0}, {SIG_DFL, [], SA_RESTORER|SA_RESETHAND, 0x7fcf754c70d0}, 8) = 0
kill(118706, SIGSEGV)                   = 0
rt_sigreturn({mask=[]})                 = 32
--- SIGSEGV {si_signo=SIGSEGV, si_code=SI_USER, si_pid=118706, si_uid=48} ---
+++ killed by SIGSEGV +++ 
_____________________________________________

it turned out to be

* mod_security
* built with --enable-pcre-jit
* path "/plugins/jqueryui/js/i18n/jquery.ui.datepicker-de.js"
* it don't matter if the file exists
* replace ".ui." with "XuiX" and it don't crash

that smells to be a bug in pcre since the identical build on Fedora 20 had no problem

pcre-8.33-9.fc20.x86_64
pcre-8.35-10.fc21.x86_64

Comment 1 Petr Pisar 2015-04-28 11:48:53 UTC
Please provide reproducer and exact versions of involved packages.

I installed:

httpd-2.4.10-15.fc21.x86_64
mod_security-2.8.0-5.fc21.x86_64
pcre-8.35-10.fc21.x86_64

sterted httpd systemd service and exectued:

$ wget --no-proxy http://localhost/plugins/jqueryui/js/i18n/jquery.ui.datepicker-de.js

httpd did not crash.

Comment 2 Petr Pisar 2015-04-28 11:58:16 UTC
I see, this crash occurs after rebuilding mod_security with --enable-pcre-jit configure option:

Apr 28 13:55:04 dhcp-0-146 systemd: Started The Apache HTTP Server.
Apr 28 13:55:05 dhcp-0-146 dbus[1129]: [system] Successfully activated service 'org.fedoraproject.Setroubleshootd'
Apr 28 13:55:07 dhcp-0-146 dhclient[1720]: bound to 10.34.4.180 -- renewal in 38573 seconds.
Apr 28 13:55:08 dhcp-0-146 org.fedoraproject.Setroubleshootd: 'list' object has no attribute 'split'
Apr 28 13:55:08 dhcp-0-146 setroubleshoot: Plugin Exception restorecon_source
Apr 28 13:55:08 dhcp-0-146 setroubleshoot: SELinux is preventing /usr/sbin/httpd from using the execmem access on a process. For complete SELinux messages. run sealert -l b1af2699-2c1c
-4d8f-9600-d7eaf86505ce
Apr 28 13:55:08 dhcp-0-146 python: SELinux is preventing /usr/sbin/httpd from using the execmem access on a process.

*****  Plugin catchall_boolean (89.3 confidence) suggests   ******************

If you want to allow httpd scripts and modules execmem/execstack
Then you must tell SELinux about this by enabling the 'httpd_execmem' boolean.
You can read 'httpd_selinux' man page for more details.
Do
setsebool -P httpd_execmem 1

*****  Plugin catchall (11.6 confidence) suggests   **************************

If you believe that httpd should be allowed execmem access on processes labeled httpd_t by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# grep httpd /var/log/audit/audit.log | audit2allow -M mypol
# semodule -i mypol.pp

Comment 3 Petr Pisar 2015-04-28 12:04:01 UTC
This is not a bug in pcre. SELinux policy prevents httpd process from writing into executable memory page which is needed to store JIT-compiled matching code.

Either enable the httpd_execmem SELinux boolean as documented in the system log error, or ask selinux-policy package to change default SELinux policy. (There was similar surprise in cupsd after enabling PCRE JIT in grep which is called from CUPS filters.)

Comment 4 Harald Reindl 2015-04-28 12:36:05 UTC
> after rebuilding mod_security with --enable-pcre-jit

hence i wrote *that* into the subject and description

> This is not a bug in pcre. SELinux policy prevents httpd process

nonsense - none of my machines crashing on Fedora 21 with --enable-pcre-jit has SELinux enabled and the point is that it only crahs on specific URL like "/plugins/jqueryui/js/i18n/jquery.ui.datepicker-de.js"

please read the original bugreport again and that time careful

Comment 5 Petr Pisar 2015-04-28 12:38:51 UTC
On the only machine I have, httpd does not crash after switching httpd_execmem SELinux boolean to 1.

Comment 6 Petr Pisar 2015-04-28 12:41:01 UTC
And if you want be rude, then provide pure PCRE reproducer using pcretest(1) tool. Otherwise it's mod_security problem.

Comment 7 Harald Reindl 2015-04-28 12:49:53 UTC
i got rude because you started with default packages until 10 minutes later read the subejct of the bugreport and frankly when i take the identical src.rpm *with* --enable-pcre-jit on a Fedora 20 systems where mod_security 2.9.0 runs for weeks now in production it's hardly the mod-security code but the newer pcre-version in F21

you can now insist in it's not a pcre-problem, close the bugreport and hope that the root cause won't hit other packages too while i continue to build mod_security without pcre-jit, my problem which drove me crazy for weeks is solved

mod_security-2.9.0-5.fc21.20150427.rh.x86_64

Comment 8 Petr Pisar 2015-04-28 12:58:23 UTC
I don't say there is no bug in PCRE. I say that the only crash after enabling JIT in mod_security I can reproduce is the SELinux denial. I'm not going to argue with you. Until you show me pcre-only reproducer, I have nothing to fix.

Comment 9 Harald Reindl 2015-04-28 13:03:08 UTC
Created attachment 1019669 [details]
modsec SPEC

well, do what you want, "show me pcre-only reproducer" in context of a library makes not much sense, if you change your mind here is the full SPEC file, until that's fixed i just stay at the last changelog entry on F21+

Comment 10 Petr Pisar 2015-04-28 13:27:36 UTC
(In reply to Harald Reindl from comment #9)
> Created attachment 1019669 [details]
> modsec SPEC
> 
> well, do what you want, "show me pcre-only reproducer" in context of a
> library makes not much sense,

It makes sense (there is pcretest(1) tool, or you can write a few lines of C code yourself, pcreapi(3) is very detailed). I'm not going to dissect tons for httpd and mod_security code full of forks and thread switches.

If you are not able to provide reproducible reproducer, maybe mod_security people will help you.

Comment 11 Harald Reindl 2015-05-04 10:26:43 UTC
mod_security is innconect, the segfaults are starting with the first 8.35 build

no segfaults:
May 04 12:16:04 Installed: pcre-8.34-1.fc21.x86_64
May 04 12:16:05 Installed: pcre-devel-8.34-1.fc21.x86_64
May 04 12:20:21 Installed: pcre-8.34-2.fc21.x86_64
May 04 12:20:21 Installed: pcre-devel-8.34-2.fc21.x86_64
May 04 12:22:59 Updated: pcre-8.34-4.fc21.x86_64
May 04 12:22:59 Updated: pcre-devel-8.34-4.fc21.x86_64

segfaults starting with:
May 04 12:23:54 Updated: pcre-8.35-1.fc21.x86_64
May 04 12:23:54 Updated: pcre-devel-8.35-1.fc21.x86_64

Comment 12 Petr Pisar 2015-05-04 10:32:17 UTC
As I said, it works for me. Bring me clear reproducer, i.e. regular expression and subject strict which segfaults.

Comment 13 Harald Reindl 2015-05-04 10:43:27 UTC
Created attachment 1021663 [details]
src.rpm

just do "rpmbuilld --rebuild" with the attached src.rpm file and *you have* a reproducer

Comment 14 Harald Reindl 2015-05-04 10:44:27 UTC
and frankly stop to re-assign this bug to mod_security - in case of a SEGFAULT after upgrade a library the caller NEVER is responsible

Comment 15 Petr Pisar 2015-05-04 11:51:43 UTC
Harald,

(1) building mod_security is not a pcre reproducer.
(2) building the mod_security works for me.
(3) starting httpd.service with "LoadModule security2_module modules/mod_security2.so" works for me.
(4) getting <http://localhost/plugins/jqueryui/js/i18n/jquery.ui.datepicker-de.js> does not causes any crash or error message for me:

[Mon May 04 13:43:18.557931 2015] [:notice] [pid 14153] ModSecurity for Apache/2.9.0 (http://www.modsecurity.org/) configured.
[Mon May 04 13:43:18.557939 2015] [:notice] [pid 14153] ModSecurity: APR compiled version="1.5.1"; loaded version="1.5.1"
[Mon May 04 13:43:18.557944 2015] [:notice] [pid 14153] ModSecurity: PCRE compiled version="8.35 "; loaded version="8.35 2014-04-04"
[Mon May 04 13:43:18.557971 2015] [:notice] [pid 14153] ModSecurity: LIBXML compiled version="2.9.1"
[Mon May 04 13:43:18.557976 2015] [:notice] [pid 14153] ModSecurity: Status engine is currently disabled, enable it by set SecStatusEngine to On.

(5) I said provide me faulting regexp and subject string. Anything else is not a PCRE reproducer.

I'm sorry you system does not work for you and you cannot provide pcre reproducer if you file a pcre bug report.

Either I will close this as WORKSFORME against pcre, or keep this opened against mod_security as you cannot provide any reproducer.

Comment 16 Harald Reindl 2015-05-04 12:24:48 UTC
well, how should i know which of the rules triggering the segfault - until now you even refused to try recent mod_security and instead all the time close and re-assign the bugreport you could also wait for responses

* NOW we know: modsecurity 2.9.0 alone don#t segfault
* so the next step would have been asking for my ruleset
  which is not the default one for a million of reasons

surely, you can insist in someone showing a simple regex which leads to the segfault, but i am long enough developer (not C/C++ but developer) to know that this leads nowhere

well, at least as maintainer of a core-package you should throw out a big warning to not enable pcre-jit in any fedora package for security reasons

Comment 17 Petr Pisar 2015-05-04 13:10:04 UTC
(In reply to Harald Reindl from comment #16)
> well, how should i know which of the rules triggering the segfault - until
> now you even refused to try recent mod_security

I don't know where you got this idea. Of course I tested the F23's mod_security against F21's pcre without any segfault.

> and instead all the time
> close and re-assign the bugreport you could also wait for responses
> 
> * NOW we know: modsecurity 2.9.0 alone don#t segfault
> * so the next step would have been asking for my ruleset
>   which is not the default one for a million of reasons
> 
And that's the reason why mod_security people should work on it now.

> surely, you can insist in someone showing a simple regex which leads to the
> segfault, but i am long enough developer (not C/C++ but developer) to know
> that this leads nowhere
>
Then you should know that dumping unrelated code (mod_security) to blamed library (pcre) does not lead anywhere.

> well, at least as maintainer of a core-package you should throw out a big
> warning to not enable pcre-jit in any fedora package for security reasons

I'm sorry but this is the first crash I've ever got reported and I even has not seen it yet. I cannot see a reason why throw anything big anywhere.

Comment 18 Harald Reindl 2015-05-04 13:15:03 UTC
> Then you should know that dumping unrelated code (mod_security) 
> to blamed library (pcre) does not lead anywhere

the opposite is true

it leads to find errors nobody is able to find with generic test-suites and expected input data, anyways - i will not use pcre-jit in the future until i see some changelog indicating that somobedy nailed the issue down

Comment 19 Fedora End Of Life 2015-11-04 12:11:00 UTC
This message is a reminder that Fedora 21 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 21. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '21'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 21 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 20 Jan Kurik 2015-12-02 19:40:57 UTC
Fedora 21 changed to end-of-life (EOL) status on 2015-12-01. Fedora 21 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 21 Harald Reindl 2016-09-03 15:50:25 UTC
just a follow-up related to "i will not use pcre-jit in the future until i see some changelog indicating that somobedy nailed the issue down"

[root@testserver:~]$ rpm -q --changelog mod_security
* Sa Sep 03 2016 Reindl Harald <h.reindl@thelounge.net>
- re-enable 'pcre-jit'

/plugins/jqueryui/js/i18n/jquery.ui.datepicker-de.js no longer crashs httpd

Comment 22 Harald Reindl 2016-09-03 15:51:24 UTC
the first change sounds related

* Di Aug 30 2016 Petr Pisar <ppisar@redhat.com> - 8.39-3
- Fix register overwite in JIT when SSE2 acceleration is enabled
- Fix matching characters above 255 when a negative character type was used
  without enabled UCP in a positive class (upstream bug #1866)


Note You need to log in before you can comment on or make changes to this bug.