Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1131682

Summary:	saml auth causing apache crashes
Product:	Red Hat Enterprise Linux 6	Reporter:	Brian J. Atkisson <batkisso>
Component:	mod_auth_mellon	Assignee:	Simo Sorce <ssorce>
Status:	CLOSED NOTABUG	QA Contact:	Namita Soman <nsoman>
Severity:	high	Docs Contact:
Priority:	urgent
Version:	6.6	CC:	dpal, dparsons, drieden, tlavigne
Target Milestone:	rc
Target Release:	---
Hardware:	All
OS:	All
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2014-08-21 14:50:38 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Brian J. Atkisson 2014-08-19 19:58:52 UTC

Description of problem:

Using mod_auth_mellon (patched for bz1130604), I'm seeing Apache occasional crash post IdP redirect to mod_auth_mellon, resulting in a browser connection reset message.  This appears in the logs:

==> /var/log/httpd/mellon01.intranet.dev.int.devlab.redhat.com-access.log <==
10.13.111.1 - - [19/Aug/2014:15:37:33 -0400] "GET /favicon.ico HTTP/1.1" 404 303
10.13.111.1 - - [19/Aug/2014:15:37:38 -0400] "POST /secret/endpoint/postResponse HTTP/1.1" 303 360
10.13.111.1 - testuser [19/Aug/2014:15:37:38 -0400] "GET /secret/ HTTP/1.1" 200 4
10.13.111.1 - testuser [19/Aug/2014:15:44:27 -0400] "GET /secret/ HTTP/1.1" 304 -
10.13.111.1 - - [19/Aug/2014:15:44:37 -0400] "GET /secret/ HTTP/1.1" 303 507
10.13.111.1 - - [19/Aug/2014:15:44:37 -0400] "GET /secret/endpoint/login?ReturnTo=https%3A%2F%2Fmellon01.intranet.dev.int.devlab.redhat.com%2Fsecret%2F&IdP=https%3A%2F%2Fsaml.deviam.redhat.com%2Fidp%2F HTTP/1.1" 200 4817



 ==> /var/log/httpd/error_log.2014-08-19 <==
*** glibc detected *** /usr/sbin/httpd: corrupted double-linked list: 0x00007faad1bbb860 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x76156)[0x7faacf562156]
/lib64/libc.so.6(+0x78f91)[0x7faacf564f91]
/usr/lib64/libxml2.so.2(xmlFreeNodeList+0x156)[0x7faac7182266]
/usr/lib64/libxml2.so.2(xmlFreeNode+0x76)[0x7faac7182bd6]
/lib64/libglib-2.0.so.0(g_datalist_clear+0xa5)[0x7faac8d43285]
/lib64/libgobject-2.0.so.0(g_object_unref+0x174)[0x7faac8185794]
/usr/lib64/liblasso.so.3(+0x22a5f)[0x7faac92a8a5f]
/lib64/libgobject-2.0.so.0(g_object_unref+0xea)[0x7faac818570a]
/etc/httpd/modules/mod_auth_mellon.so(+0xaa3f)[0x7faac9b58a3f]
/etc/httpd/modules/mod_auth_mellon.so(am_handler+0x1083)[0x7faac9b5b343]
/usr/sbin/httpd(ap_run_handler+0x70)[0x7faad0fdccd0]
/usr/sbin/httpd(ap_invoke_handler+0xce)[0x7faad0fe058e]
/usr/sbin/httpd(ap_process_request+0x1c0)[0x7faad0febc50]
/usr/sbin/httpd(+0x37ac8)[0x7faad0fe8ac8]
/usr/sbin/httpd(ap_run_process_connection+0x68)[0x7faad0fe47d8]
/usr/sbin/httpd(+0x3fad7)[0x7faad0ff0ad7]
/usr/sbin/httpd(+0x3fdea)[0x7faad0ff0dea]
/usr/sbin/httpd(ap_mpm_run+0x2cb)[0x7faad0ff111b]
/usr/sbin/httpd(main+0xb90)[0x7faad0fc89b0]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x7faacf50ad1d]
/usr/sbin/httpd(+0x16919)[0x7faad0fc7919]
======= Memory map: ========
7faac390f000-7faac3925000 r-xp 00000000 fc:03 390218                     /lib64/libgcc_s-4.4.7-20120601.so.1
7faac3925000-7faac3b24000 ---p 00016000 fc:03 390218                     /lib64/libgcc_s-4.4.7-20120601.so.1
7faac3b24000-7faac3b25000 rw-p 00015000 fc:03 390218                     /lib64/libgcc_s-4.4.7-20120601.so.1
7faac3b25000-7faac4de6000 rw-s 00000000 00:04 69347                      /dev/zero (deleted)
7faac4de6000-7faac4e10000 r-xp 00000000 fc:03 536819                     /usr/lib64/httpd/modules/mod_ssl.so
7faac4e10000-7faac5010000 ---p 0002a000 fc:03 536819                     /usr/lib64/httpd/modules/mod_ssl.so
7faac5010000-7faac5011000 r--p 0002a000 fc:03 536819                     /usr/lib64/httpd/modules/mod_ssl.so
7faac5011000-7faac5012000 rw-p 0002b000 fc:03 536819                     /usr/lib64/httpd/modules/mod_ssl.so
7faac5012000-7faac5015000 rw-p 00000000 00:00 0 
7faac5015000-7faac502e000 r-xp 00000000 fc:03 523195                     /usr/lib64/libsasl2.so.2.0.23
7faac502e000-7faac522d000 ---p 00019000 fc:03 523195                     /usr/lib64/libsasl2.so.2.0.23
7faac522d000-7faac522e000 r--p 00018000 fc:03 523195                     /usr/lib64/libsasl2.so.2.0.23
7faac522e000-7faac522f000 rw-p 00019000 fc:03 523195                     /usr/lib64/libsasl2.so.2.0.23
7faac522f000-7faac523d000 r-xp 00000000 fc:03 390279                     /lib64/liblber-2.4.so.2.10.2
7faac523d000-7faac543c000 ---p 0000e000 fc:03 390279                     /lib64/liblber-2.4.so.2.10.2
7faac543c000-7faac543d000 r--p 0000d000 fc:03 390279                     /lib64/liblber-2.4.so.2.10.2
7faac543d000-7faac543e000 rw-p 0000e000 fc:03 390279                     /lib64/liblber-2.4.so.2.10.2
7faac543e000-7faac5454000 r-xp 00000000 fc:03 394214                     /lib64/libresolv-2.12.so
7faac5454000-7faac5654000 ---p 00016000 fc:03 394214                     /lib64/libresolv-2.12.so
7faac5654000-7faac5655000 r--p 00016000 fc:03 394214                     /lib64/libresolv-2.12.so
7faac5655000-7faac5656000 rw-p 00017000 fc:03 394214                     /lib64/libresolv-2.12.so
7faac5656000-7faac5658000 rw-p 00000000 00:00 0 
7faac5658000-7faac565a000 r-xp 00000000 fc:03 396707                     /lib64/libkeyutils.so.1.3
7faac565a000-7faac5859000 ---p 00002000 fc:03 396707                     /lib64/libkeyutils.so.1.3
7faac5859000-7faac585a000 r--p 00001000 fc:03 396707                     /lib64/libkeyutils.so.1.3
7faac585a000-7faac585b000 rw-p 00002000 fc:03 396707                     /lib64/libkeyutils.so.1.3
7faac585b000-7faac5865000 r-xp 00000000 fc:03 394224                     /lib64/libkrb5support.so.0.1
7faac5865000-7faac5a64000 ---p 0000a000 fc:03 394224                     /lib64/libkrb5support.so.0.1
7faac5a64000-7faac5a65000 r--p 00009000 fc:03 394224                     /lib64/libkrb5support.so.0.1
7faac5a65000-7faac5a66000 rw-p 0000a000 fc:03 394224                     /lib64/libkrb5support.so.0.1
7faac5a66000-7faac5a8d000 r-xp 00000000 fc:03 527792                     /usr/lib64/libssh2.so.1.0.1
7faac5a8d000-7faac5c8c000 ---p 00027000 fc:03 527792                     /usr/lib64/libssh2.so.1.0.1
7faac5c8c000-7faac5c8d000 rw-p 00026000 fc:03 527792                     /usr/lib64/libssh2.so.1.0.1
7faac5c8d000-7faac5c8e000 rw-p 00000000 00:00 0 
7faac5c8e000-7faac5cc7000 r-xp 00000000 fc:03 391251                     /lib64/libnspr4.so
7faac5cc7000-7faac5ec7000 ---p 00039000 fc:03 391251                     /lib64/libnspr4.so
7faac5ec7000-7faac5ec8000 r--p 00039000 fc:03 391251                     /lib64/libnspr4.so
7faac5ec8000-7faac5eca000 rw-p 0003a000 fc:03 391251                     /lib64/libnspr4.so
7faac5eca000-7faac5ecc000 rw-p 00000000 00:00 0 
7faac5ecc000-7faac5ed0000 r-xp 00000000 fc:03 396701                     /lib64/libplc4.so
7faac5ed0000-7faac60cf000 ---p 00004000 fc:03 396701                     /lib64/libplc4.so
7faac60cf000-7faac60d0000 r--p 00003000 fc:03 396701                     /lib64/libplc4.so
7faac60d0000-7faac60d1000 rw-p 00004000 fc:03 396701                     /lib64/libplc4.so
7faac60d1000-7faac60d4000 r-xp 00000000 fc:03 396702                     /lib64/libplds4.so
7faac60d4000-7faac62d3000 ---p 00003000 fc:03 396702                     /lib64/libplds4.so
7faac62d3000-7faac62d4000 r--p 00002000 fc:03 396702                     /lib64/libplds4.so
7faac62d4000-7faac62d5000 rw-p 00003000 fc:03 396702                     /lib64/libplds4.so
7faac62d5000-7faac62fa000 r-xp 00000000 fc:03 527911                     /usr/lib64/libnssutil3.so
7faac62fa000-7faac64fa000 ---p 00025000 fc:03 527911                     /usr/lib64/libnssutil3.so
7faac64fa000-7faac6500000 r--p 00025000 fc:03 527911                     /usr/lib64/libnssutil3.so
7faac6500000-7faac6501000 rw-p 0002b000 fc:03 527911                     /usr/lib64/libnssutil3.so
7faac6501000-7faac6637000 r-xp 00000000 fc:03 527594                     /usr/lib64/libnss3.so
7faac6637000-7faac6837000 ---p 00136000 fc:03 527594                     /usr/lib64/libnss3.so
7faac6837000-7faac683c000 r--p 00136000 fc:03 527594                     /usr/lib64/libnss3.so
7faac683c000-7faac683e000 rw-p 0013b000 fc:03 527594                     /usr/lib64/libnss3.so
7faac683e000-7faac6840000 rw-p 00000000 00:00 0 
7faac6840000-7faac6868000 r-xp 00000000 fc:03 536859                     /usr/lib64/libsmime3.so
7faac6868000-7faac6a67000 ---p 00028000 fc:03 536859                     /usr/lib64/libsmime3.so
7faac6a67000-7faac6a6b000 r--p 00027000 fc:03 536859                     /usr/lib64/libsmime3.so
7faac6a6b000-7faac6a6c000 rw-p 0002b000 fc:03 536859                     /usr/lib64/libsmime3.so
7faac6a6c000-7faac6aa6000 r-xp 00000000 fc:03 536860                     /usr/lib64/libssl3.so
7faac6aa6000-7faac6ca6000 ---p 0003a000 fc:03 536860                     /usr/lib64/libssl3.so
7faac6ca6000-7faac6ca9000 r--p 0003a000 fc:03 536860                     /usr/lib64/libssl3.so
7faac6ca9000-7faac6caa000 rw-p 0003d000 fc:03 536860                     /usr/lib64/libssl3.so
7faac6caa000-7faac6cab000 rw-p 00000000 00:00 0 
7faac6cab000-7faac6cf8000 r-xp 00000000 fc:03 394226                     /lib64/libldap-2.4.so.2.10.2
7faac6cf8000-7faac6ef7000 ---p 0004d000 fc:03 394226                     /lib64/libldap-2.4.so.2.10.2
7faac6ef7000-7faac6ef9000 r--p 0004c000 fc:03 394226                     /lib64/libldap-2.4.so.2.10.2
7faac6ef9000-7faac6efb000 rw-p 0004e000 fc:03 394226                     /lib64/libldap-2.4.so.2.10.2
7faac6efb000-7faac6f2d000 r-xp 00000000 fc:03 396715                     /lib64/libidn.so.11.6.1
7faac6f2d000-7faac712c000 ---p 00032000 fc:03 396715                     /lib64/libidn.so.11.6.1
7faac712c000-7faac712d000 rw-p 00031000 fc:03 396715                     /lib64/libidn.so.11.6.1
7faac712d000-7faac7275000 r-xp 00000000 fc:03 527127                     /usr/lib64/libxml2.so.2.7.6
7faac7275000-7faac7475000 ---p 00148000 fc:03 527127                     /usr/lib64/libxml2.so.2.7.6
7faac7475000-7faac747e000 rw-p 00148000 fc:03 527127                     /usr/lib64/libxml2.so.2.7.6
7faac747e000-7faac7480000 rw-p 00000000 00:00 0 
7faac7480000-7faac74bb000 r-xp 00000000 fc:03 524306                     /usr/lib64/libxslt.so.1.1.26
7faac74bb000-7faac76bb000 ---p 0003b000 fc:03 524306                     /usr/lib64/libxslt.so.1.1.26
7faac76bb000-7faac76bd000 rw-p 0003b000 fc:03 524306                     /usr/lib64/libxslt.so.1.1.26
7faac76bd000-7faac76c6000 r-xp 00000000 fc:03 536688                     /usr/lib64/libltdl.so.7.2.1
7faac76c6000-7faac78c5000 ---p 00009000 fc:03 536688                     /usr/lib64/libltdl.so.7.2.1
7faac78c5000-7faac78c6000 rw-p 00008000 fc:03 536688                     /usr/lib64/libltdl.so.7.2.1
7faac78c6000-7faac792a000 r-xp 00000000 fc:03 536695                     /usr/lib64/libxmlsec1.so.1.2.20
7faac792a000-7faac7b29000 ---p 00064000 fc:03 536695                     /usr/lib64/libxmlsec1.so.1.2.20
7faac7b29000-7faac7b2d000 rw-p 00063000 fc:03 536695                     /usr/lib64/libxmlsec1.so.1.2.20
7faac7b2d000-7faac7b65000 r-xp 00000000 fc:03 536705                     /usr/lib64/libxmlsec1-openssl.so.1.2.20
7faac7b65000-7faac7d64000 ---p 00038000 fc:03 536705                     /usr/lib64/libxmlsec1-openssl.so.1.2.20
7faac7d64000-7faac7d68000 rw-p 00037000 fc:03 536705                     /usr/lib64/libxmlsec1-openssl.so.1.2.20
7faac7d68000-7faac7d69000 rw-p 00000000 00:00 0 
7faac7d69000-7faac7d70000 r-xp 00000000 fc:03 394215                     /lib64/librt-2.12.so
7faac7d70000-7faac7f6f000 ---p 00007000 fc:03 394215                     /lib64/librt-2.12.so
7faac7f6f000-7faac7f70000 r--p 00006000 fc:03 394215                     /lib64/librt-2.12.so
7faac7f70000-7faac7f71000 rw-p 00007000 fc:03 394215                     /lib64/librt-2.12.so
7faac7f71000-7faac7f75000 r-xp 00000000 fc:03 394229                     /lib64/libgthread-2.0.so.0.2800.8
7faac7f75000-7faac8174000 ---p 00004000 fc:03 394229                     /lib64/libgthread-2.0.so.0.2800.8
7faac8174000-7faac8175000 rw-p 00003000 fc:03 394229                     /lib64/libgthread-2.0.so.0.2800.8
7faac8175000-7faac81bf000 r-xp 00000000 fc:03 390303                     /lib64/libgobject-2.0.so.0.2800.8
7faac81bf000-7faac83be000 ---p 0004a000 fc:03 390303                     /lib64/libgobject-2.0.so.0.2800.8
7faac83be000-7faac83c0000 rw-p 00049000 fc:03 390303                     /lib64/libgobject-2.0.so.0.2800.8
7faac83c0000-7faac83c1000 rw-p 00000000 00:00 0 
7faac83c1000-7faac83ea000 r-xp 00000000 fc:03 390285                     /lib64/libk5crypto.so.3.1
7faac83ea000-7faac85ea000 ---p 00029000 fc:03 390285                     /lib64/libk5crypto.so.3.1
7faac85ea000-7faac85eb000 r--p 00029000 fc:03 390285                     /lib64/libk5crypto.so.3.1
7faac85eb000-7faac85ec000 rw-p 0002a000 fc:03 390285                     /lib64/libk5crypto.so.3.1
7faac85ec000-7faac85ed000 rw-p 00000000 00:00 0 
7faac85ed000-7faac85f0000 r-xp 00000000 fc:03 390212                     /lib64/libcom_err.so.2.1
7faac85f0000-7faac87ef000 ---p 00003000 fc:03 390212                     /lib64/libcom_err.so.2.1
7faac87ef000-7faac87f0000 r--p 00002000 fc:03 390212                     /lib64/libcom_err.so.2.1
7faac87f0000-7faac87f1000 rw-p 00003000 fc:03 390212                     /lib64/libcom_err.so.2.1
7faac87f1000-7faac88cc000 r-xp 00000000 fc:03 394223                     /lib64/libkrb5.so.3.3
7faac88cc000-7faac8acb000 ---p 000db000 fc:03 394223                     /lib64/libkrb5.so.3.3
7faac8acb000-7faac8ad5000 r--p 000da000 fc:03 394223                     /lib64/libkrb5.so.3.3
7faac8ad5000-7faac8ad7000 rw-p 000e4000 fc:03 394223                     /lib64/libkrb5.so.3.3
7faac8ad7000-7faac8b18000 r-xp 00000000 fc:03 394221                     /lib64/libgssapi_krb5.so.2.2
7faac8b18000-7faac8d18000 ---p 00041000 fc:03 394221                     /lib64/libgssapi_krb5.so.2.2
7faac8d18000-7faac8d19000 r--p 00041000 fc:03 394221                     /lib64/libgssapi_krb5.so.2.2
7faac8d19000-7faac8d1b000 rw-p 00042000 fc:03 394221                     /lib64/libgssapi_krb5.so.2.2
7faac8d1b000-7faac8e30000 r-xp 00000000 fc:03 390299                     /lib64/libglib-2.0.so.0.2800.8
7faac8e30000-7faac9030000 ---p 00115000 fc:03 390299                     /lib64/libglib-2.0.so.0.2800.8
7faac9030000-7faac9031000 rw-p 00115000 fc:03 390299                     /lib64/libglib-2.0.so.0.2800.8
7faac9031000-7faac9032000 rw-p 00000000 00:00 0 
7faac9032000-7faac9084000 r-xp 00000000 fc:03 536645                     /usr/lib64/libcurl.so.4.1.1
7faac9084000-7faac9283000 ---p 00052000 fc:03 536645                     /usr/lib64/libcurl.so.4.1.1
7faac9283000-7faac9286000 rw-p 00051000 fc:03 536645                     /usr/lib64/libcurl.so.4.1.1
7faac9286000-7faac92f4000 r-xp 00000000 fc:03 536707                     /usr/lib64/liblasso.so.3.10.0
7faac92f4000-7faac94f4000 ---p 0006e000 fc:03 536707                     /usr/lib64/liblasso.so.3.10.0
7faac94f4000-7faac94fe000 rw-p 0006e000 fc:03 536707                     /usr/lib64/liblasso.so.3.10.0
7faac94fe000-7faac94ff000 rw-p 00000000 00:00 0 
7faac94ff000-7faac96b8000 r-xp 00000000 fc:03 525099                     /usr/lib64/libcrypto.so.1.0.1e
7faac96b8000-7faac98b7000 ---p 001b9000 fc:03 525099                     /usr/lib64/libcrypto.so.1.0.1e
7faac98b7000-7faac98d2000 r--p 001b8000 fc:03 525099                     /usr/lib64/libcrypto.so.1.0.1e
7faac98d2000-7faac98de000 rw-p 001d3000 fc:03 525099                     /usr/lib64/libcrypto.so.1.0.1e
7faac98de000-7faac98e2000 rw-p 00000000 00:00 0 
7faac98e2000-7faac9944000 r-xp 00000000 fc:03 532735                     /usr/lib64/libssl.so.1.0.1e
7faac9944000-7faac9b43000 ---p 00062000 fc:03 532735                     /usr/lib64/libssl.so.1.0.1e
7faac9b43000-7faac9b47000 r--p 00061000 fc:03 532735                     /usr/lib64/libssl.so.1.0.1e
7faac9b47000-7faac9b4e000 rw-p 00065000 fc:03 532735                     /usr/lib64/libssl.so.1.0.1e
7faac9b4e000-7faac9b65000 r-xp 00000000 fc:03 536814                     /usr/lib64/httpd/modules/mod_auth_mellon.so
7faac9b65000-7faac9d64000 ---p 00017000 fc:03 536814                     /usr/lib64/httpd/modules/mod_auth_mellon.so
7faac9d64000-7faac9d66000 rw-p 00016000 fc:03 536814                     /usr/lib64/httpd/modules/mod_auth_mellon.so
7faac9d66000-7faac9d68000 r-xp 00000000 fc:03 536777                     /usr/lib64/httpd/modules/mod_proxy_connect.so
7faac9d68000-7faac9f68000 ---p 00002000 fc:03 536777                     /usr/lib64/httpd/modules/mod_proxy_connect.so
7faac9f68000-7faac9f69000 r--p 00002000 fc:03 536777                     /usr/lib64/httpd/modules/mod_proxy_connect.so
7faac9f69000-7faac9f6a000 rw-p 00003000 fc:03 536777                     /usr/lib64/httpd/modules/mod_proxy_connect.so
7faac9f6a000-7faac9f71000 r-xp 00000000 fc:03 536779                     /usr/lib64/httpd/modules/mod_proxy_http.so
7faac9f71000-7faaca170000 ---p 00007000 fc:03 536779                     /usr/lib64/httpd/modules/mod_proxy_http.so
7faaca170000-7faaca171000 r--p 00006000 fc:03 536779                     /usr/lib64/httpd/modules/mod_proxy_http.so
7faaca171000-7faaca172000 rw-p 00007000 fc:03 536779                     /usr/lib64/httpd/modules/mod_proxy_http.so
7faaca172000-7faaca178000 r-xp 00000000 fc:03 536776                     /usr/lib64/httpd/modules/mod_proxy_balancer.so
7faaca178000-7faaca377000 ---p 00006000 fc:03 536776                     /usr/lib64/httpd/modules/mod_proxy_balancer.so
7faaca377000-7faaca378000 r--p 00005000 fc:03 536776                     /usr/lib64/httpd/modules/mod_proxy_balancer.so
7faaca378000-7faaca379000 rw-p 00006000 fc:03 536776                     /usr/lib64/httpd/modules/mod_proxy_balancer.so
7faaca379000-7faaca38d000 r-xp 00000000 fc:03 536774                     /usr/lib64/httpd/modules/mod_proxy.so
7faaca38d000-7faaca58c000 ---p 00014000 fc:03 536774                     /usr/lib64/httpd/modules/mod_proxy.so
7faaca58c000-7faaca58d000 r--p 00013000 fc:03 536774                     /usr/lib64/httpd/modules/mod_proxy.so
7faaca58d000-7faaca58e000 rw-p 00014000 fc:03 536774                     /usr/lib64/httpd/modules/mod_proxy.so
7faaca58e000-7faaca59c000 r-xp 00000000 fc:03 536782                     /usr/lib64/httpd/modules/mod_rewrite.so
7faaca59c000-7faaca79b000 ---p 0000e000 fc:03 536782                     /usr/lib64/httpd/modules/mod_rewrite.so
7faaca79b000-7faaca79c000 r--p 0000d000 fc:03 536782                     /usr/lib64/httpd/modules/mod_rewrite.so
7faaca79c000-7faaca79d000 rw-p 0000e000 fc:03 536782                     /usr/lib64/httpd/modules/mod_rewrite.so
7faaca79d000-7faaca7a0000 r-xp 00000000 fc:03 536730                     /usr/lib64/httpd/modules/mod_alias.so
7faaca7a0000-7faaca99f000 ---p 00003000 fc:03 536730                     /usr/lib64/httpd/modules/mod_alias.so
7faaca99f000-7faaca9a0000 r--p 00002000 fc:03 536730                     /usr/lib64/httpd/modules/mod_alias.so
7faaca9a0000-7faaca9a1000 rw-p 00003000 fc:03 536730                     /usr/lib64/httpd/modules/mod_alias.so
7faaca9a1000-7faaca9a3000 r-xp 00000000 fc:03 536756                     /usr/lib64/httpd/modules/mod_dir.so
7faaca9a3000-7faacaba2000 ---p 00002000 fc:03 536756                     /usr/lib64/httpd/modules/mod_dir.so
7faacaba2000-7faacaba3000 r--p 00001000 fc:03 536756                     /usr/lib64/httpd/modules/mod_dir.so
7faacaba3000-7faacaba4000 rw-p 00002000 fc:03 536756                     /usr/lib64/httpd/modules/mod_dir.so
7faacaba4000-7faacabab000 r-xp 00000000 fc:03 536773                     /usr/lib64/httpd/modules/mod_negotiation.so
7faacabab000-7faacadaa000 ---p 00007000 fc:03 536773                     /usr/lib64/httpd/modules/mod_negotiation.so
7faacadaa000-7faacadab000 r--p 00006000 fc:03 536773                     /usr/lib64/httpd/modules/mod_negotiation.so
7faacadab000-7faacadac000 rw-p 00007000 fc:03 536773                     /usr/lib64/httpd/modules/mod_negotiation.so
7faacadac000-7faacadae000 r-xp 00000000 fc:03 536792                     /usr/lib64/httpd/modules/mod_vhost_alias.so
7faacadae000-7faacafad000 ---p 00002000 fc:03 536792                     /usr/lib64/httpd/modules/mod_vhost_alias.so
7faacafad000-7faacafae000 r--p 00001000 fc:03 536792                     /usr/lib64/httpd/modules/mod_vhost_alias.so
7faacafae000-7faacafaf000 rw-p 00002000 fc:03 536792                     /usr/lib64/httpd/modules/mod_vhost_alias.so
7faacafaf000-7faacafb4000 r-xp 00000000 fc:03 536750                     /usr/lib64/httpd/modules/mod_cgi.so
7faacafb4000-7faacb1b4000 ---p 00005000 fc:03 536750                     /usr/lib64/httpd/modules/mod_cgi.so
7faacb1b4000-7faacb1b5000 r--p 00005000 fc:03 536750                     /usr/lib64/httpd/modules/mod_cgi.so
7faacb1b5000-7faacb1b6000 rw-p 00006000 fc:03 536750                     /usr/lib64/httpd/modules/mod_cgi.so
7faacb1b6000-7faacb1ba000 r-xp 00000000 fc:03 536766                     /usr/lib64/httpd/modules/mod_info.so
7faacb1ba000-7faacb3b9000 ---p 00004000 fc:03 536766                     /usr/lib64/httpd/modules/mod_info.so
7faacb3b9000-7faacb3ba000 r--p 00003000 fc:03 536766                     /usr/lib64/httpd/modules/mod_info.so
7faacb3ba000-7faacb3bb000 rw-p 00004000 fc:03 536766                     /usr/lib64/httpd/modules/mod_info.so
7faacb3bb000-7faacb3c3000 r-xp 00000000 fc:03 536747                     /usr/lib64/httpd/modules/mod_autoindex.so
7faacb3c3000-7faacb5c2000 ---p 00008000 fc:03 536747                     /usr/lib64/httpd/modules/mod_autoindex.so
7faacb5c2000-7faacb5c3000 r--p 00007000 fc:03 536747                     /usr/lib64/httpd/modules/mod_autoindex.so
7faacb5c3000-7faacb5c4000 rw-p 00008000 fc:03 536747                     /usr/lib64/httpd/modules/mod_autoindex.so
7faacb5c4000-7faacb5c8000 r-xp 00000000 fc:03 536785                     /usr/lib64/httpd/modules/mod_status.so
7faacb5c8000-7faacb7c8000 ---p 00004000 fc:03 536785                     /usr/lib64/httpd/modules/mod_status.so
7faacb7c8000-7faacb7c9000 r--p 00004000 fc:03 536785                     /usr/lib64/httpd/modules/mod_status.so
7faacb7c9000-7faacb7ca000 rw-p 00005000 fc:03 536785                     /usr/lib64/httpd/modules/mod_status.so
7faacb7ca000-7faacb7ce000 r-xp 00000000 fc:03 536771                     /usr/lib64/httpd/modules/mod_mime.so
7faacb7ce000-7faacb9cd000 ---p 00004000 fc:03 536771                     /usr/lib64/httpd/modules/mod_mime.so
7faacb9cd000-7faacb9ce000 r--p 00003000 fc:03 536771                     /usr/lib64/httpd/modules/mod_mime.so
7faacb9ce000-7faacb9cf000 rw-p 00004000 fc:03 536771                     /usr/lib64/httpd/modules/mod_mime.so
7faacb9cf000-7faacb9d1000 r-xp 00000000 fc:03 536783                     /usr/lib64/httpd/modules/mod_setenvif.so
7faacb9d1000-7faacbbd1000 ---p 00002000 fc:03 536783                     /usr/lib64/httpd/modules/mod_setenvif.so
7faacbbd1000-7faacbbd2000 r--p 00002000 fc:03 536783                     /usr/lib64/httpd/modules/mod_setenvif.so
7faacbbd2000-7faacbbd3000 rw-p 00003000 fc:03 536783                     /usr/lib64/httpd/modules/mod_setenvif.so
7faacbbd3000-7faacbbd7000 r-xp 00000000 fc:03 536763                     /usr/lib64/httpd/modules/mod_headers.so
7faacbbd7000-7faacbdd6000 ---p 00004000 fc:03 536763                     /usr/lib64/httpd/modules/mod_headers.so
7faacbdd6000-7faacbdd7000 r--p 00003000 fc:03 536763                     /usr/lib64/httpd/modules/mod_headers.so
7faacbdd7000-7faacbdd8000 rw-p 00004000 fc:03 536763                     /usr/lib64/httpd/modules/mod_headers.so
7faacbdd8000-7faacbded000 r-xp 00000000 fc:03 396692                     /lib64/libz.so.1.2.3
7faacbded000-7faacbfec000 ---p 00015000 fc:03 396692                     /lib64/libz.so.1.2.3
7faacbfec000-7faacbfed000 r--p 00014000 fc:03 396692                     /lib64/libz.so.1.2.3
7faacbfed000-7faacbfee000 rw-p 00015000 fc:03 396692                     /lib64/libz.so.1.2.3
7faacbfee000-7faacbff3000 r-xp 00000000 fc:03 536755                     /usr/lib64/httpd/modules/mod_deflate.so
7faacbff3000-7faacc1f3000 ---p 00005000 fc:03 536755                     /usr/lib64/httpd/modules/mod_deflate.so
7faacc1f3000-7faacc1f4000 r--p 00005000 fc:03 536755                     /usr/lib64/httpd/modules/mod_deflate.so
7faacc1f4000-7faacc1f5000 rw-p 00006000 fc:03 536755                     /usr/lib64/httpd/modules/mod_deflate.so
7faacc1f5000-7faacc1f7000 r-xp 00000000 fc:03 536760                     /usr/lib64/httpd/modules/mod_expires.so
7faacc1f7000-7faacc3f6000 ---p 00002000 fc:03 536760                     /usr/lib64/httpd/modules/mod_expires.so
7faacc3f6000-7faacc3f7000 r--p 00001000 fc:03 536760                     /usr/lib64/httpd/modules/mod_expires.so
7faacc3f7000-7faacc3f8000 rw-p 00002000 fc:03 536760                     /usr/lib64/httpd/modules/mod_expires.so
7faacc3f8000-7faacc3fd000 r-xp 00000000 fc:03 536772                     /usr/lib64/httpd/modules/mod_mime_magic.so
7faacc3fd000-7faacc5fd000 ---p 00005000 fc:03 536772                     /usr/lib64/httpd/modules/mod_mime_magic.so
7faacc5fd000-7faacc5fe000 r--p 00005000 fc:03 536772                     /usr/lib64/httpd/modules/mod_mime_magic.so
7faacc5fe000-7faacc5ff000 rw-p 00006000 fc:03 536772                     /usr/lib64/httpd/modules/mod_mime_magic.so
7faacc5ff000-7faacc604000 r-xp 00000000 fc:03 536761                     /usr/lib64/httpd/modules/mod_ext_filter.so
7faacc604000-7faacc803000 ---p 00005000 fc:03 536761                     /usr/lib64/httpd/modules/mod_ext_filter.so
7faacc803000-7faacc804000 r--p 00004000 fc:03 536761                     /usr/lib64/httpd/modules/mod_ext_filter.so
7faacc804000-7faacc805000 rw-p 00005000 fc:03 536761                     /usr/lib64/httpd/modules/mod_ext_filter.so
7faacc805000-7faacc807000 r-xp 00000000 fc:03 536759                     /usr/lib64/httpd/modules/mod_env.so
7faacc807000-7faacca06000 ---p 00002000 fc:03 536759                     /usr/lib64/httpd/modules/mod_env.so
7faacca06000-7faacca07000 r--p 00001000 fc:03 536759                     /usr/lib64/httpd/modules/mod_env.so
7faacca07000-7faacca08000 rw-p 00002000 fc:03 536759                     /usr/lib64/httpd/modules/mod_env.so
7faacca08000-7faacca0a000 r-xp 00000000 fc:03 536770                     /usr/lib64/httpd/modules/mod_logio.so
7faacca0a000-7faaccc09000 ---p 00002000 fc:03 536770                     /usr/lib64/httpd/modules/mod_logio.so
7faaccc09000-7faaccc0a000 r--p 00001000 fc:03 536770                     /usr/lib64/httpd/modules/mod_logio.so
7faaccc0a000-7faaccc0b000 rw-p 00002000 fc:03 536770                     /usr/lib64/httpd/modules/mod_logio.so
7faaccc0b000-7faaccc10000 r-xp 00000000 fc:03 536768                     /usr/lib64/httpd/modules/mod_log_config.so
7faaccc10000-7faacce10000 ---p 00005000 fc:03 536768                     /usr/lib64/httpd/modules/mod_log_config.so
7faacce10000-7faacce11000 r--p 00005000 fc:03 536768                     /usr/lib64/httpd/modules/mod_log_config.so
7faacce11000-7faacce12000 rw-p 00006000 fc:03 536768                     /usr/lib64/httpd/modules/mod_log_config.so
7faacce12000-7faacce1c000 r-xp 00000000 fc:03 536765                     /usr/lib64/httpd/modules/mod_include.so
7faacce1c000-7faacd01b000 ---p 0000a000 fc:03 536765                     /usr/lib64/httpd/modules/mod_include.so
7faacd01b000-7faacd01c000 r--p 00009000 fc:03 536765                     /usr/lib64/httpd/modules/mod_include.so
7faacd01c000-7faacd01d000 rw-p 0000a000 fc:03 536765                     /usr/lib64/httpd/modules/mod_include.so
7faacd01d000-7faacd028000 r-xp 00000000 fc:03 536753                     /usr/lib64/httpd/modules/mod_dav_fs.so
7faacd028000-7faacd228000 ---p 0000b000 fc:03 536753                     /usr/lib64/httpd/modules/mod_dav_fs.so
7faacd228000-7faacd229000 r--p 0000b000 fc:03 536753                     /usr/lib64/httpd/modules/mod_dav_fs.so
7faacd229000-7faacd22a000 rw-p 0000c000 fc:03 536753                     /usr/lib64/httpd/modules/mod_dav_fs.so
7faacd22a000-7faacd240000 r-xp 00000000 fc:03 536752                     /usr/lib64/httpd/modules/mod_dav.so
7faacd240000-7faacd43f000 ---p 00016000 fc:03 536752                     /usr/lib64/httpd/modules/mod_dav.so
7faacd43f000-7faacd440000 r--p 00015000 fc:03 536752                     /usr/lib64/httpd/modules/mod_dav.so
7faacd440000-7faacd441000 rw-p 00016000 fc:03 536752                     /usr/lib64/httpd/modules/mod_dav.so
7faacd441000-7faacd442000 r-xp 00000000 fc:03 536742                     /usr/lib64/httpd/modules/mod_authz_default.so
7faacd442000-7faacd641000 ---p 00001000 fc:03 536742                     /usr/lib64/httpd/modules/mod_authz_default.so
7faacd641000-7faacd642000 r--p 00000000 fc:03 536742                     /usr/lib64/httpd/modules/mod_authz_default.so
7faacd642000-7faacd643000 rw-p 00001000 fc:03 536742                     /usr/lib64/httpd/modules/mod_authz_default.so
7faacd643000-7faacd645000 r-xp 00000000 fc:03 536741                     /usr/lib64/httpd/modules/mod_authz_dbm.so
7faacd645000-7faacd844000 ---p 00002000 fc:03 536741                     /usr/lib64/httpd/modules/mod_authz_dbm.so
7faacd844000-7faacd845000 r--p 00001000 fc:03 536741                     /usr/lib64/httpd/modules/mod_authz_dbm.so
7faacd845000-7faacd846000 rw-p 00002000 fc:03 536741                     /usr/lib64/httpd/modules/mod_authz_dbm.so
7faacd846000-7faacd848000 r-xp 00000000 fc:03 536743                     /usr/lib64/httpd/modules/mod_authz_groupfile.so
7faacd848000-7faacda47000 ---p 00002000 fc:03 536743                     /usr/lib64/httpd/modules/mod_authz_groupfile.so
7faacda47000-7faacda48000 r--p 00001000 fc:03 536743                     /usr/lib64/httpd/modules/mod_authz_groupfile.so
7faacda48000-7faacda49000 rw-p 00002000 fc:03 536743                     /usr/lib64/httpd/modules/mod_authz_groupfile.so
7faacda49000-7faacda4b000 r-xp 00000000 fc:03 536745                     /usr/lib64/httpd/modules/mod_authz_owner.so
7faacda4b000-7faacdc4a000 ---p 00002000 fc:03 536745                     /usr/lib64/httpd/modules/mod_authz_owner.so
7faacdc4a000-7faacdc4b000 r--p 00001000 fc:03 536745                     /usr/lib64/httpd/modules/mod_authz_owner.so
7faacdc4b000-7faacdc4c000 rw-p 00002000 fc:03 536745                     /usr/lib64/httpd/modules/mod_authz_owner.so
7faacdc4c000-7faacdc4d000 r-xp 00000000 fc:03 536746                     /usr/lib64/httpd/modules/mod_authz_user.so
7faacdc4d000-7faacde4d000 ---p 00001000 fc:03 536746                     /usr/lib64/httpd/modules/mod_authz_user.so
7faacde4d000-7faacde4e000 r--p 00001000 fc:03 536746                     /usr/lib64/httpd/modules/mod_authz_user.so
7faacde4e000-7faacde4f000 rw-p 00002000 fc:03 536746                     /usr/lib64/httpd/modules/mod_authz_user.so
7faacde4f000-7faacde51000 r-xp 00000000 fc:03 536744                     /usr/lib64/httpd/modules/mod_authz_host.so
7faacde51000-7faace050000 ---p 00002000 fc:03 536744                     /usr/lib64/httpd/modules/mod_authz_host.so
7faace050000-7faace051000 r--p 00001000 fc:03 536744                     /usr/lib64/httpd/modules/mod_authz_host.so
7faace051000-7faace052000 rw-p 00002000 fc:03 536744                     /usr/lib64/httpd/modules/mod_authz_host.so
7faace052000-7faace053000 r-xp 00000000 fc:03 536738                     /usr/lib64/httpd/modules/mod_authn_default.so
7faace053000-7faace252000 ---p 00001000 fc:03 536738                     /usr/lib64/httpd/modules/mod_authn_default.so
7faace252000-7faace253000 r--p 00000000 fc:03 536738                     /usr/lib64/httpd/modules/mod_authn_default.so
7faace253000-7faace254000 rw-p 00001000 fc:03 536738                     /usr/lib64/httpd/modules/mod_authn_default.so
7faace254000-7faace256000 r-xp 00000000 fc:03 536737                     /usr/lib64/httpd/modules/mod_authn_dbm.so
7faace256000-7faace455000 ---p 00002000 fc:03 536737                     /usr/lib64/httpd/modules/mod_authn_dbm.so
7faace455000-7faace456000 r--p 00001000 fc:03 536737                     /usr/lib64/httpd/modules/mod_authn_dbm.so
7faace456000-7faace457000 rw-p 00002000 fc:03 536737                     /usr/lib64/httpd/modules/mod_authn_dbm.so
7faace457000-7faace458000 r-xp 00000000 fc:03 536735                     /usr/lib64/httpd/modules/mod_authn_anon.so
7faace458000-7faace658000 ---p 00001000 fc:03 536735                     /usr/lib64/httpd/modules/mod_authn_anon.so
7faace658000-7faace659000 r--p 00001000 fc:03 536735                     /usr/lib64/httpd/modules/mod_authn_anon.so
7faace659000-7faace65a000 rw-p 00002000 fc:03 536735                     /usr/lib64/httpd/modules/mod_authn_anon.so
7faace65a000-7faace65c000 r-xp 00000000 fc:03 536734                     /usr/lib64/httpd/modules/mod_authn_alias.so
7faace65c000-7faace85b000 ---p 00002000 fc:03 536734                     /usr/lib64/httpd/modules/mod_authn_alias.so
7faace85b000-7faace85c000 r--p 00001000 fc:03 536734                     /usr/lib64/httpd/modules/mod_authn_alias.so
7faace85c000-7faace85d000 rw-p 00002000 fc:03 536734                     /usr/lib64/httpd/modules/mod_authn_alias.so
7faace85d000-7faace85f000 r-xp 00000000 fc:03 536739                     /usr/lib64/httpd/modules/mod_authn_file.so
7faace85f000-7faacea5e000 ---p 00002000 fc:03 536739                     /usr/lib64/httpd/modules/mod_authn_file.so
7faacea5e000-7faacea5f000 r--p 00001000 fc:03 536739                     /usr/lib64/httpd/modules/mod_authn_file.so
7faacea5f000-7faacea60000 rw-p 00002000 fc:03 536739                     /usr/lib64/httpd/modules/mod_authn_file.so
7faacea60000-7faacea66000 r-xp 00000000 fc:03 536733                     /usr/lib64/httpd/modules/mod_auth_digest.so
7faacea66000-7faacec66000 ---p 00006000 fc:03 536733                     /usr/lib64/httpd/modules/mod_auth_digest.so
7faacec66000-7faacec67000 r--p 00006000 fc:03 536733                     /usr/lib64/httpd/modules/mod_auth_digest.so
7faacec67000-7faacec68000 rw-p 00007000 fc:03 536733                     /usr/lib64/httpd/modules/mod_auth_digest.so
7faacec68000-7faacec6a000 r-xp 00000000 fc:03 536732                     /usr/lib64/httpd/modules/mod_auth_basic.so
7faacec6a000-7faacee69000 ---p 00002000 fc:03 536732                     /usr/lib64/httpd/modules/mod_auth_basic.so
7faacee69000-7faacee6a000 r--p 00001000 fc:03 536732                     /usr/lib64/httpd/modules/mod_auth_basic.so
7faacee6a000-7faacee6b000 rw-p 00002000 fc:03 536732                     /usr/lib64/httpd/modules/mod_auth_basic.so
7faacee6b000-7faaceede000 r-xp 00000000 fc:03 391559                     /lib64/libfreebl3.so
7faaceede000-7faacf0dd000 ---p 00073000 fc:03 391559                     /lib64/libfreebl3.so
7faacf0dd000-7faacf0df000 r--p 00072000 fc:03 391559                     /lib64/libfreebl3.so
7faacf0df000-7faacf0e0000 rw-p 00074000 fc:03 391559                     /lib64/libfreebl3.so
7faacf0e0000-7faacf0e4000 rw-p 00000000 00:00 0 
7faacf0e4000-7faacf0e8000 r-xp 00000000 fc:03 390232                     /lib64/libuuid.so.1.3.0
7faacf0e8000-7faacf2e7000 ---p 00004000 fc:03 390232                     /lib64/libuuid.so.1.3.0
7faacf2e7000-7faacf2e8000 rw-p 00003000 fc:03 390232                     /lib64/libuuid.so.1.3.0
7faacf2e8000-7faacf2ea000 r-xp 00000000 fc:03 394209                     /lib64/libdl-2.12.so
7faacf2ea000-7faacf4ea000 ---p 00002000 fc:03 394209                     /lib64/libdl-2.12.so
7faacf4ea000-7faacf4eb000 r--p 00002000 fc:03 394209                     /lib64/libdl-2.12.so
7faacf4eb000-7faacf4ec000 rw-p 00003000 fc:03 394209                     /lib64/libdl-2.12.so
7faacf4ec000-7faacf677000 r-xp 00000000 fc:03 390174                     /lib64/libc-2.12.so
7faacf677000-7faacf876000 ---p 0018b000 fc:03 390174                     /lib64/libc-2.12.so
7faacf876000-7faacf87a000 r--p 0018a000 fc:03 390174                     /lib64/libc-2.12.so
7faacf87a000-7faacf87b000 rw-p 0018e000 fc:03 390174                     /lib64/libc-2.12.so
7faacf87b000-7faacf880000 rw-p 00000000 00:00 0 
7faacf880000-7faacf897000 r-xp 00000000 fc:03 390198                     /lib64/libpthread-2.12.so
7faacf897000-7faacfa97000 ---p 00017000 fc:03 390198                     /lib64/libpthread-2.12.so
7faacfa97000-7faacfa98000 r--p 00017000 fc:03 390198                     /lib64/libpthread-2.12.so
7faacfa98000-7faacfa99000 rw-p 00018000 fc:03 390198                     /lib64/libpthread-2.12.so
7faacfa99000-7faacfa9d000 rw-p 00000000 00:00 0 
7faacfa9d000-7faacfac8000 r-xp 00000000 fc:03 536675                     /usr/lib64/libapr-1.so.0.3.9
7faacfac8000-7faacfcc7000 ---p 0002b000 fc:03 536675                     /usr/lib64/libapr-1.so.0.3.9
7faacfcc7000-7faacfcc9000 rw-p 0002a000 fc:03 536675                     /usr/lib64/libapr-1.so.0.3.9
7faacfcc9000-7faacfe38000 r-xp 00000000 fc:03 390296                     /lib64/libdb-4.7.so
7faacfe38000-7faad0037000 ---p 0016f000 fc:03 390296                     /lib64/libdb-4.7.so
7faad0037000-7faad003d000 rw-p 0016e000 fc:03 390296                     /lib64/libdb-4.7.so
7faad003d000-7faad0063000 r-xp 00000000 fc:03 390298                     /lib64/libexpat.so.1.5.2
7faad0063000-7faad0262000 ---p 00026000 fc:03 390298                     /lib64/libexpat.so.1.5.2
7faad0262000-7faad0265000 rw-p 00025000 fc:03 390298                     /lib64/libexpat.so.1.5.2
7faad0265000-7faad026c000 r-xp 00000000 fc:03 390178                     /lib64/libcrypt-2.12.so
7faad026c000-7faad046c000 ---p 00007000 fc:03 390178                     /lib64/libcrypt-2.12.so
7faad046c000-7faad046d000 r--p 00007000 fc:03 390178                     /lib64/libcrypt-2.12.so
7faad046d000-7faad046e000 rw-p 00008000 fc:03 390178                     /lib64/libcrypt-2.12.so
7faad046e000-7faad049c000 rw-p 00000000 00:00 0 
7faad049c000-7faad04bf000 r-xp 00000000 fc:03 536682                     /usr/lib64/libaprutil-1.so.0.3.9
7faad04bf000-7faad06be000 ---p 00023000 fc:03 536682                     /usr/lib64/libaprutil-1.so.0.3.9
7faad06be000-7faad06c0000 rw-p 00022000 fc:03 536682                     /usr/lib64/libaprutil-1.so.0.3.9
7faad06c0000-7faad06dd000 r-xp 00000000 fc:03 390211                     /lib64/libselinux.so.1
7faad06dd000-7faad08dc000 ---p 0001d000 fc:03 390211                     /lib64/libselinux.so.1
7faad08dc000-7faad08dd000 r--p 0001c000 fc:03 390211                     /lib64/libselinux.so.1
7faad08dd000-7faad08de000 rw-p 0001d000 fc:03 390211                     /lib64/libselinux.so.1
7faad08de000-7faad08df000 rw-p 00000000 00:00 0 
7faad08df000-7faad090b000 r-xp 00000000 fc:03 394007                     /lib64/libpcre.so.0.0.1
7faad090b000-7faad0b0a000 ---p 0002c000 fc:03 394007                     /lib64/libpcre.so.0.0.1
7faad0b0a000-7faad0b0b000 rw-p 0002b000 fc:03 394007                     /lib64/libpcre.so.0.0.1
7faad0b0b000-7faad0b8e000 r-xp 00000000 fc:03 394210                     /lib64/libm-2.12.so
7faad0b8e000-7faad0d8d000 ---p 00083000 fc:03 394210                     /lib64/libm-2.12.so
7faad0d8d000-7faad0d8e000 r--p 00082000 fc:03 394210                     /lib64/libm-2.12.so
7faad0d8e000-7faad0d8f000 rw-p 00083000 fc:03 394210                     /lib64/libm-2.12.so
7faad0d8f000-7faad0daf000 r-xp 00000000 fc:03 396731                     /lib64/ld-2.12.so
7faad0f11000-7faad0f12000 rw-p 00000000 00:00 0 
7faad0f12000-7faad0f21000 rw-s 00000000 00:04 69351                      /dev/zero (deleted)
7faad0f21000-7faad0f9f000 rw-s 00000000 00:04 69349                      /dev/zero (deleted)
7faad0f9f000-7faad0fa6000 rw-p 00000000 00:00 0 
7faad0fa6000-7faad0fad000 r--s 00000000 fc:03 9662                       /usr/lib64/gconv/gconv-modules.cache
7faad0fad000-7faad0fae000 rw-p 00000000 00:00 0 
7faad0fae000-7faad0faf000 r--p 0001f000 fc:03 396731                     /lib64/ld-2.12.so
7faad0faf000-7faad0fb0000 rw-p 00020000 fc:03 396731                     /lib64/ld-2.12.so
7faad0fb0000-7faad0fb1000 rw-p 00000000 00:00 0 
7faad0fb1000-7faad1005000 r-xp 00000000 fc:03 536795                     /usr/sbin/httpd
7faad1204000-7faad1208000 rw-p 00053000 fc:03 536795                     /usr/sbin/httpd
7faad1208000-7faad120b000 rw-p 00000000 00:00 0 
7faad18be000-7faad1a6f000 rw-p 00000000 00:00 0 
7faad1a6f000-7faad1bda000 rw-p 00000000 00:00 0 
7fffd1a79000-7fffd1a8e000 rw-p 00000000 00:00 0                          [stack]
7fffd1b51000-7fffd1b52000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
[Tue Aug 19 15:44:44 2014] [notice] child pid 4229 exit signal Aborted (6)

Version-Release number of selected component (if applicable):
httpd-2.2.15-38.el6.x86_64
mod_ssl-2.2.15-38.el6.x86_64
mod_auth_mellon-0.8.0-bz1130604.el6.x86_64

How reproducible:
Happens occasionally post-IdP auth, don't have a constant reproducer yet.

Steps to Reproduce:
1. login to https://sp/secret
2. redirected to IdP for Auth
3. Auth posts back to SP, but SP blows up.

Actual results:
Log messages and browser reset

Expected results:
View the actual restricted page.

Additional info:

I'm trying to get a core, but no luck. I can provide SSH access to the hosts, if needed.

[mellon01.intranet.dev.int.devlab.redhat.com] [03:56:15 PM]
[root@mellon01 conf.d]# cat 10-auth_mellon.conf
LoadModule auth_mellon_module modules/mod_auth_mellon.so
[mellon01.intranet.dev.int.devlab.redhat.com] [03:56:30 PM]
[root@mellon01 conf.d]# cat access-mellon01.intranet.dev.int.devlab.redhat.com.conf
<Location /secret>
    Require valid-user
    AuthType "Mellon"
    MellonEnable "auth"
    MellonDecoder "none"
    MellonVariable "cookie"
    MellonSecureCookie On
    MellonUser "NAME_ID"
    MellonSetEnv "e-mail" "mail"
    MellonEndpointPath "/secret/endpoint"
    MellonDefaultLoginPath "/secret"
    MellonSessionLength 86400
    MellonOrganizationURL "http://www.redhat.com"
    MellonSPPrivateKeyFile /etc/pki/tls/certs/auth-mellon.pem
    MellonSPCertFile /etc/pki/tls/certs/auth-mellon.pem
    MellonIdPMetadataFile /etc/httpd/conf/ss-idp-metadata.xml
    MellonSamlResponseDump On
    MellonSessionDump On
</Location>

[mellon01.intranet.dev.int.devlab.redhat.com] [03:56:33 PM]
[root@mellon01 conf.d]# cat auth_mellon.conf
MellonCacheSize 100
MellonLockFile "/var/run/mod_auth_mellon/lock"

[mellon01.intranet.dev.int.devlab.redhat.com] [03:56:35 PM]
[root@mellon01 conf.d]# cat vhost-mellon01.intranet.dev.int.devlab.redhat.com.conf
<VirtualHost 10.7.27.246:80>
    ServerName mellon01.intranet.dev.int.devlab.redhat.com
    ServerAlias mellon01.intranet.dev.int.devlab.redhat.com
    ServerAdmin webmaster

    RewriteEngine on
    RewriteCond %{HTTPS} !=on
    RewriteRule ^/(.*) https://%{SERVER_NAME}/$1 [NE]
</VirtualHost>

<VirtualHost 10.7.27.246:443>
    ServerName mellon01.intranet.dev.int.devlab.redhat.com
    ServerAlias mellon01.intranet.dev.int.devlab.redhat.com
    ServerAdmin webmaster

    <IfModule mod_ssl.c>
        SSLEngine on
        SSLProxyEngine on
        SSLCertificateFile /etc/pki/tls/certs/cert-mellon01.intranet.dev.int.devlab.redhat.com
        SSLCertificateKeyFile /etc/pki/tls/certs/cert-mellon01.intranet.dev.int.devlab.redhat.com
        SSLVerifyClient none
        SSLVerifyDepth 10
        SSLSessionCacheTimeout  28800

        SetEnvIf User-Agent ".*MSIE.*" \
                nokeepalive ssl-unclean-shutdown \
                downgrade-1.0 force-response-1.0
    </IfModule>

    ServerAdmin sysadmin-rhcorp
    ErrorLog logs/mellon01.intranet.dev.int.devlab.redhat.com_log
    CustomLog logs/mellon01.intranet.dev.int.devlab.redhat.com-access.log common env=!redirect

</VirtualHost>

ScriptAlias /cgi-bin/ "/var/www/cgi-bin/"

[mellon01.intranet.dev.int.devlab.redhat.com] [03:56:38 PM]
[root@mellon01 conf.d]# pwd
/etc/httpd/conf.d

Comment 4 Simo Sorce 2014-08-20 13:30:20 UTC

Can you please load debuginfo modules for mod_auth_mellon and glib and attache new backtrace when it happens again ?

Comment 5 Brian J. Atkisson 2014-08-20 13:37:59 UTC

(In reply to Simo Sorce from comment #4)
> Can you please load debuginfo modules for mod_auth_mellon and glib and
> attache new backtrace when it happens again ?

will do.

Comment 7 Brian J. Atkisson 2014-08-20 15:15:38 UTC

OK, I installed the debug packages and was able to get a core.


10.13.111.1 - testuser [20/Aug/2014:11:11:53 -0400] "GET /secret/ HTTP/1.1" 200 4
10.13.111.1 - testuser [20/Aug/2014:11:11:54 -0400] "GET /secret/ HTTP/1.1" 304 -
10.13.111.1 - testuser [20/Aug/2014:11:11:57 -0400] "GET /secret/ HTTP/1.1" 304 -
10.13.111.1 - - [20/Aug/2014:11:12:15 -0400] "GET /secret/ HTTP/1.1" 303 507
10.13.111.1 - - [20/Aug/2014:11:12:16 -0400] "GET /secret/endpoint/login?ReturnTo=https%3A%2F%2Fmellon01.intranet.dev.int.devlab.redhat.com%2Fsecret%2F&IdP=https%3A%2F%2Fsaml.deviam.redhat.com%2Fidp%2F HTTP/1.1" 200 4817

==> /var/log/httpd/error_log.2014-08-20 <==
*** glibc detected *** /usr/sbin/httpd: double free or corruption (out): 0x00007fca64d0d5b0 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x3ceb476156)[0x7fca62403156]
/lib64/libc.so.6(+0x3ceb478ca3)[0x7fca62405ca3]
/usr/lib64/libxml2.so.2(xmlFreeNs+0x29)[0x7fca5a01d7a9]
/usr/lib64/libxml2.so.2(xmlFreeNsList+0x18)[0x7fca5a01d808]
/usr/lib64/libxml2.so.2(xmlFreeNodeList+0x11d)[0x7fca5a02322d]
/usr/lib64/libxml2.so.2(xmlFreeNodeList+0x83)[0x7fca5a023193]
/usr/lib64/libxml2.so.2(xmlFreeNodeList+0x83)[0x7fca5a023193]
/usr/lib64/libxml2.so.2(xmlFreeNodeList+0x83)[0x7fca5a023193]
/usr/lib64/libxml2.so.2(xmlFreeNode+0x76)[0x7fca5a023bd6]
/lib64/libglib-2.0.so.0(g_datalist_clear+0xa5)[0x7fca5bbe4285]
/lib64/libgobject-2.0.so.0(g_object_unref+0x174)[0x7fca5b026794]
/usr/lib64/liblasso.so.3(+0x22a5f)[0x7fca5c149a5f]
/lib64/libgobject-2.0.so.0(g_object_unref+0xea)[0x7fca5b02670a]
/etc/httpd/modules/mod_auth_mellon.so(+0xaa3f)[0x7fca5c9f9a3f]
/etc/httpd/modules/mod_auth_mellon.so(am_handler+0x1083)[0x7fca5c9fc343]
/usr/sbin/httpd(ap_run_handler+0x70)[0x7fca63e7dcd0]
/usr/sbin/httpd(ap_invoke_handler+0xce)[0x7fca63e8158e]
/usr/sbin/httpd(ap_process_request+0x1c0)[0x7fca63e8cc50]
/usr/sbin/httpd(+0x37ac8)[0x7fca63e89ac8]
/usr/sbin/httpd(ap_run_process_connection+0x68)[0x7fca63e857d8]
/usr/sbin/httpd(+0x3fad7)[0x7fca63e91ad7]
/usr/sbin/httpd(+0x3fdea)[0x7fca63e91dea]
/usr/sbin/httpd(ap_mpm_run+0x2cb)[0x7fca63e9211b]
/usr/sbin/httpd(main+0xb90)[0x7fca63e699b0]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x7fca623abd1d]
/usr/sbin/httpd(+0x16919)[0x7fca63e68919]
======= Memory map: ========
7fca549b3000-7fca549c9000 r-xp 00000000 fc:03 390246                     /lib64/libgcc_s-4.4.7-20120601.so.1
7fca549c9000-7fca54bc8000 ---p 00016000 fc:03 390246                     /lib64/libgcc_s-4.4.7-20120601.so.1
7fca54bc8000-7fca54bc9000 rw-p 00015000 fc:03 390246                     /lib64/libgcc_s-4.4.7-20120601.so.1
7fca54bc9000-7fca54bd8000 rw-s 00000000 00:04 335412                     /dev/zero (deleted)
7fca54bd8000-7fca54c56000 rw-s 00000000 00:04 335409                     /dev/zero (deleted)
7fca54c56000-7fca55f17000 rw-s 00000000 00:04 335408                     /dev/zero (deleted)
7fca55f17000-7fca568e7000 r--s 00000000 fc:03 2314                       /var/run/nscd/dbWKDpzm (deleted)
7fca568e7000-7fca572b7000 r--s 00000000 fc:03 946                        /var/run/nscd/dbs0SgYY (deleted)
7fca572b7000-7fca57c87000 r--s 00000000 fc:03 945                        /var/run/nscd/dbsjw9mB (deleted)
7fca57c87000-7fca57cb1000 r-xp 00000000 fc:03 536819                     /usr/lib64/httpd/modules/mod_ssl.so
7fca57cb1000-7fca57eb1000 ---p 0002a000 fc:03 536819                     /usr/lib64/httpd/modules/mod_ssl.so
7fca57eb1000-7fca57eb2000 r--p 0002a000 fc:03 536819                     /usr/lib64/httpd/modules/mod_ssl.so
7fca57eb2000-7fca57eb3000 rw-p 0002b000 fc:03 536819                     /usr/lib64/httpd/modules/mod_ssl.so
7fca57eb3000-7fca57eb6000 rw-p 00000000 00:00 0 
7fca57eb6000-7fca57ecf000 r-xp 00000000 fc:03 533622                     /usr/lib64/libsasl2.so.2.0.23
7fca57ecf000-7fca580ce000 ---p 00019000 fc:03 533622                     /usr/lib64/libsasl2.so.2.0.23
7fca580ce000-7fca580cf000 r--p 00018000 fc:03 533622                     /usr/lib64/libsasl2.so.2.0.23
7fca580cf000-7fca580d0000 rw-p 00019000 fc:03 533622                     /usr/lib64/libsasl2.so.2.0.23
7fca580d0000-7fca580de000 r-xp 00000000 fc:03 390238                     /lib64/liblber-2.4.so.2.10.2
7fca580de000-7fca582dd000 ---p 0000e000 fc:03 390238                     /lib64/liblber-2.4.so.2.10.2
7fca582dd000-7fca582de000 r--p 0000d000 fc:03 390238                     /lib64/liblber-2.4.so.2.10.2
7fca582de000-7fca582df000 rw-p 0000e000 fc:03 390238                     /lib64/liblber-2.4.so.2.10.2
7fca582df000-7fca582f5000 r-xp 00000000 fc:03 390201                     /lib64/libresolv-2.12.so
7fca582f5000-7fca584f5000 ---p 00016000 fc:03 390201                     /lib64/libresolv-2.12.so
7fca584f5000-7fca584f6000 r--p 00016000 fc:03 390201                     /lib64/libresolv-2.12.so
7fca584f6000-7fca584f7000 rw-p 00017000 fc:03 390201                     /lib64/libresolv-2.12.so
7fca584f7000-7fca584f9000 rw-p 00000000 00:00 0 
7fca584f9000-7fca584fb000 r-xp 00000000 fc:03 390199                     /lib64/libkeyutils.so.1.3
7fca584fb000-7fca586fa000 ---p 00002000 fc:03 390199                     /lib64/libkeyutils.so.1.3
7fca586fa000-7fca586fb000 r--p 00001000 fc:03 390199                     /lib64/libkeyutils.so.1.3
7fca586fb000-7fca586fc000 rw-p 00002000 fc:03 390199                     /lib64/libkeyutils.so.1.3
7fca586fc000-7fca58706000 r-xp 00000000 fc:03 390207                     /lib64/libkrb5support.so.0.1
7fca58706000-7fca58905000 ---p 0000a000 fc:03 390207                     /lib64/libkrb5support.so.0.1
7fca58905000-7fca58906000 r--p 00009000 fc:03 390207                     /lib64/libkrb5support.so.0.1
7fca58906000-7fca58907000 rw-p 0000a000 fc:03 390207                     /lib64/libkrb5support.so.0.1
7fca58907000-7fca5892e000 r-xp 00000000 fc:03 533663                     /usr/lib64/libssh2.so.1.0.1
7fca5892e000-7fca58b2d000 ---p 00027000 fc:03 533663                     /usr/lib64/libssh2.so.1.0.1
7fca58b2d000-7fca58b2e000 rw-p 00026000 fc:03 533663                     /usr/lib64/libssh2.so.1.0.1
7fca58b2e000-7fca58b2f000 rw-p 00000000 00:00 0 
7fca58b2f000-7fca58b68000 r-xp 00000000 fc:03 390231                     /lib64/libnspr4.so
7fca58b68000-7fca58d68000 ---p 00039000 fc:03 390231                     /lib64/libnspr4.so
7fca58d68000-7fca58d69000 r--p 00039000 fc:03 390231                     /lib64/libnspr4.so
7fca58d69000-7fca58d6b000 rw-p 0003a000 fc:03 390231                     /lib64/libnspr4.so
7fca58d6b000-7fca58d6d000 rw-p 00000000 00:00 0 
7fca58d6d000-7fca58d71000 r-xp 00000000 fc:03 390233                     /lib64/libplc4.so
7fca58d71000-7fca58f70000 ---p 00004000 fc:03 390233                     /lib64/libplc4.so
7fca58f70000-7fca58f71000 r--p 00003000 fc:03 390233                     /lib64/libplc4.so
7fca58f71000-7fca58f72000 rw-p 00004000 fc:03 390233                     /lib64/libplc4.so
7fca58f72000-7fca58f75000 r-xp 00000000 fc:03 390235                     /lib64/libplds4.so
7fca58f75000-7fca59174000 ---p 00003000 fc:03 390235                     /lib64/libplds4.so
7fca59174000-7fca59175000 r--p 00002000 fc:03 390235                     /lib64/libplds4.so
7fca59175000-7fca59176000 rw-p 00003000 fc:03 390235                     /lib64/libplds4.so
7fca59176000-7fca5919b000 r-xp 00000000 fc:03 533618                     /usr/lib64/libnssutil3.so
7fca5919b000-7fca5939b000 ---p 00025000 fc:03 533618                     /usr/lib64/libnssutil3.so
7fca5939b000-7fca593a1000 r--p 00025000 fc:03 533618                     /usr/lib64/libnssutil3.so
7fca593a1000-7fca593a2000 rw-p 0002b000 fc:03 533618                     /usr/lib64/libnssutil3.so
7fca593a2000-7fca594d8000 r-xp 00000000 fc:03 533619                     /usr/lib64/libnss3.so
7fca594d8000-7fca596d8000 ---p 00136000 fc:03 533619                     /usr/lib64/libnss3.so
7fca596d8000-7fca596dd000 r--p 00136000 fc:03 533619                     /usr/lib64/libnss3.so
7fca596dd000-7fca596df000 rw-p 0013b000 fc:03 533619                     /usr/lib64/libnss3.so
7fca596df000-7fca596e1000 rw-p 00000000 00:00 0 
7fca596e1000-7fca59709000 r-xp 00000000 fc:03 533621                     /usr/lib64/libsmime3.so
7fca59709000-7fca59908000 ---p 00028000 fc:03 533621                     /usr/lib64/libsmime3.so
7fca59908000-7fca5990c000 r--p 00027000 fc:03 533621                     /usr/lib64/libsmime3.so
7fca5990c000-7fca5990d000 rw-p 0002b000 fc:03 533621                     /usr/lib64/libsmime3.so
7fca5990d000-7fca59947000 r-xp 00000000 fc:03 533620                     /usr/lib64/libssl3.so
7fca59947000-7fca59b47000 ---p 0003a000 fc:03 533620                     /usr/lib64/libssl3.so
7fca59b47000-7fca59b4a000 r--p 0003a000 fc:03 533620                     /usr/lib64/libssl3.so
7fca59b4a000-7fca59b4b000 rw-p 0003d000 fc:03 533620                     /usr/lib64/libssl3.so
7fca59b4b000-7fca59b4c000 rw-p 00000000 00:00 0 
7fca59b4c000-7fca59b99000 r-xp 00000000 fc:03 390316                     /lib64/libldap-2.4.so.2.10.2
7fca59b99000-7fca59d98000 ---p 0004d000 fc:03 390316                     /lib64/libldap-2.4.so.2.10.2
7fca59d98000-7fca59d9a000 r--p 0004c000 fc:03 390316                     /lib64/libldap-2.4.so.2.10.2
7fca59d9a000-7fca59d9c000 rw-p 0004e000 fc:03 390316                     /lib64/libldap-2.4.so.2.10.2
7fca59d9c000-7fca59dce000 r-xp 00000000 fc:03 390357                     /lib64/libidn.so.11.6.1
7fca59dce000-7fca59fcd000 ---p 00032000 fc:03 390357                     /lib64/libidn.so.11.6.1
7fca59fcd000-7fca59fce000 rw-p 00031000 fc:03 390357                     /lib64/libidn.so.11.6.1
7fca59fce000-7fca5a116000 r-xp 00000000 fc:03 528176                     /usr/lib64/libxml2.so.2.7.6
7fca5a116000-7fca5a316000 ---p 00148000 fc:03 528176                     /usr/lib64/libxml2.so.2.7.6
7fca5a316000-7fca5a31f000 rw-p 00148000 fc:03 528176                     /usr/lib64/libxml2.so.2.7.6
7fca5a31f000-7fca5a321000 rw-p 00000000 00:00 0 
7fca5a321000-7fca5a35c000 r-xp 00000000 fc:03 527548                     /usr/lib64/libxslt.so.1.1.26
7fca5a35c000-7fca5a55c000 ---p 0003b000 fc:03 527548                     /usr/lib64/libxslt.so.1.1.26
7fca5a55c000-7fca5a55e000 rw-p 0003b000 fc:03 527548                     /usr/lib64/libxslt.so.1.1.26
7fca5a55e000-7fca5a567000 r-xp 00000000 fc:03 527916                     /usr/lib64/libltdl.so.7.2.1
7fca5a567000-7fca5a766000 ---p 00009000 fc:03 527916                     /usr/lib64/libltdl.so.7.2.1
7fca5a766000-7fca5a767000 rw-p 00008000 fc:03 527916                     /usr/lib64/libltdl.so.7.2.1
7fca5a767000-7fca5a7cb000 r-xp 00000000 fc:03 533674                     /usr/lib64/libxmlsec1.so.1.2.20
7fca5a7cb000-7fca5a9ca000 ---p 00064000 fc:03 533674                     /usr/lib64/libxmlsec1.so.1.2.20
7fca5a9ca000-7fca5a9ce000 rw-p 00063000 fc:03 533674                     /usr/lib64/libxmlsec1.so.1.2.20
7fca5a9ce000-7fca5aa06000 r-xp 00000000 fc:03 536705                     /usr/lib64/libxmlsec1-openssl.so.1.2.20
7fca5aa06000-7fca5ac05000 ---p 00038000 fc:03 536705                     /usr/lib64/libxmlsec1-openssl.so.1.2.20
7fca5ac05000-7fca5ac09000 rw-p 00037000 fc:03 536705                     /usr/lib64/libxmlsec1-openssl.so.1.2.20
7fca5ac09000-7fca5ac0a000 rw-p 00000000 00:00 0 
7fca5ac0a000-7fca5ac11000 r-xp 00000000 fc:03 390229                     /lib64/librt-2.12.so
7fca5ac11000-7fca5ae10000 ---p 00007000 fc:03 390229                     /lib64/librt-2.12.so
7fca5ae10000-7fca5ae11000 r--p 00006000 fc:03 390229                     /lib64/librt-2.12.so
7fca5ae11000-7fca5ae12000 rw-p 00007000 fc:03 390229                     /lib64/librt-2.12.so
7fca5ae12000-7fca5ae16000 r-xp 00000000 fc:03 390258                     /lib64/libgthread-2.0.so.0.2800.8
7fca5ae16000-7fca5b015000 ---p 00004000 fc:03 390258                     /lib64/libgthread-2.0.so.0.2800.8
7fca5b015000-7fca5b016000 rw-p 00003000 fc:03 390258                     /lib64/libgthread-2.0.so.0.2800.8
7fca5b016000-7fca5b060000 r-xp 00000000 fc:03 390265                     /lib64/libgobject-2.0.so.0.2800.8
7fca5b060000-7fca5b25f000 ---p 0004a000 fc:03 390265                     /lib64/libgobject-2.0.so.0.2800.8
7fca5b25f000-7fca5b261000 rw-p 00049000 fc:03 390265                     /lib64/libgobject-2.0.so.0.2800.8
7fca5b261000-7fca5b262000 rw-p 00000000 00:00 0 
7fca5b262000-7fca5b28b000 r-xp 00000000 fc:03 390220                     /lib64/libk5crypto.so.3.1
7fca5b28b000-7fca5b48b000 ---p 00029000 fc:03 390220                     /lib64/libk5crypto.so.3.1
7fca5b48b000-7fca5b48c000 r--p 00029000 fc:03 390220                     /lib64/libk5crypto.so.3.1
7fca5b48c000-7fca5b48d000 rw-p 0002a000 fc:03 390220                     /lib64/libk5crypto.so.3.1
7fca5b48d000-7fca5b48e000 rw-p 00000000 00:00 0 
7fca5b48e000-7fca5b491000 r-xp 00000000 fc:03 390223                     /lib64/libcom_err.so.2.1
7fca5b491000-7fca5b690000 ---p 00003000 fc:03 390223                     /lib64/libcom_err.so.2.1
7fca5b690000-7fca5b691000 r--p 00002000 fc:03 390223                     /lib64/libcom_err.so.2.1
7fca5b691000-7fca5b692000 rw-p 00003000 fc:03 390223                     /lib64/libcom_err.so.2.1
7fca5b692000-7fca5b76d000 r-xp 00000000 fc:03 390226                     /lib64/libkrb5.so.3.3
7fca5b76d000-7fca5b96c000 ---p 000db000 fc:03 390226                     /lib64/libkrb5.so.3.3
7fca5b96c000-7fca5b976000 r--p 000da000 fc:03 390226                     /lib64/libkrb5.so.3.3
7fca5b976000-7fca5b978000 rw-p 000e4000 fc:03 390226                     /lib64/libkrb5.so.3.3
7fca5b978000-7fca5b9b9000 r-xp 00000000 fc:03 390228                     /lib64/libgssapi_krb5.so.2.2
7fca5b9b9000-7fca5bbb9000 ---p 00041000 fc:03 390228                     /lib64/libgssapi_krb5.so.2.2
7fca5bbb9000-7fca5bbba000 r--p 00041000 fc:03 390228                     /lib64/libgssapi_krb5.so.2.2
7fca5bbba000-7fca5bbbc000 rw-p 00042000 fc:03 390228                     /lib64/libgssapi_krb5.so.2.2
7fca5bbbc000-7fca5bcd1000 r-xp 00000000 fc:03 390252                     /lib64/libglib-2.0.so.0.2800.8
7fca5bcd1000-7fca5bed1000 ---p 00115000 fc:03 390252                     /lib64/libglib-2.0.so.0.2800.8
7fca5bed1000-7fca5bed2000 rw-p 00115000 fc:03 390252                     /lib64/libglib-2.0.so.0.2800.8
7fca5bed2000-7fca5bed3000 rw-p 00000000 00:00 0 
7fca5bed3000-7fca5bf25000 r-xp 00000000 fc:03 533664                     /usr/lib64/libcurl.so.4.1.1
7fca5bf25000-7fca5c124000 ---p 00052000 fc:03 533664                     /usr/lib64/libcurl.so.4.1.1
7fca5c124000-7fca5c127000 rw-p 00051000 fc:03 533664                     /usr/lib64/libcurl.so.4.1.1
7fca5c127000-7fca5c195000 r-xp 00000000 fc:03 536707                     /usr/lib64/liblasso.so.3.10.0
7fca5c195000-7fca5c395000 ---p 0006e000 fc:03 536707                     /usr/lib64/liblasso.so.3.10.0
7fca5c395000-7fca5c39f000 rw-p 0006e000 fc:03 536707                     /usr/lib64/liblasso.so.3.10.0
7fca5c39f000-7fca5c3a0000 rw-p 00000000 00:00 0 
7fca5c3a0000-7fca5c559000 r-xp 00000000 fc:03 525303                     /usr/lib64/libcrypto.so.1.0.1e
7fca5c559000-7fca5c758000 ---p 001b9000 fc:03 525303                     /usr/lib64/libcrypto.so.1.0.1e
7fca5c758000-7fca5c773000 r--p 001b8000 fc:03 525303                     /usr/lib64/libcrypto.so.1.0.1e
7fca5c773000-7fca5c77f000 rw-p 001d3000 fc:03 525303                     /usr/lib64/libcrypto.so.1.0.1e
7fca5c77f000-7fca5c783000 rw-p 00000000 00:00 0 
7fca5c783000-7fca5c7e5000 r-xp 00000000 fc:03 525552                     /usr/lib64/libssl.so.1.0.1e
7fca5c7e5000-7fca5c9e4000 ---p 00062000 fc:03 525552                     /usr/lib64/libssl.so.1.0.1e
7fca5c9e4000-7fca5c9e8000 r--p 00061000 fc:03 525552                     /usr/lib64/libssl.so.1.0.1e
7fca5c9e8000-7fca5c9ef000 rw-p 00065000 fc:03 525552                     /usr/lib64/libssl.so.1.0.1e
7fca5c9ef000-7fca5ca06000 r-xp 00000000 fc:03 536814                     /usr/lib64/httpd/modules/mod_auth_mellon.so
7fca5ca06000-7fca5cc05000 ---p 00017000 fc:03 536814                     /usr/lib64/httpd/modules/mod_auth_mellon.so
7fca5cc05000-7fca5cc07000 rw-p 00016000 fc:03 536814                     /usr/lib64/httpd/modules/mod_auth_mellon.so
7fca5cc07000-7fca5cc09000 r-xp 00000000 fc:03 536777                     /usr/lib64/httpd/modules/mod_proxy_connect.so
7fca5cc09000-7fca5ce09000 ---p 00002000 fc:03 536777                     /usr/lib64/httpd/modules/mod_proxy_connect.so
7fca5ce09000-7fca5ce0a000 r--p 00002000 fc:03 536777                     /usr/lib64/httpd/modules/mod_proxy_connect.so
7fca5ce0a000-7fca5ce0b000 rw-p 00003000 fc:03 536777                     /usr/lib64/httpd/modules/mod_proxy_connect.so
7fca5ce0b000-7fca5ce12000 r-xp 00000000 fc:03 536779                     /usr/lib64/httpd/modules/mod_proxy_http.so
7fca5ce12000-7fca5d011000 ---p 00007000 fc:03 536779                     /usr/lib64/httpd/modules/mod_proxy_http.so
7fca5d011000-7fca5d012000 r--p 00006000 fc:03 536779                     /usr/lib64/httpd/modules/mod_proxy_http.so
7fca5d012000-7fca5d013000 rw-p 00007000 fc:03 536779                     /usr/lib64/httpd/modules/mod_proxy_http.so
7fca5d013000-7fca5d019000 r-xp 00000000 fc:03 536776                     /usr/lib64/httpd/modules/mod_proxy_balancer.so
7fca5d019000-7fca5d218000 ---p 00006000 fc:03 536776                     /usr/lib64/httpd/modules/mod_proxy_balancer.so
7fca5d218000-7fca5d219000 r--p 00005000 fc:03 536776                     /usr/lib64/httpd/modules/mod_proxy_balancer.so
7fca5d219000-7fca5d21a000 rw-p 00006000 fc:03 536776                     /usr/lib64/httpd/modules/mod_proxy_balancer.so
7fca5d21a000-7fca5d22e000 r-xp 00000000 fc:03 536774                     /usr/lib64/httpd/modules/mod_proxy.so
7fca5d22e000-7fca5d42d000 ---p 00014000 fc:03 536774                     /usr/lib64/httpd/modules/mod_proxy.so
7fca5d42d000-7fca5d42e000 r--p 00013000 fc:03 536774                     /usr/lib64/httpd/modules/mod_proxy.so
7fca5d42e000-7fca5d42f000 rw-p 00014000 fc:03 536774                     /usr/lib64/httpd/modules/mod_proxy.so
7fca5d42f000-7fca5d43d000 r-xp 00000000 fc:03 536782                     /usr/lib64/httpd/modules/mod_rewrite.so
7fca5d43d000-7fca5d63c000 ---p 0000e000 fc:03 536782                     /usr/lib64/httpd/modules/mod_rewrite.so
7fca5d63c000-7fca5d63d000 r--p 0000d000 fc:03 536782                     /usr/lib64/httpd/modules/mod_rewrite.so
7fca5d63d000-7fca5d63e000 rw-p 0000e000 fc:03 536782                     /usr/lib64/httpd/modules/mod_rewrite.so
7fca5d63e000-7fca5d641000 r-xp 00000000 fc:03 536730                     /usr/lib64/httpd/modules/mod_alias.so
7fca5d641000-7fca5d840000 ---p 00003000 fc:03 536730                     /usr/lib64/httpd/modules/mod_alias.so
7fca5d840000-7fca5d841000 r--p 00002000 fc:03 536730                     /usr/lib64/httpd/modules/mod_alias.so
7fca5d841000-7fca5d842000 rw-p 00003000 fc:03 536730                     /usr/lib64/httpd/modules/mod_alias.so
7fca5d842000-7fca5d844000 r-xp 00000000 fc:03 536756                     /usr/lib64/httpd/modules/mod_dir.so
7fca5d844000-7fca5da43000 ---p 00002000 fc:03 536756                     /usr/lib64/httpd/modules/mod_dir.so
7fca5da43000-7fca5da44000 r--p 00001000 fc:03 536756                     /usr/lib64/httpd/modules/mod_dir.so
7fca5da44000-7fca5da45000 rw-p 00002000 fc:03 536756                     /usr/lib64/httpd/modules/mod_dir.so
7fca5da45000-7fca5da4c000 r-xp 00000000 fc:03 536773                     /usr/lib64/httpd/modules/mod_negotiation.so
7fca5da4c000-7fca5dc4b000 ---p 00007000 fc:03 536773                     /usr/lib64/httpd/modules/mod_negotiation.so
7fca5dc4b000-7fca5dc4c000 r--p 00006000 fc:03 536773                     /usr/lib64/httpd/modules/mod_negotiation.so
7fca5dc4c000-7fca5dc4d000 rw-p 00007000 fc:03 536773                     /usr/lib64/httpd/modules/mod_negotiation.so
7fca5dc4d000-7fca5dc4f000 r-xp 00000000 fc:03 536792                     /usr/lib64/httpd/modules/mod_vhost_alias.so
7fca5dc4f000-7fca5de4e000 ---p 00002000 fc:03 536792                     /usr/lib64/httpd/modules/mod_vhost_alias.so
7fca5de4e000-7fca5de4f000 r--p 00001000 fc:03 536792                     /usr/lib64/httpd/modules/mod_vhost_alias.so
7fca5de4f000-7fca5de50000 rw-p 00002000 fc:03 536792                     /usr/lib64/httpd/modules/mod_vhost_alias.so
7fca5de50000-7fca5de55000 r-xp 00000000 fc:03 536750                     /usr/lib64/httpd/modules/mod_cgi.so
7fca5de55000-7fca5e055000 ---p 00005000 fc:03 536750                     /usr/lib64/httpd/modules/mod_cgi.so
7fca5e055000-7fca5e056000 r--p 00005000 fc:03 536750                     /usr/lib64/httpd/modules/mod_cgi.so
7fca5e056000-7fca5e057000 rw-p 00006000 fc:03 536750                     /usr/lib64/httpd/modules/mod_cgi.so
7fca5e057000-7fca5e05b000 r-xp 00000000 fc:03 536766                     /usr/lib64/httpd/modules/mod_info.so
7fca5e05b000-7fca5e25a000 ---p 00004000 fc:03 536766                     /usr/lib64/httpd/modules/mod_info.so
7fca5e25a000-7fca5e25b000 r--p 00003000 fc:03 536766                     /usr/lib64/httpd/modules/mod_info.so
7fca5e25b000-7fca5e25c000 rw-p 00004000 fc:03 536766                     /usr/lib64/httpd/modules/mod_info.so
7fca5e25c000-7fca5e264000 r-xp 00000000 fc:03 536747                     /usr/lib64/httpd/modules/mod_autoindex.so
7fca5e264000-7fca5e463000 ---p 00008000 fc:03 536747                     /usr/lib64/httpd/modules/mod_autoindex.so
7fca5e463000-7fca5e464000 r--p 00007000 fc:03 536747                     /usr/lib64/httpd/modules/mod_autoindex.so
7fca5e464000-7fca5e465000 rw-p 00008000 fc:03 536747                     /usr/lib64/httpd/modules/mod_autoindex.so
7fca5e465000-7fca5e469000 r-xp 00000000 fc:03 536785                     /usr/lib64/httpd/modules/mod_status.so
7fca5e469000-7fca5e669000 ---p 00004000 fc:03 536785                     /usr/lib64/httpd/modules/mod_status.so
7fca5e669000-7fca5e66a000 r--p 00004000 fc:03 536785                     /usr/lib64/httpd/modules/mod_status.so
7fca5e66a000-7fca5e66b000 rw-p 00005000 fc:03 536785                     /usr/lib64/httpd/modules/mod_status.so
7fca5e66b000-7fca5e66f000 r-xp 00000000 fc:03 536771                     /usr/lib64/httpd/modules/mod_mime.so
7fca5e66f000-7fca5e86e000 ---p 00004000 fc:03 536771                     /usr/lib64/httpd/modules/mod_mime.so
7fca5e86e000-7fca5e86f000 r--p 00003000 fc:03 536771                     /usr/lib64/httpd/modules/mod_mime.so
7fca5e86f000-7fca5e870000 rw-p 00004000 fc:03 536771                     /usr/lib64/httpd/modules/mod_mime.so
7fca5e870000-7fca5e872000 r-xp 00000000 fc:03 536783                     /usr/lib64/httpd/modules/mod_setenvif.so
7fca5e872000-7fca5ea72000 ---p 00002000 fc:03 536783                     /usr/lib64/httpd/modules/mod_setenvif.so
7fca5ea72000-7fca5ea73000 r--p 00002000 fc:03 536783                     /usr/lib64/httpd/modules/mod_setenvif.so
7fca5ea73000-7fca5ea74000 rw-p 00003000 fc:03 536783                     /usr/lib64/httpd/modules/mod_setenvif.so
7fca5ea74000-7fca5ea78000 r-xp 00000000 fc:03 536763                     /usr/lib64/httpd/modules/mod_headers.so
7fca5ea78000-7fca5ec77000 ---p 00004000 fc:03 536763                     /usr/lib64/httpd/modules/mod_headers.so
7fca5ec77000-7fca5ec78000 r--p 00003000 fc:03 536763                     /usr/lib64/httpd/modules/mod_headers.so
7fca5ec78000-7fca5ec79000 rw-p 00004000 fc:03 536763                     /usr/lib64/httpd/modules/mod_headers.so
7fca5ec79000-7fca5ec8e000 r-xp 00000000 fc:03 390237                     /lib64/libz.so.1.2.3
7fca5ec8e000-7fca5ee8d000 ---p 00015000 fc:03 390237                     /lib64/libz.so.1.2.3
7fca5ee8d000-7fca5ee8e000 r--p 00014000 fc:03 390237                     /lib64/libz.so.1.2.3
7fca5ee8e000-7fca5ee8f000 rw-p 00015000 fc:03 390237                     /lib64/libz.so.1.2.3
7fca5ee8f000-7fca5ee94000 r-xp 00000000 fc:03 536755                     /usr/lib64/httpd/modules/mod_deflate.so
7fca5ee94000-7fca5f094000 ---p 00005000 fc:03 536755                     /usr/lib64/httpd/modules/mod_deflate.so
7fca5f094000-7fca5f095000 r--p 00005000 fc:03 536755                     /usr/lib64/httpd/modules/mod_deflate.so
7fca5f095000-7fca5f096000 rw-p 00006000 fc:03 536755                     /usr/lib64/httpd/modules/mod_deflate.so
7fca5f096000-7fca5f098000 r-xp 00000000 fc:03 536760                     /usr/lib64/httpd/modules/mod_expires.so
7fca5f098000-7fca5f297000 ---p 00002000 fc:03 536760                     /usr/lib64/httpd/modules/mod_expires.so
7fca5f297000-7fca5f298000 r--p 00001000 fc:03 536760                     /usr/lib64/httpd/modules/mod_expires.so
7fca5f298000-7fca5f299000 rw-p 00002000 fc:03 536760                     /usr/lib64/httpd/modules/mod_expires.so
7fca5f299000-7fca5f29e000 r-xp 00000000 fc:03 536772                     /usr/lib64/httpd/modules/mod_mime_magic.so
7fca5f29e000-7fca5f49e000 ---p 00005000 fc:03 536772                     /usr/lib64/httpd/modules/mod_mime_magic.so
7fca5f49e000-7fca5f49f000 r--p 00005000 fc:03 536772                     /usr/lib64/httpd/modules/mod_mime_magic.so
7fca5f49f000-7fca5f4a0000 rw-p 00006000 fc:03 536772                     /usr/lib64/httpd/modules/mod_mime_magic.so
7fca5f4a0000-7fca5f4a5000 r-xp 00000000 fc:03 536761                     /usr/lib64/httpd/modules/mod_ext_filter.so
7fca5f4a5000-7fca5f6a4000 ---p 00005000 fc:03 536761                     /usr/lib64/httpd/modules/mod_ext_filter.so
7fca5f6a4000-7fca5f6a5000 r--p 00004000 fc:03 536761                     /usr/lib64/httpd/modules/mod_ext_filter.so
7fca5f6a5000-7fca5f6a6000 rw-p 00005000 fc:03 536761                     /usr/lib64/httpd/modules/mod_ext_filter.so
7fca5f6a6000-7fca5f6a8000 r-xp 00000000 fc:03 536759                     /usr/lib64/httpd/modules/mod_env.so
7fca5f6a8000-7fca5f8a7000 ---p 00002000 fc:03 536759                     /usr/lib64/httpd/modules/mod_env.so
7fca5f8a7000-7fca5f8a8000 r--p 00001000 fc:03 536759                     /usr/lib64/httpd/modules/mod_env.so
7fca5f8a8000-7fca5f8a9000 rw-p 00002000 fc:03 536759                     /usr/lib64/httpd/modules/mod_env.so
7fca5f8a9000-7fca5f8ab000 r-xp 00000000 fc:03 536770                     /usr/lib64/httpd/modules/mod_logio.so
7fca5f8ab000-7fca5faaa000 ---p 00002000 fc:03 536770                     /usr/lib64/httpd/modules/mod_logio.so
7fca5faaa000-7fca5faab000 r--p 00001000 fc:03 536770                     /usr/lib64/httpd/modules/mod_logio.so
7fca5faab000-7fca5faac000 rw-p 00002000 fc:03 536770                     /usr/lib64/httpd/modules/mod_logio.so
7fca5faac000-7fca5fab1000 r-xp 00000000 fc:03 536768                     /usr/lib64/httpd/modules/mod_log_config.so
7fca5fab1000-7fca5fcb1000 ---p 00005000 fc:03 536768                     /usr/lib64/httpd/modules/mod_log_config.so
7fca5fcb1000-7fca5fcb2000 r--p 00005000 fc:03 536768                     /usr/lib64/httpd/modules/mod_log_config.so
7fca5fcb2000-7fca5fcb3000 rw-p 00006000 fc:03 536768                     /usr/lib64/httpd/modules/mod_log_config.so
7fca5fcb3000-7fca5fcbd000 r-xp 00000000 fc:03 536765                     /usr/lib64/httpd/modules/mod_include.so
7fca5fcbd000-7fca5febc000 ---p 0000a000 fc:03 536765                     /usr/lib64/httpd/modules/mod_include.so
7fca5febc000-7fca5febd000 r--p 00009000 fc:03 536765                     /usr/lib64/httpd/modules/mod_include.so
7fca5febd000-7fca5febe000 rw-p 0000a000 fc:03 536765                     /usr/lib64/httpd/modules/mod_include.so
7fca5febe000-7fca5fec9000 r-xp 00000000 fc:03 536753                     /usr/lib64/httpd/modules/mod_dav_fs.so
7fca5fec9000-7fca600c9000 ---p 0000b000 fc:03 536753                     /usr/lib64/httpd/modules/mod_dav_fs.so
7fca600c9000-7fca600ca000 r--p 0000b000 fc:03 536753                     /usr/lib64/httpd/modules/mod_dav_fs.so
7fca600ca000-7fca600cb000 rw-p 0000c000 fc:03 536753                     /usr/lib64/httpd/modules/mod_dav_fs.so
7fca600cb000-7fca600e1000 r-xp 00000000 fc:03 536752                     /usr/lib64/httpd/modules/mod_dav.so
7fca600e1000-7fca602e0000 ---p 00016000 fc:03 536752                     /usr/lib64/httpd/modules/mod_dav.so
7fca602e0000-7fca602e1000 r--p 00015000 fc:03 536752                     /usr/lib64/httpd/modules/mod_dav.so
7fca602e1000-7fca602e2000 rw-p 00016000 fc:03 536752                     /usr/lib64/httpd/modules/mod_dav.so
7fca602e2000-7fca602e3000 r-xp 00000000 fc:03 536742                     /usr/lib64/httpd/modules/mod_authz_default.so
7fca602e3000-7fca604e2000 ---p 00001000 fc:03 536742                     /usr/lib64/httpd/modules/mod_authz_default.so
7fca604e2000-7fca604e3000 r--p 00000000 fc:03 536742                     /usr/lib64/httpd/modules/mod_authz_default.so
7fca604e3000-7fca604e4000 rw-p 00001000 fc:03 536742                     /usr/lib64/httpd/modules/mod_authz_default.so
7fca604e4000-7fca604e6000 r-xp 00000000 fc:03 536741                     /usr/lib64/httpd/modules/mod_authz_dbm.so
7fca604e6000-7fca606e5000 ---p 00002000 fc:03 536741                     /usr/lib64/httpd/modules/mod_authz_dbm.so
7fca606e5000-7fca606e6000 r--p 00001000 fc:03 536741                     /usr/lib64/httpd/modules/mod_authz_dbm.so
7fca606e6000-7fca606e7000 rw-p 00002000 fc:03 536741                     /usr/lib64/httpd/modules/mod_authz_dbm.so
7fca606e7000-7fca606e9000 r-xp 00000000 fc:03 536743                     /usr/lib64/httpd/modules/mod_authz_groupfile.so
7fca606e9000-7fca608e8000 ---p 00002000 fc:03 536743                     /usr/lib64/httpd/modules/mod_authz_groupfile.so
7fca608e8000-7fca608e9000 r--p 00001000 fc:03 536743                     /usr/lib64/httpd/modules/mod_authz_groupfile.so
7fca608e9000-7fca608ea000 rw-p 00002000 fc:03 536743                     /usr/lib64/httpd/modules/mod_authz_groupfile.so
7fca608ea000-7fca608ec000 r-xp 00000000 fc:03 536745                     /usr/lib64/httpd/modules/mod_authz_owner.so
7fca608ec000-7fca60aeb000 ---p 00002000 fc:03 536745                     /usr/lib64/httpd/modules/mod_authz_owner.so
7fca60aeb000-7fca60aec000 r--p 00001000 fc:03 536745                     /usr/lib64/httpd/modules/mod_authz_owner.so
7fca60aec000-7fca60aed000 rw-p 00002000 fc:03 536745                     /usr/lib64/httpd/modules/mod_authz_owner.so
7fca60aed000-7fca60aee000 r-xp 00000000 fc:03 536746                     /usr/lib64/httpd/modules/mod_authz_user.so
7fca60aee000-7fca60cee000 ---p 00001000 fc:03 536746                     /usr/lib64/httpd/modules/mod_authz_user.so
7fca60cee000-7fca60cef000 r--p 00001000 fc:03 536746                     /usr/lib64/httpd/modules/mod_authz_user.so
7fca60cef000-7fca60cf0000 rw-p 00002000 fc:03 536746                     /usr/lib64/httpd/modules/mod_authz_user.so
7fca60cf0000-7fca60cf2000 r-xp 00000000 fc:03 536744                     /usr/lib64/httpd/modules/mod_authz_host.so
7fca60cf2000-7fca60ef1000 ---p 00002000 fc:03 536744                     /usr/lib64/httpd/modules/mod_authz_host.so
7fca60ef1000-7fca60ef2000 r--p 00001000 fc:03 536744                     /usr/lib64/httpd/modules/mod_authz_host.so
7fca60ef2000-7fca60ef3000 rw-p 00002000 fc:03 536744                     /usr/lib64/httpd/modules/mod_authz_host.so
7fca60ef3000-7fca60ef4000 r-xp 00000000 fc:03 536738                     /usr/lib64/httpd/modules/mod_authn_default.so
7fca60ef4000-7fca610f3000 ---p 00001000 fc:03 536738                     /usr/lib64/httpd/modules/mod_authn_default.so
7fca610f3000-7fca610f4000 r--p 00000000 fc:03 536738                     /usr/lib64/httpd/modules/mod_authn_default.so
7fca610f4000-7fca610f5000 rw-p 00001000 fc:03 536738                     /usr/lib64/httpd/modules/mod_authn_default.so
7fca610f5000-7fca610f7000 r-xp 00000000 fc:03 536737                     /usr/lib64/httpd/modules/mod_authn_dbm.so
7fca610f7000-7fca612f6000 ---p 00002000 fc:03 536737                     /usr/lib64/httpd/modules/mod_authn_dbm.so
7fca612f6000-7fca612f7000 r--p 00001000 fc:03 536737                     /usr/lib64/httpd/modules/mod_authn_dbm.so
7fca612f7000-7fca612f8000 rw-p 00002000 fc:03 536737                     /usr/lib64/httpd/modules/mod_authn_dbm.so
7fca612f8000-7fca612f9000 r-xp 00000000 fc:03 536735                     /usr/lib64/httpd/modules/mod_authn_anon.so
7fca612f9000-7fca614f9000 ---p 00001000 fc:03 536735                     /usr/lib64/httpd/modules/mod_authn_anon.so
7fca614f9000-7fca614fa000 r--p 00001000 fc:03 536735                     /usr/lib64/httpd/modules/mod_authn_anon.so
7fca614fa000-7fca614fb000 rw-p 00002000 fc:03 536735                     /usr/lib64/httpd/modules/mod_authn_anon.so
7fca614fb000-7fca614fd000 r-xp 00000000 fc:03 536734                     /usr/lib64/httpd/modules/mod_authn_alias.so
7fca614fd000-7fca616fc000 ---p 00002000 fc:03 536734                     /usr/lib64/httpd/modules/mod_authn_alias.so
7fca616fc000-7fca616fd000 r--p 00001000 fc:03 536734                     /usr/lib64/httpd/modules/mod_authn_alias.so
7fca616fd000-7fca616fe000 rw-p 00002000 fc:03 536734                     /usr/lib64/httpd/modules/mod_authn_alias.so
7fca616fe000-7fca61700000 r-xp 00000000 fc:03 536739                     /usr/lib64/httpd/modules/mod_authn_file.so
7fca61700000-7fca618ff000 ---p 00002000 fc:03 536739                     /usr/lib64/httpd/modules/mod_authn_file.so
7fca618ff000-7fca61900000 r--p 00001000 fc:03 536739                     /usr/lib64/httpd/modules/mod_authn_file.so
7fca61900000-7fca61901000 rw-p 00002000 fc:03 536739                     /usr/lib64/httpd/modules/mod_authn_file.so
7fca61901000-7fca61907000 r-xp 00000000 fc:03 536733                     /usr/lib64/httpd/modules/mod_auth_digest.so
7fca61907000-7fca61b07000 ---p 00006000 fc:03 536733                     /usr/lib64/httpd/modules/mod_auth_digest.so
7fca61b07000-7fca61b08000 r--p 00006000 fc:03 536733                     /usr/lib64/httpd/modules/mod_auth_digest.so
7fca61b08000-7fca61b09000 rw-p 00007000 fc:03 536733                     /usr/lib64/httpd/modules/mod_auth_digest.so
7fca61b09000-7fca61b0b000 r-xp 00000000 fc:03 536732                     /usr/lib64/httpd/modules/mod_auth_basic.so
7fca61b0b000-7fca61d0a000 ---p 00002000 fc:03 536732                     /usr/lib64/httpd/modules/mod_auth_basic.so
7fca61d0a000-7fca61d0b000 r--p 00001000 fc:03 536732                     /usr/lib64/httpd/modules/mod_auth_basic.so
7fca61d0b000-7fca61d0c000 rw-p 00002000 fc:03 536732                     /usr/lib64/httpd/modules/mod_auth_basic.so
7fca61d0c000-7fca61d7f000 r-xp 00000000 fc:03 390241                     /lib64/libfreebl3.so
7fca61d7f000-7fca61f7e000 ---p 00073000 fc:03 390241                     /lib64/libfreebl3.so
7fca61f7e000-7fca61f80000 r--p 00072000 fc:03 390241                     /lib64/libfreebl3.so
7fca61f80000-7fca61f81000 rw-p 00074000 fc:03 390241                     /lib64/libfreebl3.so
7fca61f81000-7fca61f85000 rw-p 00000000 00:00 0 
7fca61f85000-7fca61f89000 r-xp 00000000 fc:03 390294                     /lib64/libuuid.so.1.3.0
7fca61f89000-7fca62188000 ---p 00004000 fc:03 390294                     /lib64/libuuid.so.1.3.0
7fca62188000-7fca62189000 rw-p 00003000 fc:03 390294                     /lib64/libuuid.so.1.3.0
7fca62189000-7fca6218b000 r-xp 00000000 fc:03 390191                     /lib64/libdl-2.12.so
7fca6218b000-7fca6238b000 ---p 00002000 fc:03 390191                     /lib64/libdl-2.12.so
7fca6238b000-7fca6238c000 r--p 00002000 fc:03 390191                     /lib64/libdl-2.12.so
7fca6238c000-7fca6238d000 rw-p 00003000 fc:03 390191                     /lib64/libdl-2.12.so
7fca6238d000-7fca62518000 r-xp 00000000 fc:03 390183                     /lib64/libc-2.12.so
7fca62518000-7fca62717000 ---p 0018b000 fc:03 390183                     /lib64/libc-2.12.so
7fca62717000-7fca6271b000 r--p 0018a000 fc:03 390183                     /lib64/libc-2.12.so
7fca6271b000-7fca6271c000 rw-p 0018e000 fc:03 390183                     /lib64/libc-2.12.so
7fca6271c000-7fca62721000 rw-p 00000000 00:00 0 
7fca62721000-7fca62738000 r-xp 00000000 fc:03 390185                     /lib64/libpthread-2.12.so
7fca62738000-7fca62938000 ---p 00017000 fc:03 390185                     /lib64/libpthread-2.12.so
7fca62938000-7fca62939000 r--p 00017000 fc:03 390185                     /lib64/libpthread-2.12.so
7fca62939000-7fca6293a000 rw-p 00018000 fc:03 390185                     /lib64/libpthread-2.12.so
7fca6293a000-7fca6293e000 rw-p 00000000 00:00 0 
7fca6293e000-7fca62969000 r-xp 00000000 fc:03 536675                     /usr/lib64/libapr-1.so.0.3.9
7fca62969000-7fca62b68000 ---p 0002b000 fc:03 536675                     /usr/lib64/libapr-1.so.0.3.9
7fca62b68000-7fca62b6a000 rw-p 0002a000 fc:03 536675                     /usr/lib64/libapr-1.so.0.3.9
7fca62b6a000-7fca62cd9000 r-xp 00000000 fc:03 390189                     /lib64/libdb-4.7.so
7fca62cd9000-7fca62ed8000 ---p 0016f000 fc:03 390189                     /lib64/libdb-4.7.so
7fca62ed8000-7fca62ede000 rw-p 0016e000 fc:03 390189                     /lib64/libdb-4.7.so
7fca62ede000-7fca62f04000 r-xp 00000000 fc:03 390306                     /lib64/libexpat.so.1.5.2
7fca62f04000-7fca63103000 ---p 00026000 fc:03 390306                     /lib64/libexpat.so.1.5.2
7fca63103000-7fca63106000 rw-p 00025000 fc:03 390306                     /lib64/libexpat.so.1.5.2
7fca63106000-7fca6310d000 r-xp 00000000 fc:03 390242                     /lib64/libcrypt-2.12.so
7fca6310d000-7fca6330d000 ---p 00007000 fc:03 390242                     /lib64/libcrypt-2.12.so
7fca6330d000-7fca6330e000 r--p 00007000 fc:03 390242                     /lib64/libcrypt-2.12.so
7fca6330e000-7fca6330f000 rw-p 00008000 fc:03 390242                     /lib64/libcrypt-2.12.so
7fca6330f000-7fca6333d000 rw-p 00000000 00:00 0 
7fca6333d000-7fca63360000 r-xp 00000000 fc:03 536682                     /usr/lib64/libaprutil-1.so.0.3.9
7fca63360000-7fca6355f000 ---p 00023000 fc:03 536682                     /usr/lib64/libaprutil-1.so.0.3.9
7fca6355f000-7fca63561000 rw-p 00022000 fc:03 536682                     /usr/lib64/libaprutil-1.so.0.3.9
7fca63561000-7fca6357e000 r-xp 00000000 fc:03 390203                     /lib64/libselinux.so.1
7fca6357e000-7fca6377d000 ---p 0001d000 fc:03 390203                     /lib64/libselinux.so.1
7fca6377d000-7fca6377e000 r--p 0001c000 fc:03 390203                     /lib64/libselinux.so.1
7fca6377e000-7fca6377f000 rw-p 0001d000 fc:03 390203                     /lib64/libselinux.so.1
7fca6377f000-7fca63780000 rw-p 00000000 00:00 0 
7fca63780000-7fca637ac000 r-xp 00000000 fc:03 390248                     /lib64/libpcre.so.0.0.1
7fca637ac000-7fca639ab000 ---p 0002c000 fc:03 390248                     /lib64/libpcre.so.0.0.1
7fca639ab000-7fca639ac000 rw-p 0002b000 fc:03 390248                     /lib64/libpcre.so.0.0.1
7fca639ac000-7fca63a2f000 r-xp 00000000 fc:03 390250                     /lib64/libm-2.12.so
7fca63a2f000-7fca63c2e000 ---p 00083000 fc:03 390250                     /lib64/libm-2.12.so
7fca63c2e000-7fca63c2f000 r--p 00082000 fc:03 390250                     /lib64/libm-2.12.so
7fca63c2f000-7fca63c30000 rw-p 00083000 fc:03 390250                     /lib64/libm-2.12.so
7fca63c30000-7fca63c50000 r-xp 00000000 fc:03 390181                     /lib64/ld-2.12.so
7fca63e3f000-7fca63e40000 rw-p 00000000 00:00 0 
7fca63e40000-7fca63e47000 rw-p 00000000 00:00 0 
7fca63e47000-7fca63e4e000 r--s 00000000 fc:03 9662                       /usr/lib64/gconv/gconv-modules.cache
7fca63e4e000-7fca63e4f000 rw-p 00000000 00:00 0 
7fca63e4f000-7fca63e50000 r--p 0001f000 fc:03 390181                     /lib64/ld-2.12.so
7fca63e50000-7fca63e51000 rw-p 00020000 fc:03 390181                     /lib64/ld-2.12.so
7fca63e51000-7fca63e52000 rw-p 00000000 00:00 0 
7fca63e52000-7fca63ea6000 r-xp 00000000 fc:03 536795                     /usr/sbin/httpd
7fca640a5000-7fca640a9000 rw-p 00053000 fc:03 536795                     /usr/sbin/httpd
7fca640a9000-7fca640ac000 rw-p 00000000 00:00 0 
7fca64a42000-7fca64bf3000 rw-p 00000000 00:00 0 
7fca64bf3000-7fca64d69000 rw-p 00000000 00:00 0 
7fffb0d50000-7fffb0d65000 rw-p 00000000 00:00 0                          [stack]
7fffb0dff000-7fffb0e00000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
[Wed Aug 20 11:12:23 2014] [notice] child pid 19289 exit signal Aborted (6), possible coredump in /tmp/apache-cores

Comment 15 Simo Sorce 2014-08-20 18:34:14 UTC

Any chance to get access to debuginfo repos on that machine, most of the debuginfo package are missing. Especcially xmlsec1, glib2 and lasso.

Comment 16 Brian J. Atkisson 2014-08-20 18:51:20 UTC

(In reply to Simo Sorce from comment #15)
> Any chance to get access to debuginfo repos on that machine, most of the
> debuginfo package are missing. Especcially xmlsec1, glib2 and lasso.

My satellite-sync finally finished, I've subscribed that system to both rhel-x86_64-server-6-debuginfo and rhel-x86_64-server-6-beta-debuginfo.  These packages have been installed now:

 glib2-debuginfo               x86_64             2.26.1-7.el6_5                rhel-x86_64-server-6-debuginfo                  3.3 M
 lasso-debuginfo               x86_64             2.4.0-4.el6                   rhel-x86_64-server-6-beta-debuginfo             865 k
 xmlsec1-debuginfo             x86_64             1.2.20-4.el6                  rhel-x86_64-server-6-beta-debuginfo             919 k

Comment 17 Simo Sorce 2014-08-20 19:00:36 UTC

Thanks, unfortunately try as I migfht, I am not seeing the crash, I will try to analyze the cores you have there, but being a doublefree I was hoping I could catch it live under valgrind ... so far no luck whether I use valgrind or not.

Comment 18 Simo Sorce 2014-08-20 19:12:48 UTC

I got most of the debuginfo files but it seem glib2 debuginfo is stuck to 2.26 where we have glib2 2.28 installed :-/

Comment 19 Brian J. Atkisson 2014-08-20 19:19:00 UTC

(In reply to Simo Sorce from comment #18)
> I got most of the debuginfo files but it seem glib2 debuginfo is stuck to
> 2.26 where we have glib2 2.28 installed :-/

It doesn't look like rhel-x86_64-server-6-beta-debuginfo channel has the glib2-2.28.8-1.el6.x86_64 debuginfo package on RHN.  I'll look for it in brew and elsewhere.

Comment 20 Brian J. Atkisson 2014-08-20 19:24:27 UTC

(In reply to Simo Sorce from comment #18)
> I got most of the debuginfo files but it seem glib2 debuginfo is stuck to
> 2.26 where we have glib2 2.28 installed :-/

I found the package on engarchive2 and installed it on mellon01:

[root@mellon01 ~]# rpm -qa | grep glib2
glib2-2.28.8-1.el6.x86_64
glib2-debuginfo-2.28.8-1.el6.x86_64

I'll drop rel-eng a note that it's missing on RHN.

Comment 22 Simo Sorce 2014-08-20 20:27:04 UTC

I think I found the gneral are where this happens, but it is a bit hard to know specifically where as it is *many* layer deep into xml nodes abstraction code ...

Comment 23 Simo Sorce 2014-08-21 13:51:47 UTC

Ok, I am 99.9% positive I found the cause.
The machine had the package lass0-2.4.0-4 instead of the latest lasso-2.4.0-5 which had a ton of coverity fixes.

I now installed -5 and I can't repro anymore.

If you can confirm I will close this bug as INVALID.

Comment 24 Brian J. Atkisson 2014-08-21 14:01:22 UTC

(In reply to Simo Sorce from comment #23)
> Ok, I am 99.9% positive I found the cause.
> The machine had the package lass0-2.4.0-4 instead of the latest
> lasso-2.4.0-5 which had a ton of coverity fixes.
> 
> I now installed -5 and I can't repro anymore.
> 
> If you can confirm I will close this bug as INVALID.

Awesome, I'll attempt to reproduce today.  I take it lasso-2.4.0-5
will be in 6.6 GA?

Thank you for looking into this!

Comment 25 Simo Sorce 2014-08-21 14:08:07 UTC

Yes lasso-2.4.0-5 has been sitting in the errata for quite a while already

Comment 26 Brian J. Atkisson 2014-08-21 14:37:24 UTC

Looks like it's stable, I have been been able to produce the crash at all.  +1 to closing this bug.

Comment 27 Simo Sorce 2014-08-21 14:50:38 UTC

Ok.

lasso fixes take care of this problem, closeing as NOTABUG