Red Hat Bugzilla – Bug 811291
[abrt] 389-ds-base-1.2.10.4-2.fc16: index_range_read_ext: Process /usr/sbin/ns-slapd was killed by signal 11 (SIGSEGV)
Last modified: 2012-06-20 03:15:05 EDT
This bug is created as a clone of upstream ticket: https://fedorahosted.org/389/ticket/336 https://bugzilla.redhat.com/show_bug.cgi?id=808770 (''Fedora'') {{{ libreport version: 2.0.8 abrt_version: 2.0.7 backtrace_rating: 4 cmdline: /usr/sbin/ns-slapd -D /etc/dirsrv/slapd-ELADIAN-COM -i /var/run/dirsrv/slapd-ELADIAN-COM.pid -w /var/run/dirsrv/slapd-ELADIAN-COM.startpid comment: crash_function: index_range_read_ext executable: /usr/sbin/ns-slapd kernel: 3.1.6-1.fc16.x86_64 pid: 19109 pwd: /var/log/dirsrv/slapd-ELADIAN-COM reason: Process /usr/sbin/ns-slapd was killed by signal 11 (SIGSEGV) time: Sat 31 Mar 2012 11:08:28 AM EDT uid: 996 username: dirsrv backtrace: Text file, 165873 bytes dso_list: Text file, 8455 bytes maps: Text file, 45972 bytes build_ids: :28579c89bda7f08dace218884d7a5561f3f5f89d :ccd23fb93a7924ddd27ea9e7f054fa5309c92f32 :ee2f04900ae1f07517d91eba300ef385fccab1b8 :89017ab0b75e533e4294f74ee8f7db3daceea536 :bb5c9b5cca6d04252b061a2f840ab4ddfda9e733 :3ba5302e34546a45e9a85ac3429e7bc6ed624011 :638db099ff5fb986d1a092629d1bb1dca5be4904 :6a8868ece50dc746e4771ff0b5f6f314a45f4a77 :056358b51146e7dc68f8f712cf2b208bb4bd3654 :0f0c9eed63ed279ceb0aef109eff8bcbb351e4bb :0bd515c1c778bb4b8f870a900d1543da39d6b662 :5083c24b4fd7dbf3cac9cf4c15876b0dfdceb4bb :2639b5594fe4d7897db63ecd1ebb9681438317a8 :8d4c1cc04914e6f8dce0b485275a05d510543280 :48e9e9baf9f0cbd3653cb77ed45b83a7ee8aef2e :2f71e81631ba518b573b7678465bcce24ae8530f :1b2a3adcb5fa5b0594f5c9182e6a12e83d47b4b8 :4d7facfa8aead1e0060760f76c77f3ae0ed5becc :ecd5ba5435e91ca72281c7c74cf39bf92e01b3a5 :5ac56dbb5866c475e1b0c3e4ceeb390b0f7508c0 :34e8b3cee73a6d0b8ac17c69ba4b1170724db64c :aac33d0080dfd5b813520f4e8a5c36378e720b60 :a58aa84cd66c6863b87b0879885229c390c55162 :59292332bd88fce31f7557e60ff08d44ce254ec8 :cce23724534ba89f256874d714ee35672bc9f28b :b01d2bc881174324d6dc5cbf0a3605239c70c72e :ef95ed2b74ff19753827117d4e556f38b33e74b5 :9e12672793058b2f142b642851576ad036a2c4e6 :33324b7717adf03b8578f0b91fe2d53d765d3120 :0487340b0c803f5800eadd5238d3d9b2435ceeec :c9f8613367b2e2e1d577edf20afa7ff17b3397ad :0a49b6fd90960659888f0be3bb576495f9d41959 :ac2b4cc5fed8f857807a8830b4773f85be9642b2 :9f9aa00c80e740cbcf6628a52548bc352b90c898 :01175202f32ef3226de9abfe9384866107ba242c :a0b2903547454e47fec356c1fa1a69a7593893c5 :ea44661a40777d6be2f79058c161d3bb4962891a :a9791ba4315ed37156e8316c3e1380d8af2e33dd :4b4285058f7f6b39b7e7c45df83fd36c6833bfe4 :e49b84b81d7329cc916ec84b2ec5bbf2f8852ff9 :6f130628ef6eb5e6c4203baef32d8382af19737a :516780dcc6528878fd0301dbb03b30252111de56 :723cc9153ba157f7be2a431e089b875d9168e763 :8822ec50a7ca08984b34af0b3096794a5085c473 :1a212c7f1515542b310ba92f6109efc9b5bf2b6e :7f1fbc967e3fba1e7008912447f54b7260204d1c :919cd69551c389bdb0131a27a4d7ffc7c1ea6d2b :57088c4741c6f62eb5d39889da7533bf7cc157f0 :64c606de82a17e176ea5c69c9571f4bcfd6c080d :065f24e263b3a0d4316bfeedf855df8c2c35c03a :c49811439aeff6168ae82e7cf808a8c03b1c9e48 :0ce9819eb7014de7e72eec40979dd5e4d6566dca :bccc101a872bfd4670545e87b5d4004a2f890052 :c314e9cb57367719196b6f665fe6773a2d08add6 :54aeedc57e5f902a83f20308416ac2213627e5ca :5ba371439acb075c48406bd7c66c80594d91c7e7 :e2e67b3ae2579e1667b73ad385f61552931024e9 :b4272a5a90d21f312b476d02b575b7222ff17e91 :6502dd4813f98137c70e6e05cc43828f5c2263c1 :dacf32a9a2f2b5077ba944e7b835cca6f637f78f :fee9f980714278bbbbf7fa217189472cd6446f6d :20efc30fde6abc23cfe693739cff23d67ee9d333 :75eaece6e0d1e075520cdfad5c398fdab4370846 :63365768c3b6cad3c5a303e46543c113a5326b80 :e814ef8432b7dec42a0b8ec12b9abd9fc7f57b40 :a28b15796672928bb44b262c88052ae601bf5955 :2982ba94b3d248b639aaa6218be955bb159bb2e3 :36fc045d11fc55f1b455247938e651666e2c127e :f1e8182609d9eae40a8ddbbb72c9346166014f1a :b010321c92dc1b5abf0007da423b778542817883 :efd7e116c58ad407b8936018a86a399993a11e79 :552354979eebf3d6c27346691a5b520ccf8f4e75 :aad91f4f4e3c8a5d8cf3be0deac2ac041b4e4e1c :e5429e0905bedecc534b057783916ba43e06a66e :788f31e354f4ad30dd74b958d821c980b37d04c2 :68c27733d3164d71c325fe621dd9bddf0f97ab47 :eac10d05c580f122193f7560fb72e743af88edb3 :a9a3ce4a014e11a12a90a0979d113504adada32d :b9d3a4213a482d034bebc7dc1ed2901a734a894d :789c958162e98ef211c3bebd162ab587eb9ece93 :b5918178168db4577a7744fef13d892cb5ac6d3f :83cb4627bed5c756d4e6f23badf8a6b4b042f4e6 :040f78c63e49df93936f710287bdc74ae83111c0 :e567a4b498aa9dfc484575e666a17925f0944318 :2d4ac24a95409c92b794b521e85670af3d416590 :5734da0b584b126fcdfc17210761d923ff95cb26 :4613bd742f4dca43dc32603622cca5692dd60c24 :12f7326885e79c7a690d8b6a22109af42760f6af :b3db147cfb8af552c8dd1b47c81dc66760eaea80 environ: :BOOT_IMAGE=/vmlinuz-3.1.6-1.fc16.x86_64 :PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin :SYSFONT=latarcyrheb-sun16 :LANG=en_US.UTF-8 :PWD=/ :KEYTABLE=us :'ulimit -n 8192' :KRB5_KTNAME=/etc/dirsrv/ds.keytab :'export KRB5_KTNAME=/etc/dirsrv/ds.keytab' :SERVER_DIR=/usr/lib64/dirsrv :SERVERBIN_DIR=/usr/sbin :CONFIG_DIR=/etc/dirsrv/slapd-ELADIAN-COM :INST_DIR=/var/lib/dirsrv/scripts-ELADIAN-COM :RUN_DIR=/var/run/dirsrv :PRODUCT_NAME=slapd var_log_messages: :Mar 25 03:25:11 ipa1-nyc1 ns-slapd: GSSAPI Error: An invalid name was supplied (Hostname cannot be canonicalized) :Mar 26 08:49:48 ipa1-nyc1 ns-slapd: GSSAPI Error: An invalid name was supplied (Hostname cannot be canonicalized) :Mar 26 14:29:19 ipa1-nyc1 ns-slapd: GSSAPI Error: An invalid name was supplied (Hostname cannot be canonicalized) :Mar 26 16:36:33 ipa1-nyc1 ns-slapd: GSSAPI Error: An invalid name was supplied (Hostname cannot be canonicalized) :Mar 27 15:30:23 ipa1-nyc1 kernel: [7150921.037468] ns-slapd[27549]: segfault at 0 ip 00007f136092ac07 sp 00007f13497ee650 error 4 in libback-ldbm.so[7f13608ec000+8d000] :Mar 27 16:07:36 ipa1-nyc1 ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Credentials cache file '/tmp/krb5cc_996' not found) :Mar 28 04:39:12 ipa1-nyc1 kernel: [7198275.020687] ns-slapd[15998]: segfault at 0 ip 00007f388d8eec07 sp 00007f38757ee650 error 4 in libback-ldbm.so[7f388d8b0000+8d000] :Mar 28 07:01:26 ipa1-nyc1 ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Credentials cache file '/tmp/krb5cc_996' not found) :Mar 28 09:44:27 ipa1-nyc1 kernel: [7216590.050214] ns-slapd[24987]: segfault at 0 ip 00007f7c62c08c07 sp 00007f7c4b7f2650 error 4 in libback-ldbm.so[7f7c62bca000+8d000] :Mar 28 10:00:59 ipa1-nyc1 ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Credentials cache file '/tmp/krb5cc_996' not found) :Mar 28 17:50:59 ipa1-nyc1 kernel: [7245793.824167] ns-slapd[27051]: segfault at 17d ip 00000031d74810a4 sp 00007fff17327358 error 4 in libc-2.14.90.so (deleted)[31d7400000+1ab000] :Mar 28 17:51:43 ipa1-nyc1 ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Credentials cache file '/tmp/krb5cc_996' not found) :Mar 28 17:53:33 ipa1-nyc1 ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Credentials cache file '/tmp/krb5cc_996' not found) :Mar 28 17:53:59 ipa1-nyc1 ns-slapd[14914]: [28/Mar/2012:17:53:59 -0400] - Information: Non-Secure Port Disabled :Mar 28 17:56:06 ipa1-nyc1 ns-slapd[15846]: [28/Mar/2012:17:56:06 -0400] - Information: Non-Secure Port Disabled :Mar 28 17:56:08 ipa1-nyc1 ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Credentials cache file '/tmp/krb5cc_996' not found) :Mar 28 18:09:51 ipa1-nyc1 ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Credentials cache file '/tmp/krb5cc_996' not found) :Mar 29 20:33:09 ipa1-nyc1 kernel: [7341921.236497] ns-slapd[20295]: segfault at 0 ip 00007f4e472cb0a7 sp 00007f4e267e8650 error 4 in libback-ldbm.so[7f4e4728c000+8d000] :Mar 29 21:41:19 ipa1-nyc1 ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Credentials cache file '/tmp/krb5cc_996' not found) :Mar 29 22:16:29 ipa1-nyc1 ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Credentials cache file '/tmp/krb5cc_996' not found) :Mar 31 11:08:28 ipa1-nyc1 kernel: [7480792.701830] ns-slapd[19141]: segfault at 0 ip 00007f644bb230a7 sp 00007f6427fe3650 error 4 in libback-ldbm.so[7f644bae4000+8d000] :Mar 31 11:08:39 ipa1-nyc1 abrt[8052]: Saved core dump of pid 19109 (/usr/sbin/ns-slapd) to /var/spool/abrt/ccpp-2012-03-31-11:08:28-19109 (477249536 bytes) }}}
Please add steps to reproduce/verify
Steps to reproduce/verify: 1) set up two masters for multi-master replication 2) enable the USN plugin on both servers 3) add indexes for uidNumber and gidNumber e.g. dn: cn=uidnumber,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config objectclass: top objectclass: nsIndex cn: uidnumber nssystemindex: false nsindextype: eq nsmatchingrule: integerOrderingMatch dn: cn=gidnumber,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config objectclass: top objectclass: nsIndex cn: gidnumber nssystemindex: false nsindextype: eq nsmatchingrule: integerOrderingMatch in a separate shell/window, do the following: usn=0 ; while [ 1 ] ; do ii=100 ; while [ $ii -gt 0 ] ; do ii=`expr $ii - 1` ; ldapsearch -xLLL -h localhost -p PORTNUMBEROFMASTER2 -D "cn=directory manager" -w password -b dc=example,dc=com -E pr=100/noprompt '(&(objectClass=posixAccount)(uid=*)(uidNumber=*)(gidNumber=*)(entryusn>='$usn')(!(entryusn='$usn')))' > /dev/null 2>&1 & done ; wait ; done That is, do 100 ldapsearch processes all doing a simple paged range search, and wait for them all to complete, then do it again in another shell/window, add 1000 posix users like this: dn: uid=testuser$ii,ou=people,dc=example,dc=com objectclass: inetOrgPerson objectclass: posixAccount cn: Test User$ii sn: User$ii uidNumber: 500$ii gidNumber: 500$ii homeDirectory: /home/testuser$ii confirm that all users are replicated to both servers next, delete all 1000 users - I do something like this: ii=1000 ; while [ $ii -gt 0 ] ; do dn="uid=testuser$ii,ou=people,dc=example,dc=com" ldapdelete -x -h localhost -p PORTNUMBEROFMASTER1 -D "cn=directory manager" -w password "$dn" ii=`expr $ii - 1` done Usually you will see the ldapsearches complete with a message of "Done ..." - if you see them complete with an message like "Exit: 255 ..." or "Error: ..." that usually means the server has crashed.
Technical note added. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. New Contents: Cause: Performing delete and ranged search operations against the directory server under a high load. Consequence: Directory server crashes. Fix: Entries may be deleted out from under a ranged search request. Server should handle this case by not returning deleted entries and not crashing. Result: Server does not crash when performing range searches and deletions while under a high load.
As given in comment #3, I ran simplepaged ldapsearch, ldapdelete in multiple threads. I didn't see any crash in the server. Hence, marking the bug as Verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHSA-2012-0813.html