Bug 1026956

Summary:	1.2.11.29 crash when removing entries from cache
Product:	Red Hat Enterprise Linux 6	Reporter:	Nathan Kinder <nkinder>
Component:	389-ds-base	Assignee:	Rich Megginson <rmeggins>
Status:	CLOSED ERRATA	QA Contact:	Sankar Ramalingam <sramling>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	6.5	CC:	jgalipea, nhosoi, vashirov
Target Milestone:	rc
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	389-ds-base-1.2.11.15-34.el6	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:	1026931	Environment:
Last Closed:	2014-10-14 07:52:01 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Bug Depends On:	1026931
Bug Blocks:	1061410

Description Nathan Kinder 2013-11-05 17:46:32 UTC

+++ This bug was initially created as a clone of Bug #1026931 +++

This bug is created as a clone of upstream ticket:
https://fedorahosted.org/389/ticket/47577

Steps:
1) setup up 2 way MMR
2) make sure the entry cache is full, so that search requests must evict old entries from the cache to make room for new entries
3) generate an ADD load against one of the masters
4) repeatedly search for (objectclass=nsTombstone) from the base suffix

--- Additional comment from Rich Megginson on 2013-11-05 12:44:26 EST ---

Steps to reproduce:

1) setup one instance of directory server with suffix dc=example,dc=com
2) Use ldclt like this to add a lot of entries:
    ldclt -h $HOST -p $PORT -D "$LDCLTDN" -w "$LDCLTPW" \
        -e add,person,incr,commoncounter,noloop \
        -r1 -R1000000 \
        -n8 \
        -f CN=XXXXXXX -b DC=EXAMPLE,DC=com \
        -q &
3) sleep 120 to allow some entries to search/mod/del
4) Use ldclt like this to do a lot of modifies:
    ldclt -h $HOST -p $PORT -D "$LDCLTDN" -w "$LDCLTPW" \
        -e incr,commoncounter,attreplace=sn:valueXXXXXXXXXX \
        -r1 -R1000000 \
        -n8 \
        -f CN=XXXXXXX -b DC=EXAMPLE,DC=com \
        -q &
5) sleep 30 to allow mods to progress
6) Use ldclt like this to do a lot of deletes:
    ldclt -h $HOST -p $PORT -D "$LDCLTDN" -w "$LDCLTPW" \
        -e delete,incr,commoncounter,noloop \
        -r1 -R1000000 \
        -n8 \
        -f CN=XXXXXXX -b DC=EXAMPLE,DC=com \
        -q &
7) use ldclt like this to do a lot of searches:
    ldclt -h $HOST -p $PORT -D "$LDCLTDN" -w "$LDCLTPW" \
        -e esearch,incr,commoncounter \
        -r1 -R1000000 \
        -n8 \
        -f CN=XXXXXXX -b DC=EXAMPLE,DC=com \
        -q &

You should see a crash within 10 minutes or so

Comment 2 Viktor Ashirov 2014-07-23 13:00:41 UTC

Followed steps mentioned in https://bugzilla.redhat.com/show_bug.cgi?id=1026956#c0

ldclt version 4.23
ldclt[24626]: Starting at Wed Jul 23 08:06:24 2014

ldclt[24626]: Average rate:   34.88/thr  (  27.90/sec), total:    279
ldclt[24626]: Average rate:   41.88/thr  (  33.50/sec), total:    335
ldclt[24626]: Average rate:   44.88/thr  (  35.90/sec), total:    359
ldclt[24626]: Average rate:   35.00/thr  (  28.00/sec), total:    280
ldclt[24626]: Average rate:   41.25/thr  (  33.00/sec), total:    330
ldclt[24626]: Average rate:   43.25/thr  (  34.60/sec), total:    346
ldclt[24626]: Average rate:   34.50/thr  (  27.60/sec), total:    276
ldclt[24626]: Average rate:   35.38/thr  (  28.30/sec), total:    283
ldclt[24626]: Average rate:   39.00/thr  (  31.20/sec), total:    312
ldclt[24626]: Average rate:   40.38/thr  (  32.30/sec), total:    323
ldclt[24626]: Average rate:   44.00/thr  (  35.20/sec), total:    352
ldclt[24626]: Average rate:   39.00/thr  (  31.20/sec), total:    312
ldclt version 4.23
ldclt[24648]: Starting at Wed Jul 23 08:08:24 2014

ldclt[24626]: Average rate:   15.50/thr  (  12.40/sec), total:    124
ldclt[24648]: Average rate:   15.00/thr  (  12.00/sec), total:    120
ldclt[24626]: Average rate:   20.88/thr  (  16.70/sec), total:    167
ldclt[24648]: Average rate:   21.12/thr  (  16.90/sec), total:    169
ldclt version 4.23
ldclt version 4.23
ldclt[24662]: Starting at Wed Jul 23 08:08:54 2014
ldclt[24661]: Starting at Wed Jul 23 08:08:54 2014


ldclt[24626]: Average rate:   21.38/thr  (  17.10/sec), total:    171
ldclt[24648]: Average rate:   21.00/thr  (  16.80/sec), total:    168
ldclt[24661]: Average rate: 30432.50/thr  (24346.00/sec), total: 243460
ldclt[24662]: Average rate:    7.25/thr  (   5.80/sec), total:     58
ldclt[24626]: Average rate:    6.62/thr  (   5.30/sec), total:     53
ldclt[24648]: Average rate:    7.75/thr  (   6.20/sec), total:     62
ldclt[24661]: Average rate: 27393.88/thr  (21915.10/sec), total: 219151

<snip>

After several hours running ldclt DS didn't crash. Hence marking as verified.

Comment 3 errata-xmlrpc 2014-10-14 07:52:01 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-1385.html