Bug 220127

While reindexing, it holds a write lock on vlv:
[vlv.c]
   1249 int
   1250 ldbm_back_ldbm2index(Slapi_PBlock *pb)
   1251 {
    [...]
   1493     /* Bug 603120: slapd dumps core while indexing and deleting the db a
       t the
   1494      * same time. Now added the lock for the indexing code too.
   1495      */
   1496     vlv_acquire_lock(be);
   1497     while (1) {

Actually, the lock does not need to be a write lock because when reindexing, the
backend is put into the read mode.  So, no other threads have a chance to update
the backend.  I propose to demote the lock in vlv_acquire_lock to the read lock:
[vlv.c]
   2000 void
   2001 vlv_acquire_lock(backend *be)
   2002 {
   2003     LDAPDebug(LDAP_DEBUG_TRACE, "vlv_acquire_lock => trying to acquire t
       he lock\n", 0, 0, 0);    2004     PR_RWLock_Wlock(be->vlvSearchList_lock);
   2005 }

And it turned out this bug is a duplicate of bug [171081] ldapsearch hung at
browsing index creation.

With the change, search works fine and update is blocked while the backend is
being reindexed:
Index: vlv.c
===================================================================
RCS file: /cvs/dirsec/ldapserver/ldap/servers/slapd/back-ldbm/vlv.c,v
retrieving revision 1.12
diff -t -w -U4 -r1.12 vlv.c
--- vlv.c       7 Dec 2006 21:15:00 -0000       1.12
+++ vlv.c       19 Dec 2006 01:41:22 -0000
@@ -2000,9 +2000,9 @@
 void
 vlv_acquire_lock(backend *be)
 {
         LDAPDebug(LDAP_DEBUG_TRACE, "vlv_acquire_lock => trying to acquire the
lock\n", 0, 0, 0);
-        PR_RWLock_Wlock(be->vlvSearchList_lock);
+        PR_RWLock_Rlock(be->vlvSearchList_lock);
 }



$ ./ldapsearch -p <port> -D "cn=Directory Manager" -w pw -b
"dc=sfbay,dc=redhat,dc=com" "(cn=*)" dn
dn: uid=EDommety9996, ou=Product Development, dc=sfbay,dc=redhat,dc=com
dn: uid=JSauck9997, ou=Product Testing, dc=sfbay,dc=redhat,dc=com
[...]

$ ./ldapmodify  -p <port> -D "cn=Directory Manager" -w pw
dn: uid=JSauck9997, ou=Product Testing, dc=sfbay,dc=redhat,dc=com
changetype: modify
replace: mail
mail: js

modifying entry uid=JSauck9997, ou=Product Testing, dc=sfbay,dc=redhat,dc=com
ldap_modify: DSA is unwilling to perform
ldap_modify: additional info: database is read-only

Is it really a duplicate?  Is it possible that we need to acquire a write lock
under some other circumstance, but not reindex?  If so, should we introduce some
flag that lets the code acquire a write lock if not reindexing?  What happens if
you reindex an attribute that is involved in a vlv operation?

If the change is safe under the above circumstances, then I approve.  The next
step would be to find out from the ops team what is their desired resolution to
this issue.  I'm hoping that it is acceptable to shutdown for several minutes to
reindex (actually, offline indexing is much faster than online, so downtime
should be even less) because we don't want to have to release another service pack.

We would take a short shutdown to re-index. But it would need ot be on the
ordder of 5-10 minutes to add the index. Also.. we need to ensure that if we
have replication, there is a strategy to index both nodes in the cluster.

(In reply to comment #4)
> We would take a short shutdown to re-index. But it would need ot be on the
> ordder of 5-10 minutes to add the index. Also.. we need to ensure that if we
> have replication, there is a strategy to index both nodes in the cluster.

10 minutes might be long enough, but you'll have to run some tests to find out
for sure.

The fix for the bug 171081 also solved this problem.

To verify using these steps, marking as modified, instead of duplicate.
Steps to reproduce the problem:
While running db2index, run ldapsearch
$ ./db2index.pl -D "cn=Directory Manager" -w pw -n userRoot -t givenname
$ ./ldapsearch -p <port> -D "cn=Directory Manager" -w pw -b
"dc=sfbay,dc=redhat,dc=com" "(cn=*)" dn