Bug 107303 - lucene on shared filesystems
lucene on shared filesystems
Status: NEW
Product: Red Hat Web Application Framework
Classification: Retired
Component: other (Show other bugs)
nightly
All Linux
medium Severity medium
: ---
: ---
Assigned To: ccm-bugs-list
Jon Orris
:
Depends On:
Blocks: 108447
  Show dependency treegraph
 
Reported: 2003-10-16 13:57 EDT by Richard Li
Modified: 2008-05-01 11:39 EDT (History)
0 users

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Richard Li 2003-10-16 13:57:19 EDT
At the CCM level, update locking is done by synchronizing on a static
variable in LuceneLock.java. This doesn't work across JVMs.

The standard lucene locking system involves basically 'touching' a
file in the lucene index directory. The problem is that with NFS,
the there can be a significant delay between creating a file on
one server & it appearing on the other. This is an obvious race
condition, which means standard lucene locking is only viable when
you are controllling a bunch of writers on a single machine with
a local filesystem.
Comment 1 Carsten Clasohm 2004-02-13 04:56:42 EST
Taken from a mail I just wrote, here is an idea for fixing this, which
should work both on Oracle and PostgreSQL:

>>

Because all machines access the same Oracle database, you can create a
lock in the database, write wrappers around the search and indexing
code, and use the database lock to ensure that only one server at a
time writes to the index, and that no read operations occur during
writing.

Locking can be done as follows, assuming you created a table
lucene_lock for this purpose. Don't use an existing table, as the
locking will block queries on that table.

Before performing a read operation, obtain a shared lock:

  (start database transaction)
  lock table lucene_lock in share mode;
  (perform the Lucene read operation)
  (end database transaction)

For writing, we need an exclusive lock:

  (start database transaction)
  lock table lucene_lock in exclusive mode;
  (update the Lucene index)
  (end database transaction)

It is important to terminate the transaction once the read or write
operation has finished, to release the lock as fast as possible.

<<
Comment 2 Carsten Clasohm 2004-02-24 07:52:36 EST
I refined this idea. Here is how it could be implemented with Lucene 1.3:

>>

The class definitions in Lucene 1.3 have been changed in such a way
that one can derive a class from FSDirectory with custom locking
code. This has the advantage that IndexReader and IndexWriter have
full control over when locks are created and released. In contrast to
the solution for 1.2, the solution for 1.3 allows parallel access by
index readers and one writer. Therefore, updating the index will not
block search queries.

First, create a new database table:

create table lucene_locks (
    name    varchar2(4000)
            constraint lucene_locks_pk primary key
);

Define the data operation "setLock" in one of your PDL files like
this:

  insert into lucene_locks (name)
  values (:lockName || ' ' || :indexPath);

Unfortunately, FSDirectory declares makeLock() as final, so we cannot
inherit from FSDirectory and overwrite makeLock(). So, copy
FSDirectory.java, change the class name and package, and modify
makeLock as follows.

  class DBLockDirectory extends Directory {
    ...

    public final Lock makeLock(String name) {
      Session lockSession = new Session();
      final TransactionContext txn = lockSession.getTransactionContext();

      final DataOperation setLock = 
	lockSession.retrieveDataOperation("setLock");
      setLock.setParameter("indexPath", directory.getCanonicalPath());
      setLock.setParameter("lockName", name);

      return new Lock() {
	public boolean obtain() throws IOException {
	  if (!DISABLE_LOCKS) {
	    if (txn.inTxn())
	      return false;
	    txn.beginTxn();
	    setLock.execute();
	  }
	  return true;
	}
	public void release() {
	  if (!DISABLE_LOCKS) {
	    if (!txn.inTxn())
	      return false;
	    txn.abortTxn();
          }
	  return true;
	}
	public boolean isLocked() {
	  // not supported
	  return false;
	}
      };
    }

    ...
  };

In your code, you have to change all IndexReader, IndexWriter and
IndexSearcher constructor calls to use the constructors which accept a
Directory to identify the Lucene index, and then
pass a DBLockDirectory object to it, which can be obtained with
DBLockDirectory.getDirectory().

<<
Comment 3 Carsten Clasohm 2004-02-26 11:01:07 EST
The code won't work because Session uses the thread connection, on
which we cannot begin and abort a transaction while doing other work.
Instead, we have to use a Connection directly:

  class DBLockDirectory extends Directory {
    ...

    public Lock makeLock(final String name) {

      return new Lock() {
	private boolean locked = false;
	private Connection lockConnection;

	public boolean obtain() throws IOException {
	  if (!DISABLE_LOCKS) {
	    if (locked)
	      return false;

	    try {
	      Connection lockConnection = ConnectionManager.getConnection();
	      lockConnection.setAutoCommit(false);

	      PreparedStatement setLock = 
		lockConnection.prepareStatement("insert into lucene_locks (name)
values (? || ' ' || ?)");
	      setLock.setString(1, directory.getCanonicalPath());
	      setLock.setString(2, name);
	      setLock.execute();
	    } catch (SQLException e) {
	      throw new RuntimeException("Error will obtaining lock " + name
+ ": " + e);
	    }

	    locked = true;
	  }
	  return true;
	}
	public void release() {
	  if (!DISABLE_LOCKS) {
	    if (!locked)
	      return false;

	    try {
	      lockConnection.rollback();
	      lockConnection.close();
	      lockConnection = null;
	    } catch (SQLException e) {
	      throw new RuntimeException("Error will obtaining lock " + name
+ ": " + e);
	    }

	    locked = false;
          }
	  return true;
	}
	public boolean isLocked() {
	  return locked;
	}
      };
    }

    ...
  };
Comment 4 Carsten Clasohm 2004-03-09 05:56:26 EST
See changelist 41152 for an implementation of DBLockDirectory with
Lucene 1.3.
Comment 5 Carsten Clasohm 2004-03-18 04:52:39 EST
Plus changelist 41474, which modifies the Initializer and LuceneSearch
classes to use DBLockDirectory.

Note You need to log in before you can comment on or make changes to this bug.