Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1117335

Summary:	CR2 can't properly start leveldb-JNI store (regression CR1 -> CR2) leveldbjni 1.17 to 1.13
Product:	[JBoss] JBoss Data Grid 6	Reporter:	Tomas Sykora <tsykora>
Component:	Infinispan	Assignee:	Dave Stahl <dstahl>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Martin Gencur <mgencur>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	6.3.0	CC:	dstahl, jdg-bugs, tkirby, vchepeli
Target Milestone:	CR3
Target Release:	6.3.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2015-01-26 14:04:21 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1104639

Description Tomas Sykora 2014-07-08 13:29:10 UTC

Suspicious commit: https://github.com/infinispan/jdg/commit/38536fd836aa829a6ce4695d7ef0c25d904b0d37

We are unable to properly start leveldb-JNI store for a server. 

JON UI was used during the process of finding this bug. This is the regression from 6.3.0 CR1 to 6.3.0 CR2. With CR1 server, we are able to successfully configure and START leveldb-JNI store for a particular cache.

With CR2, the same process, server logs:

06:55:32,190 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-1) MSC000001: Failed to start service jboss.infinispan.clustered.default: org.jboss.msc.service.StartException in service jboss.infinispan.clustered.default: Failed to start service
	at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1767) [jboss-msc-1.0.4.GA-redhat-1.jar:1.0.4.GA-redhat-1]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.7.0_09-icedtea]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.7.0_09-icedtea]
	at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_09-icedtea]
Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.persistence.manager.PersistenceManagerImpl.start() on object of type PersistenceManagerImpl
	at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:170)
	at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
	at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
	at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
	at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
	at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216)
	at org.infinispan.CacheImpl.start(CacheImpl.java:696)
	at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:575)
	at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:530)
	at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:410)
	at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:424)
	at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:89)
	at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:80)
	at org.infinispan.server.infinispan.SecurityActions$4.run(SecurityActions.java:103)
	at org.infinispan.server.infinispan.SecurityActions$4.run(SecurityActions.java:100)
	at org.infinispan.security.Security.doPrivileged(Security.java:76)
	at org.infinispan.server.infinispan.SecurityActions.doPrivileged(SecurityActions.java:47)
	at org.infinispan.server.infinispan.SecurityActions.startCache(SecurityActions.java:108)
	at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:79)
	at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) [jboss-msc-1.0.4.GA-redhat-1.jar:1.0.4.GA-redhat-1]
	at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) [jboss-msc-1.0.4.GA-redhat-1.jar:1.0.4.GA-redhat-1]
	... 3 more
Caused by: java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no leveldbjni32-1.13-redhat.001 in java.library.path, no leveldbjni-1.13-redhat.001 in java.library.path, no leveldbjni in java.library.path, /tmp/libleveldbjni-32-1-6897481072767310431.13-redhat: libleveldb.so.1: cannot open shared object file: No such file or directory]
	at org.fusesource.hawtjni.runtime.Library.doLoad(Library.java:182)
	at org.fusesource.hawtjni.runtime.Library.load(Library.java:140)
	at org.fusesource.leveldbjni.JniDBFactory.<clinit>(JniDBFactory.java:48)
	at java.lang.Class.forName0(Native Method) [rt.jar:1.7.0_09-icedtea]
	at java.lang.Class.forName(Class.java:264) [rt.jar:1.7.0_09-icedtea]
	at org.infinispan.commons.util.Util.loadClassStrict(Util.java:139)
	at org.infinispan.commons.util.Util.loadClass(Util.java:83)
	at org.infinispan.commons.util.Util.getInstance(Util.java:223)
	at org.infinispan.persistence.leveldb.LevelDBStore.newDbFactory(LevelDBStore.java:76)
	at org.infinispan.persistence.leveldb.LevelDBStore.init(LevelDBStore.java:59)
	at org.infinispan.persistence.manager.PersistenceManagerImpl.createLoadersAndWriters(PersistenceManagerImpl.java:579)
	at org.infinispan.persistence.manager.PersistenceManagerImpl.start(PersistenceManagerImpl.java:113)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_09-icedtea]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_09-icedtea]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_09-icedtea]
	at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0_09-icedtea]
	at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
	... 23 more

Comment 2 Tomas Sykora 2014-07-08 13:34:30 UTC

6.3.0 target set, it looks quite important.

Comment 3 Dave Stahl 2014-07-08 13:36:59 UTC

leveldb 1.13 build and integrated into jdg-6.3.0

Comment 4 Dave Stahl 2014-07-08 13:39:14 UTC

(In reply to Dave Stahl from comment #3)
> leveldb 1.13 build and integrated into jdg-6.3.0

Updated wrong bug.  Setting back to new

Comment 5 Dave Stahl 2014-07-08 19:24:02 UTC

This looks like libleveldbjni.so was linked using a dynamic library instead of static.  Will rebuild and hand off for verification.

Comment 6 Tomas Sykora 2014-07-09 05:54:33 UTC

Awesome Dave! 

I verified your fresh build is working.

1) I used: leveldbjni-all-1.13-redhat.002.jar instead of leveldbjni-all-1.13-redhat.001.jar
2) I changed module.xml descriptor accordingly 
3) I configured levelDB JNI store for a cache
4) I restarted a server and saw this very nice and beautiful result in the console:

01:48:39,128 INFO  [org.infinispan.persistence.leveldb.LevelDBStore] (MSC service thread 1-1) ISPN023005: Using JNI LevelDB implementation: org.fusesource.leveldbjni.JniDBFactory

No ERRORS, I consider levelDB store started properly. 

I expect this issue to be ON_QA for JDG 6.3.0 CR3 build.

Comment 7 Tomas Sykora 2014-07-10 11:24:12 UTC

Fantastic. 

It is working smoothly with CR3 now :)

VERIFIED

Comment 8 Ion Savin 2014-07-11 17:36:15 UTC

*** Bug 1118785 has been marked as a duplicate of this bug. ***