Bug 1020411

Summary:	LevelDB tests are not running on Solaris, Windows and HP-UX machines
Product:	[JBoss] JBoss Data Grid 6	Reporter:	Anna Manukyan <amanukya>
Component:	Infinispan	Assignee:	Tristan Tarrant <ttarrant>
Status:	NEW ---	QA Contact:	Martin Gencur <mgencur>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	6.2.0, 6.3.0, 6.2.1, 6.4.0, 6.3.1, 6.5.0, 6.4.1, 6.6.1	CC:	afield, jdg-bugs, tsykora
Target Milestone:	---
Target Release:	6.4.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Bug Depends On:
Bug Blocks:	1178701, 1178702

Description Anna Manukyan 2013-10-17 15:41:10 UTC

When running the tests for LevelDB project (infinispan-cachestore-leveldb) on Windows (2008r2 and 2012) and Solaris (10, 11, Sparc) machines, the following error appears and the tests for JNI configuration are failing:

The error is given for the Windows machine:

org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.persistence.manager.PersistenceManagerImpl.start() on object of type PersistenceManagerImpl
	at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
	at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
	at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
	at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
	at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
	at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216)
	at org.infinispan.CacheImpl.start(CacheImpl.java:668)
	at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:539)
	at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:502)
	at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:395)
	at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:368)
	at org.infinispan.persistence.BaseStoreFunctionalTest.testPreloadAndExpiry(BaseStoreFunctionalTest.java:81)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:715)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:907)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1237)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
	at org.testng.TestRunner.privateRun(TestRunner.java:767)
	at org.testng.TestRunner.run(TestRunner.java:617)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
	at org.testng.SuiteRunner.access$000(SuiteRunner.java:37)
	at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368)
	at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no leveldbjni64-1.7 in java.library.path, no leveldbjni-1.7 in java.library.path, no leveldbjni in java.library.path, C:\Documents and Settings\hudson\Local Settings\Temp\leveldbjni-64-1-2713005400187874283.7: Can't find dependent libraries]
	at org.fusesource.hawtjni.runtime.Library.doLoad(Library.java:182)
	at org.fusesource.hawtjni.runtime.Library.load(Library.java:140)
	at org.fusesource.leveldbjni.JniDBFactory.<clinit>(JniDBFactory.java:48)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:270)
	at org.infinispan.commons.util.Util.loadClassStrict(Util.java:119)
	at org.infinispan.commons.util.Util.loadClass(Util.java:80)
	at org.infinispan.commons.util.Util.getInstance(Util.java:203)
	at org.infinispan.persistence.leveldb.LevelDBStore.newDbFactory(LevelDBStore.java:70)
	at org.infinispan.persistence.leveldb.LevelDBStore.init(LevelDBStore.java:53)
	at org.infinispan.persistence.manager.PersistenceManagerImpl.createLoadersAndWriters(PersistenceManagerImpl.java:508)
	at org.infinispan.persistence.manager.PersistenceManagerImpl.start(PersistenceManagerImpl.java:114)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:183)
	... 31 more

As a result neverending NoClassDefFoundError-s are thrown:

2013-10-17 05:53:14,914 ERROR [Util] (testng-ConfigurationTest) ISPN000905: Unable to load org.fusesource.leveldbjni.JniDBFactory from any of the following classloaders: [sun.misc.Launcher$AppClassLoader@1b186cf9, sun.misc.Launcher$AppClassLoader@1b186cf9, sun.misc.Launcher$AppClassLoader@1b186cf9]
java.lang.NoClassDefFoundError: Could not initialize class org.fusesource.leveldbjni.JniDBFactory
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:270)
	at org.infinispan.commons.util.Util.loadClassStrict(Util.java:119)
	at org.infinispan.commons.util.Util.loadClass(Util.java:80)
	at org.infinispan.commons.util.Util.getInstance(Util.java:203)
	at org.infinispan.persistence.leveldb.LevelDBStore.newDbFactory(LevelDBStore.java:78)
	at org.infinispan.persistence.leveldb.LevelDBStore.init(LevelDBStore.java:53)
	at org.infinispan.persistence.manager.PersistenceManagerImpl.createLoadersAndWriters(PersistenceManagerImpl.java:508)
	at org.infinispan.persistence.manager.PersistenceManagerImpl.start(PersistenceManagerImpl.java:114)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:183)
	at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
	at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
	at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
	at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
	at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216)
	at org.infinispan.CacheImpl.start(CacheImpl.java:668)
	at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:539)
	at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:502)
	at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:395)
	at org.infinispan.persistence.leveldb.config.ConfigurationTest.testConfigBuilder(ConfigurationTest.java:64)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:715)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:907)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1237)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
	at org.testng.TestRunner.privateRun(TestRunner.java:767)
	at org.testng.TestRunner.run(TestRunner.java:617)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
	at org.testng.SuiteRunner.access$000(SuiteRunner.java:37)
	at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368)
	at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:724)


For Solaris machine, the exception is completely the same only with the following difference:

Caused by: java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no leveldbjni32-1.7 in java.library.path, no leveldbjni-1.7 in java.library.path, no leveldbjni in java.library.path]


After research, I've found out the following issue which seems to be similar but for other platform:

https://github.com/fusesource/leveldbjni/issues/41

I tried the suggestion given here - added -Djava.io.tmpdir=. to my mvn command and after the UnsatisfiedLinkError disappeared, but I'm still getting the following exception:

org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.persistence.manager.PersistenceManagerImpl.start() on object of type PersistenceManagerImpl
	at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
	at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
	at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
	at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
	at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
	at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216)
	at org.infinispan.CacheImpl.start(CacheImpl.java:668)
	at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:539)
	at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:502)
	at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:395)
	at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:368)
	at org.infinispan.persistence.BaseStoreFunctionalTest.testPreloadStoredAsBinary(BaseStoreFunctionalTest.java:115)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:715)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:907)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1237)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
	at org.testng.TestRunner.privateRun(TestRunner.java:767)
	at org.testng.TestRunner.run(TestRunner.java:617)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
	at org.testng.SuiteRunner.access$000(SuiteRunner.java:37)
	at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368)
	at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
	at java.lang.Thread.run(Thread.java:662)
Caused by: org.infinispan.commons.CacheException: Unable to start cache loaders
	at org.infinispan.persistence.manager.PersistenceManagerImpl.start(PersistenceManagerImpl.java:156)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:183)
	... 32 more
Caused by: org.infinispan.commons.CacheConfigurationException: Unable to instantiate class org.fusesource.leveldbjni.JniDBFactory
	at org.infinispan.commons.util.Util.loadClass(Util.java:82)
	at org.infinispan.commons.util.Util.getInstance(Util.java:203)
	at org.infinispan.persistence.leveldb.LevelDBStore.newDbFactory(LevelDBStore.java:70)
	at org.infinispan.persistence.leveldb.LevelDBStore.init(LevelDBStore.java:53)
	at org.infinispan.persistence.manager.PersistenceManagerImpl.createLoadersAndWriters(PersistenceManagerImpl.java:508)
	at org.infinispan.persistence.manager.PersistenceManagerImpl.start(PersistenceManagerImpl.java:114)
	... 37 more
Caused by: java.lang.ClassNotFoundException: org.fusesource.leveldbjni.JniDBFactory
	at org.infinispan.commons.util.Util.loadClassStrict(Util.java:132)
	at org.infinispan.commons.util.Util.loadClass(Util.java:80)
	... 42 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.fusesource.leveldbjni.JniDBFactory
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:249)
	at org.infinispan.commons.util.Util.loadClassStrict(Util.java:119)
	... 43 more



I've run the maven command with -X option, and checked the classpath - it contains all necessary libraries - leveldbjni* .

Comment 2 Vitalii Chepeliuk 2014-04-07 08:37:30 UTC

*** Bug 1084910 has been marked as a duplicate of this bug. ***

Comment 3 Alan Field 2014-09-03 08:45:05 UTC

Removing from 6.3.1, since there is no fix right now

Comment 4 Martin Gencur 2014-11-07 07:38:20 UTC

Removing the flag because we did not mention it in last release either. We'll just mention where the LevelDB store is supported (on which platform).

Comment 5 Martin Gencur 2014-12-11 10:26:00 UTC

*** Bug 1171723 has been marked as a duplicate of this bug. ***

Comment 6 Martin Gencur 2015-01-14 13:54:51 UTC

*** Bug 1178701 has been marked as a duplicate of this bug. ***

Comment 7 Martin Gencur 2015-01-14 13:55:04 UTC

*** Bug 1178702 has been marked as a duplicate of this bug. ***

Comment 8 Vitalii Chepeliuk 2015-06-11 08:02:42 UTC

Following tests fail
test.integration.as.InfinispanStoreLevelDBIT.testCacheManager
server.test.cs.jdbc.string.StringBasedStorePassivationDisabledSinglenodeIT.testStoreDataWithHotRodClient
server.test.configs.ExampleConfigsIT.testLevelDBCacheStoreConfig
persistence.leveldb.JavaLevelDBCacheStoreTest.testStoreAndRemove
persistence.leveldb.LevelDBStoreTest.tearDown
persistence.leveldb.JniLevelDBCacheStoreTest.tearDown
persistence.leveldb.config.ConfigurationTest.testXmlConfig60
persistence.leveldb.LevelDBStoreTest.testStoreAndRemove
persistence.leveldb.JniLevelDBStoreFunctionalTest.testTwoCachesSameCacheStore
persistence.leveldb.JniLevelDBStoreFunctionalTest.testStoreByteArrays
persistence.leveldb.JniLevelDBStoreFunctionalTest.testRestoreAtomicMap
persistence.leveldb.JniLevelDBStoreFunctionalTest.testPreloadStoredAsBinary
persistence.leveldb.JniLevelDBStoreFunctionalTest.testPreloadAndExpiry
persistence.leveldb.JniLevelDBCacheStoreTest.setUp
server.test.cs.leveldb.LevelDBCacheStoreIT.testDataSurvivesRestart
server.test.cs.leveldb.LevelDBCacheStoreIT.testDataRetrievableViaLevelDbApi