The tests fail in the 'infinispan-cachestore-leveldb' module and because of this the install step doesn't complete and the features.xml file is not pushed to the local repo.

The <hr> comes from a "not found" reply from one of the remote maven repos configured to resolve artifacts inside Karaf.

Upgrading the leveldbjni version to 1.13-redhat.002 + clean install in infinispan-cachestore-leveldb solved the problem.

The OOME and other errors are probably side-effects of the feature not being installed. Please open a new issue if they still occur with CR3 (which contains the version fix).

See the leveldbjni issue for details.

*** This bug has been marked as a duplicate of bug 1118785 ***

Ion Savin <isavin> updated the status of jira ISPN-4500 to Closed

Moving target version to JDG 6.4.0

Just like before the OSGi test failure is a side-effect of the tests in LevelDB failing -> the bundle/feature is not installed and the OSGi tests can't load them.

The failure in the LevelDB module is:

Test suite progress: tests succeeded: 28, failed: 0, skipped: 0.
org.infinispan.commons.CacheConfigurationException: Unable to instantiate class org.fusesource.leveldbjni.JniDBFactory
	at org.infinispan.commons.util.Util.loadClass(Util.java:85)
	at org.infinispan.commons.util.Util.getInstance(Util.java:223)
	at org.infinispan.persistence.leveldb.LevelDBStore.newDbFactory(LevelDBStore.java:78)
	at org.infinispan.persistence.leveldb.LevelDBStore.init(LevelDBStore.java:60)
	at org.infinispan.persistence.leveldb.LevelDBStoreTest.createStore(LevelDBStoreTest.java:83)
	at org.infinispan.persistence.BaseStoreTest.setUp(BaseStoreTest.java:78)
	at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
	at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:564)
	at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:213)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:653)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
	at org.testng.TestRunner.privateRun(TestRunner.java:767)
	at org.testng.TestRunner.run(TestRunner.java:617)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
	at org.testng.SuiteRunner.access$000(SuiteRunner.java:37)
	at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368)
	at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.ClassNotFoundException: Could not load requested class : org.fusesource.leveldbjni.JniDBFactory
	at org.infinispan.commons.util.OsgiClassLoader.findClass(OsgiClassLoader.java:89)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:270)
	at org.infinispan.commons.util.Util.loadClassStrict(Util.java:139)
	at org.infinispan.commons.util.Util.loadClass(Util.java:83)
	... 26 more

The OsgiClassLoader appears in the exception because it appears to be the last classloader from the list which failed to load that class.

Locally the test for the leveldb module run OK on the branch mentioned.

Please check the environment setup. On a new run there's this failure which could and it could be the cause for the tests failing in cachestore-leveldb and then integrationtests/osgi:

Caused by: java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no leveldbjni32-1.13-redhat.002 in java.library.path, no leveldbjni-1.13-redhat.002 in java.library.path, no leveldbjni in java.library.path]

The build is:
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/JDG/view/FUNC/job/edg-60-ispn-testsuite-solaris/211/USE_JDK=JDK7,jdk=java17_default,label=sol11_sparc64/

Leveldb not running on solaris is an old bug https://bugzilla.redhat.com/show_bug.cgi?id=1020411. So maybe we could just disable them and see if the osgi tests pass.

Jakub, what is the current status of this issue?

I can still see the issue in JDG_6.4.ER7, e.g. here:

https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/JDG/view/FUNC/job/edg-60-ispn-testsuite-windows/334/USE_JDK=JDK7,jdk=java17_default,label_exp=w2k8r2%20&&%20x86_64/testReport/junit/org.infinispan.it.osgi.features/OSGiKarafFeaturesTest/testCleanInstall

LevelDB modules was turned off in this run. Test just failed, on OOM.

PR: https://github.com/infinispan/jdg/pull/391

See the PR for details on how to run tests.

Looking at the CI test failures for the last mentioned failure I can see failures in other modules besides the OSGi one. For example:

https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/JDG/view/FUNC/job/edg-60-ispn-testsuite-windows/334/USE_JDK=JDK7,jdk=java17_default,label_exp=w2k8r2%20&&%20x86_64/testReport/junit/org.infinispan.persistence.remote/RemoteStoreRawValuesTest/testLoadAndStoreMarshalledValues/

If the tests are running using the "ignore test failure" flag the build won't fail when there are test failures in other modules but the install phase for that module won't execute either.

Because the install phase doesn't execute the bundles and feature files for those modules won't be present in the local maven repository and as a result the OSGi tests will fail but they are not at fault.

The solution is to fix the failures in the other modules.

Sorry for ignorance, I'm not very familiar with OSGi ... but shouldn't maven download missing artifacts from remote repository in this case?

I think my PR will ensure that we will get a better error message when the artifacts are not in the local repo nor in any of the remote ones.

so if I get it right, as appropriate maven repo is part of the release, IMHO we can fix it in our test suite scripts by skipping maven install and instead of it downloading the maven repo from release, right?

PR integrated into jdg-6.4.x