Hide Forgot
Description of problem: Environment: BRMS 6.1.0, 2 node domain mode cluster, 1 helix/zookeeper. If you delete a project in one node of the cluster, you get "org.eclipse.jgit.api.errors.ConcurrentRefUpdateException: Could not lock HEAD. RefUpdate return code was: LOCK_FAILURE". ======= [Server:server-one] 16:42:02,517 ERROR [org.jboss.as.ejb3.invocation] (Thread-92) JBAS014134: EJB Invocation failed on component IOWatchServiceExecutorImpl for method public abstract void org.uberfire.backend.server.io.watch.IOWatchServiceExecutor.execute(org.uberfire.java.nio.file.WatchKey,org.uberfire.backend.server.util.Filter): javax.ejb.EJBException: java.lang.RuntimeException: org.eclipse.jgit.api.errors.ConcurrentRefUpdateException: Could not lock HEAD. RefUpdate return code was: LOCK_FAILURE [Server:server-one] at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleExceptionInNoTx(CMTTxInterceptor.java:217) [jboss-as-ejb3-7.5.0.Final-redhat-21.jar:7.5.0.Final-redhat-21] ... [Server:server-one] at org.uberfire.backend.server.io.watch.IOWatchServiceExecutor$$$view2.execute(Unknown Source) [uberfire-backend-server-0.5.3.Final-redhat-1.jar:0.5.3.Final-redhat-1] [Server:server-one] at org.uberfire.backend.server.io.watch.AbstractIOWatchService$2.execute(AbstractIOWatchService.java:145) [uberfire-backend-server-0.5.3.Final-redhat-1.jar:0.5.3.Final-redhat-1] [Server:server-one] at org.uberfire.backend.server.io.watch.AbstractIOWatchService$1.run(AbstractIOWatchService.java:86) [uberfire-backend-server-0.5.3.Final-redhat-1.jar:0.5.3.Final-redhat-1] [Server:server-one] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_67] [Server:server-one] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_67] [Server:server-one] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_67] [Server:server-one] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_67] [Server:server-one] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_67] [Server:server-one] Caused by: java.lang.RuntimeException: org.eclipse.jgit.api.errors.ConcurrentRefUpdateException: Could not lock HEAD. RefUpdate return code was: LOCK_FAILURE [Server:server-one] at org.uberfire.java.nio.fs.jgit.util.JGitUtil.commit(JGitUtil.java:608) [uberfire-nio2-jgit-0.5.3.Final-redhat-1.jar:0.5.3.Final-redhat-1] [Server:server-one] at org.uberfire.java.nio.fs.jgit.util.JGitUtil.commit(JGitUtil.java:506) [uberfire-nio2-jgit-0.5.3.Final-redhat-1.jar:0.5.3.Final-redhat-1] [Server:server-one] at org.uberfire.java.nio.fs.jgit.JGitFileSystemProvider.commit(JGitFileSystemProvider.java:1853) [uberfire-nio2-jgit-0.5.3.Final-redhat-1.jar:0.5.3.Final-redhat-1] [Server:server-one] at org.uberfire.java.nio.fs.jgit.JGitFileSystemProvider.access$600(JGitFileSystemProvider.java:147) [uberfire-nio2-jgit-0.5.3.Final-redhat-1.jar:0.5.3.Final-redhat-1] [Server:server-one] at org.uberfire.java.nio.fs.jgit.JGitFileSystemProvider$4.close(JGitFileSystemProvider.java:836) [uberfire-nio2-jgit-0.5.3.Final-redhat-1.jar:0.5.3.Final-redhat-1] [Server:server-one] at org.uberfire.io.impl.AbstractIOService.write(AbstractIOService.java:609) [uberfire-io-0.5.3.Final-redhat-1.jar:0.5.3.Final-redhat-1] [Server:server-one] at org.uberfire.io.impl.AbstractIOService.write(AbstractIOService.java:498) [uberfire-io-0.5.3.Final-redhat-1.jar:0.5.3.Final-redhat-1] [Server:server-one] at org.uberfire.io.impl.AbstractIOService.write(AbstractIOService.java:506) [uberfire-io-0.5.3.Final-redhat-1.jar:0.5.3.Final-redhat-1] [Server:server-one] at org.uberfire.io.impl.cluster.IOServiceClusterImpl.write(IOServiceClusterImpl.java:775) [uberfire-io-0.5.3.Final-redhat-1.jar:0.5.3.Final-redhat-1] [Server:server-one] at org.kie.workbench.common.screens.explorer.backend.server.ExplorerServiceImpl.cleanup(ExplorerServiceImpl.java:463) [kie-wb-common-project-explorer-backend-6.2.0.Final-redhat-4.jar:6.2.0.Final-redhat-4] [Server:server-one] at org.kie.workbench.common.screens.explorer.backend.server.ExplorerServiceImpl.onProjectDelete(ExplorerServiceImpl.java:449) [kie-wb-common-project-explorer-backend-6.2.0.Final-redhat-4.jar:6.2.0.Final-redhat-4] ... [Server:server-one] at org.kie.workbench.common.screens.explorer.backend.server.ExplorerServiceImpl$Proxy$_$$_WeldClientProxy.onProjectDelete(ExplorerServiceImpl$Proxy$_$$_WeldClientProxy.java) [kie-wb-common-project-explorer-backend-6.2.0.Final-redhat-4.jar:6.2.0.Final-redhat-4] ... [Server:server-one] at org.guvnor.common.services.project.backend.server.PathObserverService.fireDeleteEvent(PathObserverService.java:63) [guvnor-project-backend-6.2.0.Final-redhat-4.jar:6.2.0.Final-redhat-4] [Server:server-one] at org.guvnor.common.services.project.backend.server.PathObserverService.onBatchResourceChanges(PathObserverService.java:46) [guvnor-project-backend-6.2.0.Final-redhat-4.jar:6.2.0.Final-redhat-4] [Server:server-one] at org.guvnor.common.services.project.backend.server.PathObserverService$Proxy$_$$_WeldClientProxy.onBatchResourceChanges(PathObserverService$Proxy$_$$_WeldClientProxy.java) [guvnor-project-backend-6.2.0.Final-redhat-4.jar:6.2.0.Final-redhat-4] ... [Server:server-one] at org.uberfire.backend.server.io.watch.IOWatchServiceExecutorImpl.execute(IOWatchServiceExecutorImpl.java:110) [uberfire-backend-server-0.5.3.Final-redhat-1.jar:0.5.3.Final-redhat-1] ... [Server:server-one] Caused by: org.eclipse.jgit.api.errors.ConcurrentRefUpdateException: Could not lock HEAD. RefUpdate return code was: LOCK_FAILURE [Server:server-one] at org.uberfire.java.nio.fs.jgit.util.JGitUtil.commit(JGitUtil.java:593) [uberfire-nio2-jgit-0.5.3.Final-redhat-1.jar:0.5.3.Final-redhat-1] [Server:server-one] ... 122 more ======== Because of the combination with BZ1323572, AsyncWatchService becomes unfunctional and servers get various issues. How reproducible: Always Steps to Reproduce: 1. Start a BRMS cluster 2. Login to business-central 3. Create a project 4. Delete a project Actual results: "org.eclipse.jgit.api.errors.ConcurrentRefUpdateException: Could not lock HEAD. RefUpdate return code was: LOCK_FAILURE" is thrown Expected results: No exception.
This LOCK_FAILURE is caused by committing "/explorer/user.nav" of "upstream/brmsAdmin-uf-user". So the problem is that such a remote branch is retrieved in ExplorerServiceImpl.cleanup() https://github.com/droolsjbpm/kie-wb-common/blob/6.1.x/kie-wb-common-screens/kie-wb-common-project-explorer/kie-wb-common-project-explorer-backend/src/main/java/org/kie/workbench/common/screens/explorer/backend/server/ExplorerServiceImpl.java#L959 Remote branches are retrieved because JGitFileSystemProvider.rescanForExistingRepositories() instantiates JGitFileSystem with ListMode.ALL. https://github.com/uberfire/uberfire/blob/0.5.x/uberfire-nio2-backport/uberfire-nio2-impls/uberfire-nio2-jgit/src/main/java/org/uberfire/java/nio/fs/jgit/JGitFileSystemProvider.java#L397 This was accidentally fixed by another BZ. https://bugzilla.redhat.com/show_bug.cgi?id=1269333 https://github.com/uberfire/uberfire/commit/a72a56efb7242749526cbf5ce3c5a03c86f66a39 So this issue no longer exists in BRMS 6.2.0. I filed this BZ for record. I'm closing this BZ.