Description of problem: If failover/failback happens multiple times then there are lots of log messages: ... 09:24:24,333 INFO [org.hornetq.jms.server] (HQ119000: Activation for server HornetQServerImpl::serverUUID=2533423e-b745-11e4-b5d4-cb40437956fb) HQ121003: JMS Server Manager Running cached command for destroyConnectionFactory for RemoteConnectionFactory ... 09:24:24,334 INFO [org.hornetq.jms.server] (HQ119000: Activation for server HornetQServerImpl::serverUUID=2533423e-b745-11e4-b5d4-cb40437956fb) HQ121003: JMS Server Manager Running cached command for destroyConnectionFactory for RemoteConnectionFactory in backup server log. Number of those log messages increases with each failover. Steps to Reproduce (dedicated topology with shared store): Start the live. Start the backup. Kill-9 the live. Backup becomes live. Start the live. Live becomes live, backup becomes backup. Kill-9 the live. Backup becomes live again. ... Actual results: Cached commands are never cleared and only added to list after failback. Expected results: Clear cached commands after start of backup server.
This is a consequence of https://issues.jboss.org/browse/HORNETQ-1047. To improve user experience, the log output has been updated to explain that this is expected behaviour. 2.3.25.x PR: https://github.com/hornetq/hornetq/pull/2030 2.4.x PR: https://github.com/hornetq/hornetq/pull/2031 Upstream PR: https://github.com/apache/activemq-artemis/pull/59
Ryan Emerson <remerson> updated the status of jira HORNETQ-1484 to Coding In Progress
2.3.x PR: https://github.com/hornetq/hornetq/pull/2034
2.3.25.x Commit: https://github.com/hornetq/hornetq/commit/1801575166011d6e75823a403e61223b82d358cf
Ryan Emerson <remerson> updated the status of jira HORNETQ-1484 to Resolved
Problem is, that this log is stacking and stacking and stacking. After few failovers it can be really long and user will be spammed with useless information. So this solution is not fix.
Ondřej would an acceptable solution be for the logging level to be changed to DEBUG with the new message explaining that the repeated commands are expected? This is how the log is now handled upstream.
I think, that it would be better if that "cache" which holds this command history is cleaned after failback.
This is not possible because of https://issues.jboss.org/browse/HORNETQ-1047. Clearing the cache will cause a regression.
And what about making this cache history as set. If you avoid duplications, problem will be fixed without regression.
I don't believe avoiding duplication is possible in this case because it is necessary for the backup server to execute the cached commands in the same order that they were originally executed, so that the state of the failed live server can be reproduced. The commands stored in the cache can create/remove topics, therefore the order of execution is important. For example, if we did not respect duplicate commands then the following workflow would not be reproducible: JMSServerManager.createTopic("example" ...) JMSServerManager.removeTopicFromJNDI("example") JMSServerManager.createTopic("example" ...) // Identical topic to above
Ok, in this case is acceptable to change logging level to DEBUG.
Ryan Emerson <remerson> updated the status of jira HORNETQ-1484 to Reopened
Verified with 6.4.5.CP.CR1
Retroactively bulk-closing issues from released EAP 6.4 cumulative patches.