Description of problem: The cache thread sometimes does not seem to return from calls to the wallaby api, or does not return very quickly. If cumin is shutting down during one of these calls it can cause the shutdown thread to timeout. Two cases that have been seen: A call to get data takes a long time 10-15 seconds or greater which is longer than the 5 second allowed shutdown time. The call to delBroker to clean up the broker object for wallaby operations does not return and does not raise an exception (or maybe just takes a really, really, long time to return) Version-Release number of selected component (if applicable): anything 2.2+ I believe How reproducible: 25-50% ? Steps to Reproduce: 1. Set up cumin and wallaby on a broker 2. Populate wallaby with tags and features 3. Start Cumin 4. Stop Cumin 5. Look for messages in web.log that look like this 5004 2013-01-15 17:45:48,845 DEBUG WallabyOperations: waiting for cache maintenance thread to exit <there may be other stuff in here> 5004 2013-01-15 17:45:53,845 INFO Shutdown thread timed out, exiting Trying to time the shutdown may help reproduce. If 'service cumin stop' is issued immediately after a lot message that says "WallabyOperations: refreshing XXX" it might increase reproducibility. trace -f /var/log/cumin-web | grep works nicely. Actual results: The shutdown thread sometimes times out Expected results: Ideally, the shutdown thread should not timeout Additional info:
Note, this has now been seen on both el5 and el6
MRG-Grid is in maintenance and only customer escalations will be considered. This issue can be reopened if a customer escalation associated with it occurs.