Bug 959616
| Summary: | Deadlock during clean shutdown of backup during activation | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [JBoss] JBoss Enterprise Application Platform 6 | Reporter: | Miroslav Novak <mnovak> | ||||
| Component: | HornetQ | Assignee: | Clebert Suconic <csuconic> | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | |||||
| Severity: | high | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 6.1.0 | CC: | anmiller, ataylor, csuconic, jdoyle, mlittle, myarboro, rsvoboda | ||||
| Target Milestone: | ER7 | Keywords: | Regression | ||||
| Target Release: | EAP 6.1.1 | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Known Issue | |||||
| Doc Text: |
An JBoss Enterprise Application Platform 6 server that is configured as a HornetQ backup server will fail to shutdown in the following situation:
* Message-Driven Beans are locally deployed on the server, and
* the shutdown command is performed during the server's activation.
In this scenario the shutdown will result in a deadlock that prevents the shutdown process from completing. Once deadlocked, the server must be forcibly terminated. On Red Hat Enterprise Linux 6 this can be done using the `kill -9` command.
To avoid this situation, only attempt to shutdown the server before or after activation, and not during the journal loading process.
|
Story Points: | --- | ||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2013-09-16 20:20:56 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
|
Description
Miroslav Novak
2013-05-04 07:09:46 UTC
Created attachment 743465 [details]
threadump.txt
Not sure if this is a blocker for release (knowing where we are), but I want the triage team reviews this one. Is there any user visible consequence of this deadlock? When using a backup and issuing a clean shutdown only. And even so you can kill -9 I don't think this should be a blocker because: - it won't affect production. - it affect only a few users - for the few using this feature and eventually issuing a clean shutdown can do a kill -3 And Mainly: - that's a risk change to be done under pressure. I don't think we should do it now. Customer will need to use "kill -9 ..." to kill the server where HQ is configured as backup when this is hit. So far I saw this dead lock only on backup and with EAP 6.1.0.ER6. But it's hard to say whether it's regression. This issue is problematic for our failover tests. I'll have to kill server when dead lock is hit during clean shutdown (tear down phase) so the test won't hang. It's not a test blocker but would help to have it fixed. I agree with Clebert, but this should be fixed immediately after 6.1 goes out. I agree, not a blocker. It seems this issue will only happen if you have MDBs on the backup. Most backup users do it remotely. Maybe for the tests you could do something like removing the pooled connection from the standalone. Documented as Known Issue for EAP 6.1.0 I am fixing this issue on master and 2.3.x. It turns out to be an easy fix, and I have replicated it with a byteman test. If you guys want to I can make a new release with this fix here. After investigating this issue, this will only happen if you shutdown the server while activating. a workaround for QE would be to sleep a few seconds before shutting down the server. A Workaround for customers will be not shut down during activation. (i.e... it would be a really rare event, shutdown during activation). This will be fixed on next release just in case, but it's definitely not a big deal. BZ is in incorrect state. Fix for this issue is present in EAP 6.1.1.ER7(HQ 2.3.5.Final) Verified in EAP 6.1.1.ER7. I can no longer hit the issue. Great work, Clebert! |